Make sure to take copious notes on date, time and what happened. I also suggest bringing a tape recorder every time you have a conversation with the abuser through your phone or a small recording device you can carry in your purse.
Then when you have proof, contact an attorney. The more proof you have, the better your case.
Try not to speak to them and email all correspondences and keep diligent records.
I was subject to harassment as a single female in my building as well, and I know what you are going through.