The biggest concern is the crazy homeless people that spill over from GG park. I have never heard of any real problems with them, but they are often messy and generally unpleasant. Nothing like in the Mission, but not as tame as less urban areas. Whenever anybody would set up camp in a particular area, SFPD was pretty good about moving them along.
With the combination of the SFPD and the UCSF PD constantly in the area, it always feels very safe, pretty much any time of day and at any location.