There are lots of night life in that neighborhood so there is that but as for the the proximity to sixth, I think that if you have lived in the city for a while and you have "mentality" you should be fine there. Man, you would be surprised at how many people do not know the little things of living in the city like taking everything out of your car when you park stuff like that. If you know those little things I would not let it get in the way of a great new place to live.
I used to have an office on Mission between 5th and 6th. I parked my car on Howard and 6th, so every day I used to walk the "most interesting" part of the 6th between Howard and Mission. Even in the morning you could see street scenes worth the camera. Since then, the neighborhood is improving. There is an effort underway to revitalize this part of the City. In my opinion it is getting better and it will get better, however it is a long process as there is plenty of "hotels" over there. At the same time you definitely have to very careful venturing on the street at night or late evening. Most of the time - it will be nothing else than annoying "street scenes". However every now and then someone might urgently need the money to feed the habit. But I am sure you know about it, as you lived on Mission (which part). You walk half a block and you are in "different" City. Hope it helps. It is a great part of the City as you are close to "anywhere".
If you found an amazing place and already work down the street, I'd say take the plunge and experience the continued transformation of that area of the city firsthand.