This area used to be a lot worse. It's cleaned up a bit and over the past 7-10 years. I bought my first home on the 400 block of Wallace Avenue in 2004. We saw / heard more activity because of being so close to Benicia street and there used to be a lot more drug issues in the particular area. Not there anymore. Literally, two months after I bought the house someone was shot and killed on the corner of Starr and Wallace next to the middle school. Other than that there was no other crime the or incident that was a concern for use while we lived there. The shooting was a one off incident and it was likely gang related and outside of their territory. My kids have played soccer for the Pal league and use the field on Wallace there for over 12 years.
It's not the best or greatest of neighborhoods, but it's definitely NOT the CREST! I have no regrets about buying our first home in that area and I would do it again. All our neighbors were great and we all got to know one another and watched out for each other.
You will be fine and hopefully learn to like the community as we have over the years and I am originally from San Rafael, Marin County.