No matter what you do, there will always be someone with something negative to say. I no longer live in Raleigh, but during 18 years I was there I lived downtown (in Mordecai, walking distance to Oakwood) and in NW Raleigh near Millbrook and Glenwood. I noticed your current ZIP Code is 27612.
Is being downtown important to you? One of the advantages to living close to downtown is that I could be ANYWHERE in 15 minutes. If your work, and free time activities revolve around downtown, then you may be happier living in one of those neighborhoods, even if you need to install a security system.
NW Raleigh does offer plenty of convenience for those working in RTP OR downtown, and you will get more house for your money, so instead of the "small house" you are looking at, you might end up with a garage or a pool in another location. It all comes down to what makes YOU happy.
Best of luck!
Keller Williams Realty Professionals
Fort Lauderdale, FL 33308