Having lived in both Alexandra and Annapolis, I will tell you EMPHATICALLY that Annapolis is a very lovely place to live, and prices are reasonable. (Old Navy guy...with friends who were assigned to McNair - back when they still had the little golf course on the grounds. Nice, but gone.) The drive to Fort McNair isn't bad if done early/late to avoid the lane reduction at 50&495.
The long walks around Annapolis, or the safe/secure/serene walks around the Naval Academy will more than make up for it. While there is much to like about Alexandra VA, Annapolis just won "Prettiest Town in America." Believe it. Also note: Our County (Anne Arundal) has *the most* waterfront of any county in the country. The green areas, bike paths, and "small town feel" will impress. Can you tell I like it?
Right now, there are 134 rentals in Annapolis. (Probably +/- by the time you read this.)
Make it easy on yourself, and skip over to http://www.realtorcurthess.com/rentals/
to find the one you will love. You'll find photos, maps, and anything you need to help you decide.
Wishing you the best...I grew up in NJ, exit-0, back when "regular people" could still afford to buy a home there.