@Marie You are being way too sensationalist! I've lived in the Bayview for 5 years+ now and its not THAT bad or scary. You're not going to see anything you couldn't see in other parts of SF.
No, it's not Noe Valley but can you afford a house with a backyard there? I would advise trying to buy in the Silver Terrace which is the area west of 3rd St., south of Palou(?), rather than in "the flats" east of 3rd.
The biggest drawbacks are the lack of local stores/restaurants that you'd actually want to go to but as another poster said this is (slowly) changing. Latest addition is All Good Pizza right opposite the fabulous Flora Grubb Garden Center and Fresh&Easy market.