Define "not so great."
If "family friendly" is high up on your priority list, pass on the house and keep looking.
If "not so great" means it's been going down hill for the past few years, pass on the house and keep looking.
However, I've known plenty of people who've bought homes they loved in areas that they didn't. In those cases, they really liked the house, often they didn't have kids, the area itself was showing indications of improving, the home was in an otherwise good location (close to work, shopping, entertainment), etc.
Generally, stay away from homes in declining neighborhoods. Otherwise, weigh your priorities (crime, school system, proximity to work, proximity to entertainment, traits of the home itself, affordability) and make your decision.
One other point: Unless you buy into an improving neighborhood, it's likely that when you decide to sell there will be the same reluctance from some buyers that you're currently experiencing.
Hope that helps.