I have such a listing now in the Clinton Hill section. The house is every bit worth it, but the price is at least 20% beyond its most likely sale price.
Part of the pricing of the property is conformity. How well does the property conform to the surrounding inventory. You are right in saying that most who want an upscale home will look in a more upscale community. Conformity is related very closely to situs, i.e., "location, location, location." A 2M contemporary in even a $2M Victorian community will lose market value. However, consider this. While your neighbor may lose some value because of the surrounding stock, the surrounding stock is gaining in value because of your neighbor!
I just looked at the clock and saw the time. You're lucky. I can be really long-winded! I love what I do!... more