You will hear various opinions of what has been "hard" for this market. In comparison to what the media has put out there nationally, Madison has faired well. In comparison to another part of the state, that is more area by area, even neighborhood by neighborhood. The best thing to do is know how it has affected you and your situation.
If you are in need of selling, have a REALTOR complete an analysis for your property to know the current value in the market place. At that time, also assess how your area and neighborhood has been affected. If you are looking at buying, see what the prices in the neighborhood have been doing.
Hope that helps. I would be happy to help you and answer any questions that you may have.