If it's going to cost you $4000 to refinance into a lower rate, and knock $100 per month off your payment - you aren't going to "actually" see any of those savings and recoup the money you spent on that fee for over 3 years. If you are going to be staying in that house for the long-term, i.e. 10-15 years, then you'd be saving a fortune in interest and it makes sense. If you are uncertain about your future, and may sell the house and move in 3 years - then you would be actually losing money by refinancing in the long run and it makes no sense.
You'll want to shop around for the lower rates, and lowest associated "fees" before making a decision. Once you have an idea of the true cost, you'll want to crunch the numbers and make sure that the amount of reduction in interest, justifies any associated fees, in correlation with how long you plan to remain in your property.
Hope that helps!
In addition, our market has been so volatile throughout the course of the past six years, those who may have the equity may just still be considering when would be the best time to make a move or which move to make regarding their real assets, whether selling or refinancing.
Ignorance is a big factor. I had several experiences that opened my eyes to this issue...
Aside from lack of knowledge, reasons might be different, to mention a few...
1.Property owner knows for sure he will be selling the property in a very near future, so doesn't need the headache of tons of paperwork and refinancing process as he will be out of the house in several months anyway.
2. Property owner is an elderly person and can't manage the process
3. Property owner has a little or no knowledge about the process and doesn't want to or is scared to take action
4. Property owner heard stories from friends and neighbors (stories which are not necessary true) and based on them makes wrong assumptions and doesn't take action
5. Property owner is in financial distress and cant afford closing costs on the new loan