Depends on your current loan terms, how long you've had your current loan, what rate/fees you can lock in currently and your objectives (lower payment, lower term, both, etc.)
Most of my clients who had high 4's on a 30 year fixed have refinanced to high 3's or low 4's with a NO COST and NO FEE refinance... it usually makes sense if you drop the rate at least 0.50%. The downside is resetting the term back to a 30 year and possibly paying more over the term, but if you play your cards right you can always pay your old higher payment and shorten the term from where you were originally. On the other hand, some folks don't mind resetting the term if lowering payment is more important.
Even better idea for some folks who don't like the idea of going back to a 30 year term is to drop the rate AND pick up a 25 or 20 year term. Depending on your current loan, this can lower your payment AND allow you to shave off years. But if you get the payment close to your old payment it's still makes sense especially if you dropped the rate for free.
Feel free to call me if you wish, it should take about 5 minutes to break it down for you. No obligation, I really enjoy crunching numbers and exploring different options. :)