Late to the game, but the first thing i would do is make sure you are working with a correct value. While Zillow and other online AVM's are a decent tool to get a ballpark value, they can often be way off. This is especially true if your property is located in a niche market like Historic Frederick or similar market.
It may be that you have enough equity to refinance the home without tapping your primary residence, or it may be that you have even less equity than you thought. Either way, you want to know this before you spend any time or money working on a refi solution.
My advice is to have a few Realtors who sell a lot of property in the location of your home give you a market analysis. (Of course, if this is YOUR market, you will know if the Zillow estimate is accurate). Once you know the true value, you will be better postioned to make an informed decision as to whether it's worth the cost of refinancing. My guess is that you will only come down 1 to 1.25% without paying a lot of fees. That's not going to save much on that size loan, so it may just be best to stay with what you have.
Happy to discuss options. I've been doing this for 25 years. Pop me an email if you like at firstname.lastname@example.org Thanks and good luck!... more