If interest were the only "cost" in borrowing money, life would be a lot simpler. There would be no application fees, title insurance premiums or "points" to consider, and whenever rates fell by some significant fraction, you'd run out and refi.
Life is more complicated than that, especially with the array of creative mortgage products that didn't exist a generation ago. A reduced rate is great, but if locking it in will cost you $4,000, is it worth it? If not, what is the better rate "worth" to you?
The list of one-time costs is finite, and it's good to have at least a basic understanding of each one. These ones, for example, are pretty common.
Title insurance: Charged by title companies to guarantee your title is free of liens and encumbrances.
Appraisals: Charged by appraisers, required by lenders, to determine the value of the home to be financed.
Broker commissions: Charged by a broker, if you elect to use one, for his trouble in finding you the best rate.
Credit applications fees, loan origination fees, paperwork processing fees, etc
Another kind of cost is extremely important but often optional. "Discount points" let you buy down your interest rate to something lower. A point is usually worth one percent of the loan amount.
Aside from one-time costs, certain ones, if you incur them, are ongoing. Premiums for mortgage insurance, if it's required by the lender, may boost your monthly payments by a few percentage points for the life of the loan.
Much of the science of considering a refi is driven by the concept of "breaking even." How many months will it take you, for example, to earn back those $3,600 in fees and costs if your monthly rate is lowered by $100? The answer, of course, is 36. You would thus "break even" after three years.
Again, life would be simpler if someone merely handed you a rate sheet with a bottom line cost of $3,600, but it doesn't work that way. So many costs are percentages, based on the amount you'll borrow. One point on a $250,000 loan costs more than one point on a $200,000 loan.
Typically, the break-even question is expressed like this: What interest rate would justify the refi? The best, most exacting way to answer this question is to use an online tool such as Trulia's mortgage calculator.