Is this an FHA loan?
Are they buying down the mortgage rate; 6.5% to 6% is two points (or 2%).
Call 1-800 - WHY - DREW for details.
First and foremost, do some comparison shopping and find a broker who is looking out for your interests.
Powder House Realty Group
You can also ask for some discounts like recording fees (you could do it yourself), application fees could be reduced, you could find your own title company that will charge you less, ask for free credit report, close at the end of a month, etc.
I would take the GFE he gave you and use it to shop your loan, rate and terms. If you'd like, you can email me your credit score, loan parameters (purchase price, loan amount, loan type) and I will email you back another Godd Faith Estimate. I don't even need to see the one he gave you. I would prefer you use it to help get yourself a lower rate and/or fees. If your broker doesn't budge, then you know he's only concerned about profit and not getting you the best loan possible.
Feel free to contact me. I promise you I will not contact you further unless you request it.
In the interest of full disclosure I am not a mortgage broker, nor do I play one on TV, but that fee seems high to me.