The loan officer admitted that it was a mistake on the phone. He preferred to speak over the matter on the phone. If he replied in an email, it could be used as evidence of his wrong doing.
In this market, with little to no inventory, it's rare that a buyer asks, or a seller agrees, to pay closing costs related to a buyer's loan.
But no matter what the situation, it seems that the lender did not charge you correctly for your loan. If they will not rectify it, you can always sue them. You can go to small claims court, or sign up for something like prepaid legal and have an attorney write letters to see if they will correct their mistake. I'd start with a phone calle to management of your lender. This is most likely a misunderstanding, or error on the part of your loan officer.
Of course you can REQUEST a refund from your mortgage lender. That does not assure you that you will get one. It looks like the one who benefited from the lenders mistake was the seller, not the lender, so you might think about asking the seller to compensate you on that. My comments are merely musings in a public forum for entertainment, and are not meant to be taken by jujal9 as professional advice. The people that should be advising you on this are: 1. your real estate agent 2. your mortgage broker or loan officer 3. your escrow officer (you have already tried that) 4. Advice of a real estate attorney.
The reason I suggested persons 1 through 3. is that these people have already been paid, and they still owe follow up service to you. An attorney will charge new fees to you for her advice.