Loan officers do not determine underwriting guidelines, they follow them.
As it pertains to fees owed, it depends upon the agreement you entered. Ask for a copy of your agreement and review the terms. He may be calling a bluff. Even if the agreement states you owe the money, it may or may not be enforceable. Check with an attorney once you have a copy of the agreement.