This loan officer is not only controlling but misleading you - you should go over his head to the firm manager and ask for your files to be given to another lender. If you dont get a good response, leave them and ask for any fees you paid to be refunded, because you will just have to pay those fees again to another lender.
If you need a referral to excellent lenders here in Atlanta, I can provide some.
You say you "chose his agent" - if you did not sign a Buyer Brokerage Agreeement with the agent, you might want to keep looking. You should take an active role in making sure you have the most qualified buyers agent you can find to assist you - ask them what they can do for you that other agents can't - that narrows the field quickly!
After interviewing a few agents, if you can't find one with standout qualifications and services, the choice comes down to who you feel most comfortable with.
Broker/Owner - Realtor
Professional Buyers Broker
New Home Construction Expert
ICC Code Certified Building Inspection Expert
â€œLuxury Homes, Investments, Property Management & Rehabâ€¦ Incomparable Client Representation & Due-Diligenceâ€