4.6% sounds like a lower than average rate. What you may be finding out the hard way is that just because a lender is offering the best rate, there is much more that goes into picking a lender than rate...primarily their ability to fund the loan as promised.
My advice to you is to set rate aside and speak to a reputable lender with a local presence. Get a referral from your agent or friends if you need to.
From the sounds of it things shouldn't be this difficult.
Best of luck