Oh, good grief, yes, GET OUT!!!! If you have done nothing that's cost you any money so far, go, run in the opposite direction!!! It's never worth it to stick with somebody that bothers you that much. This transaction is too important and you need to build your team with people who you KNOW are in your corner. If you don't want to even TALK to that person, it's hard to do all of the things you have to do to buy your house. Same goes for your agent.
And, this should be a joyful experience!!! Discord like that only spoils it~