If this Realtor has helped you sell or find a home this payment is their bread and butter. Many people do not understand that Realtor's do all the work upfront to get a property to closing day in order to get paid. So, think this through and what's fair is fair.
My greatest suggestion is go to the broker of record in the firm and discuss your situation for possible quick resolution or better yet consult a real estate attorney.
(Personally, as an agent, I never take a fee from a buyer, but that 's the way I choose to roll. My reputation and referrals from my clients means more than a one time fee, but that's me)
As mentioned earlier, if you agreed to pay a buyers agent fee then yes you need to pay the contractually agreed upon amount.
If you purchase a property through this Realtor then it would pay that fee, I'm assuming.
So if you decided to abandon this Realtor and go with a new one then you may be obligated to pay him a fee. I'm not an attorney so of course seek legal advice if you're unclear about what your agreement means.
If its written in the contract, then you'll have to pay it. If its not written in the contract that you either agreed to pay it, or if it was written you would and the agent waived it, then you won't.
However, if nothing is written about the Buyer's agent fee anywhere, no Buyer's Agency Agreement, not in the sales contract, etc., then the agent is going to have a hard time getting you to pay it. Also, the buyer's agent usually doesn't have the power to prevent the closing, certainly not over a fee. The agent's recourse would usually be to sue you. You may want to call their broker.
You didn't say why the closing date had come and gone. Are you still in escrow? What else is holding up the closing? The only things that normally hold up closing are funding on the buyer's side, title issues on the seller's side, on a short sale the seller's lender having final review of the HUD1, etc.
Anyway, check everything you signed to see if its written down and addressed, one way or the other. And if its not, start with the agent's broker then proceed to a real estate attorney.