Yes, you have no agreement with an agent-You are Selling your home on your own.
IT is your right to do that, If the Buyer has signed an agreement to pay their broker than it is the responsibility of the buyer to pay, OR if there is no agreement to pay than it is presumed that the agent will procure a commission as being offered in the MLS for instance, 2.5 or 3%.
If you are not listed on the MLS than there is no commission offered.
There for the agent must find the buyer's a property that is listed on the MLS and is offering a commission in order to get paid.
Did the buyer find your home on their own? If so, you and the buyer are procuring cause not the buyer's agent.
Did you sign a one party listing contract? If so, then that may change the situation a bit.