Does the buyer have a Realtor representing them?
If not, your Realtor could explain that in order for a smooth process, based on your last experience, he needs to have his own representation. Then your Realtor can coordinate with the buyer's Realtor and keep the sides apart.
A note on descrepancies in assessor's records. It is very common to have differences in both living area and lot size in the assessor's records. I am curious, how large was the difference? Was it "material"? Sometimes buyers will pick at anything to get out of a transaction when they are simply scared and confused. Sometimes the quality of Realtor representing the buyer can make a big difference. If the buyer's Realtor says " you know it is common for there to be a difference in the square footage" and then explains why, then the buyer doesn't get upset. Or, if there are other concerns, the Realtor can help assure the buyers that they are making the right choice. It is a big deal to buy the most expensive purchase you may ever make, it is stressful.
It may be that now, upon time to reflect, they have in fact decided that yes, this is the home for them.
On top of that, the question might arise as to why you're not selling. Suppose, for instance, the buyer is handicapped. Or is black. Or is Jewish. Or has a child. You can't discriminate on the basis of race, religion, or handicap. But you certainly can discriminate because someone is nasty, or simply because you don't like him/her. Or because he/she drives a Toyota. You get the idea. The problem is: What if the buyer belongs to one of those protected classes, but that's not the reason you're refusing to sell? You say it's because he was nasty and lied; he says it's because he's a member of the protected class.
Your agent and his/her broker probably have a procedure to follow. At the least, you should document everything that's occurred--in writing. Be as specific as you can.
You're right: If the man behaved that way the first time around, he hasn't changed. One of my past bosses had a great saying which probably can't be repeated in full here; fill in the blanks: "Don't get into a p*ssi*g contest with a skunk."
I understand your agent's position...but I certainly understand yours, too. And, if the property was only on the market since mid-January, you may well get more offers soon. Whether you sell to him really depends on how much you need/want to sell right away. If you don't have to, I personally would agree with you; I wouldn't want to sell to him.