I would think that the fence simply "belongs" to the neighbor. If anyone wants to move it, that is their thing.
As far as moving the shed: I would gather, since you said it would cost a lot, that it is a big, heavy, wooden shed. If that is the case, I will tell you how to move, at no cost!
Go to the Home Improvement store and buy a dozen of those rounds poles that they use for stakeing new trees. Using a floor jack, lift the shed and stick the poles under it; probably a little at a time, until you have it totally on the parallel poles. You will find that two men can easily roll the shed where you want it. repeat the process to remove the poles. Return the poles to the Home Improvement store after cleaning them.
Good luck with everything.
I'm not a lawyer, so this isn't legal advice. But you've sold the darn house with the encroaching fence and shed. You're out of it.
There could be a question about whether you misrepresented the situation to the buyer, so perhaps the buyer has some claim or complaint against you. Still, if you disclosed in writing that the fence and shed were on your neighbor's property, that's pretty clear.
Certainly the buyers can waive a survey, but the survey wouldn't have contributed materially to the facts--the buyers already knew (and by buying, acknowledged) that the fence and the shed were on the neighbor's property. Whether you and your then-neighbor argued about it really doesn't matter.
The shed and the portion of the fence on the neighbor's property belong to the neighbor. If the buyer and the neighbor want the fence and shed moved, that's their business.
Let me guess what may be happening. Your old neighbor is concerned that the area that your fence is intruding on, and the shed, somehow mean that the property is no longer his. He's worried that the new buyer owns it. That's why he's telling the new buyer that he argued with you about it. He's trying to establish that he raised that concern while you were there, so that the property wouldn't automatically revert to you. The new buyer wants to get along with the neighbor. The buyer doesn't care about whose land it is, but he wants to get along with his new neighbor. And now--based on what the old neighbor says--you're the bad guy. The new buyer would be content with leaving the fence and shed where it is, but with his neighbor raising a ruckus, he wants the problem solved.
Unfortunately, all this is mucked up, with statements orally, not in writing. With encroachments. With Realtors playing lawyers giving advice about whether waiving a survey has any bearing in the matter.
You're likely legally in the clear. Again, that's not a legal opinion. After consulting with a lawyer, though, you'll still be faced with a question of whether to pay for moving the fence and shed just to keep everyone happy, or whether--perhaps--a sternly-worded letter from your lawyer will put the burden back on the buyer and the neighbor . . . where it very likely belongs.
Hope that helps.
The fact that you disclosed it, did not solve the problem and it probably is going to get a whole lot worse, so why complicate your life over something you should have resolved long ago.