Once you close on the property the encroachment issues become your problem and expense to cure. The time to speak with an attorney about the potential issues and ramifications is now, before you close. Your agent cannot give you the kind of legal advice you need at this point in time.
Most states have laws regarding what is called adverse possession. If the neighbors sheds on this property have been there for a specific period of time (often 7 years) and meet the other criteria about hostile and open possession, the neighbors may actually be able to claim the land their sheds sit on as their own. As I said, talk to an attorney now, before you close so you know exactly what you are dealing with. Your husband is wrong, this has the potential to be a big issue.