There is no need, imo, no need to simply walk away, and call the home a "money pit", without pursuing what might be involved in correcting the issues you identified.
Ad addition was made without a permit that may have items not up to code.
Someone who is not an agent (ie: a mortgage rep pushing his services) may not understand that we, as realtors, deal with all kinds of unexpected issues in selling homes - especially inspection issues..........part, if not much, of our job involves coming up with solutions - not just telling people to walk away and find a new house!
I had a closing this past year with similar problems.
There were a number of issues inside and out (around and including the pool) either with open permits or items not up to code.
Many of the problems dated back to previous owners.
The seller was informed.
He called in the town inspectors (after he had called the building department) who identified what specifically needed to be done............and.......... he did what he needed to do to bring everything up to code.!
He really had few choices as..........he was now aware of deficiencies in the home, and would have to DISCLOSE them to any future buyer, so he chose to deal with them then, and keep his current buyer in place.
He had the work done.............the various inspectors came back...reinspected, and issued the permits.
We closed and everyone was happy.
Once again, - IF you have an agent, please let him or her advise you.
If you use the As-Is contract, then simple, you send a rejection of the Inspection notice and add a without prejudice offer saying what you would be willing to accept. I had this exact same issue come up this year. The list agent and the seller were aware of an unpermitted garage conversion. When my buyer found out, he simply said, the inspection is rejected, but we will proceed if you lower the price $20,000. They did. Do not for one second assume that your seller does not know about the unpermitted work.
If you used the with repair contract, then I agree you should discuss your next step with your agent.
The zoning department should be able to provide you with advice on this matter. If you value the transaction and keeping it together, get the facts before going to the seller and making additional requests.
Finding out what, if anything you might be responsible for would be a good place to begin.
There are two issues to deal with.
One is purely adminstrative and can be corrected in a few days.
The second, often created by inspectors gone goofy, will require you to ask yourself, "Are you feeling lucky?"
Your agent, along with the collaboration of the listing agent can make these things dissppear.
Time for a meet up with your agent and get the process laid out. The only thing unknown and presents the greatest risk to everyone is your indecisiveness.
Trolling for opinions from strangers on the internet is refelctive is that uncertainty. Hopefully, the seller realizes this also and won't allow themselves to be jerked around.
The good news is - the square footage reflects the addition...........so the taxes shouldn't change/increase.
What is your real estate agent suggesting you do?
Are you buying a home that's "for sale by owner"?
Are you buying a home that is represented by an agent for the seller, and you had them write up the offer for you?
There are a few different outcomes of purchasing a home that has work done that was not permitted and is not up to code. The city, county, or other governing entity can fine you, or tell you it must be taken down and redone.
Let the seller know what turned up during the inspection and you expect them to remedy the situation or lower the price by the amount of work needed to get the home up to code. Do this quickly as the contract you signed probably has a time limit for making such a claim.
I hope all turns out well for you.