If the craftsmanship of the work is apparent, you might get lucky. That brings me to the second point. Has the work been done well? Are you planning to stay a long time.?
Can you buy or have seller buy a home warranty?
Luxury Home Director
This may apply in your case, after the 1994 Northridge earthquake cities could not handle the permit requests & didn't have enough inspectors to handle the onslaught of requests. There were many chimney repairs completed without the usual inspections &/or permits. I have been an agent in the SF Valley & Westside for going on 18 years & have dealt with these issues in the past.
Brad Reed, Realtor
Permits don't guarantee work is done right, just that someone from the local jurisdiction signed off on it. If your inspector has concerns for the quality of workmanship, you should request additional inspections by qualified specialists; in this case a licensed roofer and a masonry worker.
I've seen Permitted work which should not have been approved and I've seen homeowners do work without permits which exceeds all code requirements. See what the experts say.