I would be very interested to hear if an agant was involved in this transaction. It is possible for an agent to forget, however, I suspect one was not.
In effect, you missed off the contingency, and have no right of recourse from the result of the inspection and if you decided not to buy, you could have yourself a bit of a problem.
However, if both parties are reasonable, you could work something out. (This should be easier with a 3rd party, which is why I asked if an agent was involved)
If you are not happy with the resupts of the inspection and you want out, I would run, not walk to the door of a good Real Estate Attorney.