I hope your agent has reviewed the situation with you and discussed your options clearly and concisely. Your contract is the operating document; take another look and see if you have any recourse regarding "Property Condition".
Best of luck...
(Last thought - with interest rates still nudging down, it might be to your financial benefit to "start the entire process over again..." - it might end up saving you some money on your rate. Just a thought!)