There are more elements to an "inspection contingency" than just the physical condition of the house. The buyers also have the right to review disclosures and investigate lots of factors that may influence their purchasing decision that are not visible to the eye and have nothing to do with the structural integrity of the building itself--factors like nearby environmental hazards, local zoning/permit regulations, presence of nearby registered sex offenders, school district policies, a neighbor's fierce dog, a past insurance claim on the property, a cloud on the title history, noise or traffic at certain times of the day or the week from a nearby church, theater, bar, construction site, etc....all of these fall into the realm of what the buyer can "inspect."
So if the buyer's agent has included an "inspection contingency" in the purchase contract, the buyers SHOULD be able to find SOME grounds for getting out of the contract without risking their deposit during the "inspection contingency" period.
The seller does not get to judge the "worthiness" or "validity" of the buyers' stated reason for withdrawing from the contract during the inspection contingency period....but the buyers should be careful not to do or say anything that would leave them open to an accusation of acting "in bad faith," which could complicate the picture.
So it's fine for the agent to say, "My clients decided the street is too busy" or "The wife decided it's too far for her to walk to the store" -- but should NOT say, "My clients had no intention of really buying...they just wanted to see how low a price the seller would accept."
If you'd like, we'll be happy to discuss in more detail how the contingencies process works, in regular sales, in short sales and in bank-owned "REO" sales.