Essentially buyers can't just arbitrarily walk on a deal and keep their earnest money (good faith deposit). They have to have a reason protected under the fully executed contract.
Consult an attorney or your agent. They can tell you specifics once they look at the contract.
This will be very frustrating but nobody will be able to answer your question without seeing the contracts and reviewing the transaction. Real estate transactions are very complicated things and in your circumstance there are numerous ways for the buyer to get out depending on how the contract is written and there are numerous different courses where you could pursue the money.
My advice is to have you review your situation with your agent and ask the same questions. If you don't like their answer or didn't use an agent, it is time to get an attorney involved. It has been my experience over the years that earnest money fights aren't worth the trouble.
If the buyer is not backing out because of one of the standard contingencies (i.e. unsatisfactory home inspection, financing request declined, etc.) or another contingency written into the accepted offer, then the seller technically could be entitled to that deposit. However, as Cameron pointed out, it may not be worth the aggravation and cost incurred (since an attorney may be required to help fight for it).