Whether or not the earnest money deposit has to be refunded all depends on how the purchase contract was written up and what reasons the Buyer has given for not moving forward with the sale. Read the contract, was this first week the Buyer's due dilegence period, was certain conditions need to be met in order for the sale to move forward? Furthermore, if you keep the Buyers deposit, wrongfully, the buyer can put a lien of interest on your real estate and clouding the title. Read the contract. Did you use a Realtor?

If they used once of the objections such as loan condtitions or inspection objection. Didnt you use a Realtor?
There are ways buyers can terminate the contract however they cant usually just call you up and say "nevermind". The answer to this question would be in the contract that is between you and the buyer.

Are you asking if you have to refund his earnest money? If so, most likly. However it depends on why they backed out. There are certain aspects to the contract that are there so the buyer can get out of a contract. It also depends on how the contract was written up. With out all that information, I can't tell you for sure if you need to refund the earnest money or not.
Are you working with a real estate professional that can help you with that?

Thanks ~ Andrea Haitz