If you cancel the escrow prior to your contingency period expiring..you have a right to your deposit back. If you cancel after the contingency period expires, you will lose it.
if you do cancel, the seller must agree to return it and technically speaking, can give you a difficult time in doing so. In 10 years of practice however, I've only seen that happen once, however!
Typically after 17 days (depending how the purchase contract is written and if the default timelines are changed) the buyer will be required to 'remove all contingencies.'. Once contingencies are removed is when your earnest money deposit is at risk.
If you have any questions your agent (if you have one) can walk you through the contract contingencies.
If I can be of further help please do not hesitate to ask or write.