I agree with Carl. Given your situation they would hopefully oblige (bring proof of the layoffs).
If this does not work a review of the contract by an RE Attorney followed by a stern letter might solve the issue.
Agents are not lawyers so we can not practice law by giving you legal advice but my understanding is: once you release all the contingencies: if you decided to back out, they would be able to keep your deposit and sue you for any loss they may incur (such as the legal fees & the difference on what you had agreed to pay for the home & the price they get for it from another buyer) . They could sue you for the full price of the home &/or punitive damages but I doubt they would be able to win more than their actual losses since it is unlikely that they wouldn't be able to resell the home to someone else.
Read your contract and see if there is a 'liquidated damages clause' (usually a paragraph) which would have spelled out at the time the contract was written what the monetary value of damages would be in the event of breach of contract by one of the parties. If the property is 1-4 units, Civil Code 1675 limits the damages to 3% of the purchase price unless there is a judicial or arbitration decision.
Many buyers do not realize that they can have their own agent represent them in a purchase of a new home (even if the builder is not cooperating~i.e not paying a commission) & that their own agent would be an asset in helping them through the whole process.
Congratulations on your new home it is a great time to buy!