Your question is "What are our legal rights?" The answer is get an attorney. Realtors are not lawyers and cannot tell you what your legal rights are. We can only direct you as to working with a contract, we cannot interpret the contract - only explain it. Yes, there is a difference.
I would read your contract and check the critical dates. That should give you an idea of whether you are still under contract . Certain actions must be performed in a timely manner to stay in contract. So read your contract and any addenda carefully.
As to your down payment or earnest money that should be in an escrow account with your realtor's broker. If you are in contract then it should be handled at closing. If you are not then you have to read your contract to see if you violated any of the critical dates of action and forfeited your downpayment. If not, and you are no longer in contract for another reason then a simple request for your deposit to your agent's broker should get a refund to you.
Now, read your contract, speak with your agent, call the agent's broker to confirm your understanding and if necessary get an attorney,