Good lesson going forward .... maybe consider signed paperwork spelling out all scenarios before giving someone money?
A lawyer can give you a definitive answer. However, the whole idea of a deposit--regardless of size--is to bind you to your commitment. And you sacrifice the deposit if you default. You acknowledge you changed your mind. If there had been a written agreement AND if the agreement had outlined those terms regarding a deposit (nonrefundable if the renter defaults), you'd clearly lose your deposit. Sure, it's possible that the written agreement would have provided for a return of your deposit . . . but there was no written agreement. And you acknowledge, again, that there was nothing either in writing or verbally regarding the disposition of the deposit in case of default.
So, how could you make the argument that you're owed a refund? Well, you could argue that you didn't agree (verbally or in writing) that the deposit was non-refundable. But I can imagine a judge asking, "Well what did you think the deposit was for?" You could try arguing that there was no lease, and therefore that a deposit for a non-existing lease is impossible. But, again, I can see a judge asking, "Well, what was the money for, then, if not a deposit? If it was simply a gift to the landlord, then it was a gift and he gets to keep it."
Sorry. Your position appears very weak. But, then again, I'm not a lawyer and only a lawyer can really straighten this out. However, as a practical matter, you're likely to find that a lawyer is going to cost you more than you might recover even if you were successful.
In the old days business was done with a handshake and all parties kept their word. These days, I tell my clients and customers that my word is so good, I put it in writing. I don't mean to cause you any added pain, but why would you turn money over to someone without a written understanding?
If you have resistance in getting your money back, you'll need to opinion of an attorney as to where you stand and how to proceed. The landlord has acted just as recklessly by accepting the money without a contract if he turned away other tenants. He may feel entitled to it if this were the case. Whether he is or not may be up to a judge.