Although I'm not an attorney & you should seek competent legal advice. I would say you could win in a small claims or otherwise courtroom because you removed your loan contingency based on what your lender stated.
However, you should check your paperwork. If the lender only told you VERBALLY, you may not have a case. If the lender told you IN WRITING, check to see if there were any other If this' or if thats, like "your loan is approved so long as.....your credit stays the same or your income stays the same. If your credit or income changed, the lender will say this is your fault.
I don't know how much of a deposit you have at stake or if the sellers will seriously suffer as a result of your not closing, BUT you let them know that if they put up a fight, so will you based on your lender telling you to remove the contingency.
Prior to court action, please know that (if you are using a Realtor in this transaction) you do have other mediation options that may solve the problem for less cost to you & will keep you out of an actual courtroom.
Realtor Since 1996
Main Street Realtors