The short answer is read your contracts. There will be clause(s) explaining what you agree to do, what the seller agrees to do, and what happens if home has defects or financing problems. Your Buyer's Agent should be helping you!
If your purchase agreement was contingent upon you getting FHA financing and you are unable to get an FHA loan you should be able to rescind the offer without penalty. That is a quick and easy answer but I would recommend that you get the assistance of and attorney as there often is extinuating circumstances. If the seller is threatening litigation it becomes even more important for you to consult an attorney
IF your inspection contingency can be exercised, then you may have the ability to terminate the contract on those grounds. If not, then that is up to you and your Agent how that is handled, or if it even can be.