Not knowing exactly what is causing your scores to be that low, my guess would be that you do not have any revolving accounts open. Is this correct? If you do not have any; I would look into opening ONE credit card for each of you. Not a store credit card, a visa or mastercard. If it is determined that you cannot get a regular credit card then look into getting a secured credit card. Google it. Several banks will come up. Once you receve the cards in the mail charge gas or something small, then pay it off immediately. Keep doing that and in about two months you should see an improvement to your scores. If all that is open and active on your credit is installment loans and mortgage loans, your credit will not improve.
The minimum credit score requirement these days is 620 for a FHA loan. USDA will go down to 580 but you cannot have any derogatory items reporting in the last 12 months. Plus for a USDA loan you'd have to purchase a new home out in parts of Leander, Jonestown, Lago Vista, etc. Not Cedar Park.

John has a fake profile, I wouldn't pay any attention to him.

Katrina,
You should contact a LOCAL lender and have then run your exact situation thru the qualifying program. Opinions of agents and brokers don't matter unlesss they are giving you the loan. If you need lender
contacts I can point you to a good one. The lender should be local because you want to be treated as
a person, not just a number like the internet lenders usually handle people.
John

5/3 bank does refinances without a credit score, check with your Broker and see if they use them.

Primary borrower's credit score needs to be 620. There are a few lenders left who will finance borrowers with scores below 620. Write a good explanation letter for the medical lates.