Welcome to the US. First lenders will look at the credit of the primary wage earner or the middle score or the borrower with the lowest scores. My first suggestion would be to build your credit and improve your husbands.
Scores are just a part of the bankâ€™s decision process. Having good job time, proving your income, having assets & down payment are more considerations.
If you try to buy when you are marginally qualified, you will pay higher rates & fees and have fewer choices in lenders. By improving your credit, you will be able to afford more house at the same payment.
You will need to meet with a local lender to discuss your specific situation. As a lender in the past, I have worked with people in your situation to help them qualify. Ask the lenders you talk to if they can do the same for you.