If you and your hsuband never had credit, you wouldn't have any credit score. Go to your local non profit housing group for credit counseling. The counselor will go over your credit report with you. They will tell you what you are doing wrong and what you should be doing. Listen to them and when your score is 640+ you can think of buying.
In most of the world, entire families live in homes that most Americans would consider a walk-in closet. While it will be tough, I am completely sure you can survive it.
I hope 2012 will bring you and your family health, happiness and good news on the job front.