Right now you can get into an FHA or VA loan with a 580+ middle score but for a conventional loan you'll need at least a 680+ and a larger down payment.
The best bet is if your score is under a 620 is to work with a mortgage professional who can help you raise your scores.
I checked with a mortgage broker here in Boulder. She said the mortgage companies are moving their score requirements up to the 640-660 range for FHA loans. Since credit requirements are moving targets, you should find a good mortgage broker early in the home-buying process and learn what kinds of loans and interest rates are available. Also you should learn about credit scores and do what you can to improve your score. The blog post below may help.
Prudential Real Estate of the Rockies
620 is pretty easy.... depending on the credit profile itself aside from the score.
580 is doable, but many get denied for the very reasons for the 580.... rather than the score itself.