If you are applying through a mortgage lender, then you would be using the "Guaranteed" version of USDA financing, which means the mortgage lender makes the loan and USDA is guaranteeing that loan in case you default on it. The Guaranteed version's primary benefit is that it allows higher income limits to be eligible for USDA financing. The other version is called "Direct" as you apply directly with USDA, it has a lot of benefits, including lower fees, a payment subsidy (if needed), and more flexible underwriting guidelines (as they don't add any "overlays", which mortgage lenders do for their version). The downfall of the Direct version is that it has a lower eligible income threshold, and whether you see it as good or bad, you deal directly with USDA, and often their underwriting turn times can be slow (as in weeks for underwriting, just depends on how busy the local field office is).
As you can imagine, most mortgage loan officers who don't work for USDA are primarily familiar with the Guaranteed version of the program, because it's all we can do. A 620 (or even 640 here in California) is a common minimum score requirement for USDA financing, and I bet you could call the Guaranteed office and find out which (if any) lenders are sending them loans with scores ~580 - if there are none then that is a good sign there isn't any. The reason 640 is a common threshold is because that is a cutoff where easier underwriting guidelines apply pertaining to one's credit. USDA specifically states in regards to "Credit Worthiness/Derogatory Credit: Credit Waivers":
"Manually underwritten loans, and GUS loans that receive a â€œReferâ€ or â€œRefer
with Cautionâ€ underwriting recommendation, approved by the lender for
applicants with eligible credit scores of 640 and above do not require the
lender to submit supporting documentation to Rural Development."
And so it alleviates a lot of the underwriter's concern that the loan would have a high probability of being "guaranteed" by USDA, because they had reassurance that with a 640 score they don't need to be as tough (more or less meaning, "asking for lot of documentation") on underwriting. It used to be a 620, which is why a lot of USDA lenders are also using that as their minimum score. But not many go below 620. I still recommend you aim for a 640 score, just so you have an easier overall time in underwriting. Credit score isn't everything though, read the "Update" link below to read through the entire guidelines. It's good to know what they are, and not just take a loan officer's word for it (but make sure to understand that lenders add "overlay" guidelines, so not all USDA mortgage lenders are the same).
Even though I am in California, I work for a bank HQ'd in Layton, UT, and we do USDA financing. We require a 640 score for our USDA program however.
The links I mentioned:
USDA Guaranteed's main guidelines: http://www.rurdev.usda.gov/regs/regs/pdf/1980d.pdf
Update to guidelines in October of 2010, includes documentation requirements pertaining to pretty much everything, including the 640 score information: http://www.rurdev.usda.gov/SupportDocuments/an4543.pdf
These are "updates" (called Administrative Notices, AN's) to the USDA program that are announced, for example clarification of guidelines, general information, reminders, etc.: http://www.rurdev.usda.gov/rd-an_list.html
Web Reference: http://www.trulia.com/blog/shanethemortgageman/