680 may sound good enough for many loans, but the question is what is your middle score?
That's not the highest one, nor the lowest one, but the one in the middle.
Most lenders use the middle score to determine several things about your loan, including whether you qualify for the program or not.
Depending on how much you have saved up for closing costs and down payment, you might qualify for a nice, low interest conventional loan (assuming there are no gotchas in your credit report).
A gotcha would be something like 2 years ago you filed bankruptcy, but now your middle score is 668. Sorry, Charlie, no loan for you, probably.
You might have only $3,000 saved up and want to purchase a $140k house. Sorry, not enough down payment.
So, average score or middle score, that's not the whole story.