Second, there may be private lenders out there willing to lend you the remainder. The interest rates and points will be somewhat higher than conventional loans--several points and maybe 12% (just guessing, but probably in that range)--but considering you'd only be financing $50,000, the difference between, say, 6% interest and 12% interest won't be terrible.
Another option: Seller financing. Put a big chunk down--say $40,000. Have the seller finance the remainder, again at an above-market interest rate. The benefits for the seller: He receives a large amount of the sales proceeds up front, and is earning above-market rate on the remainder. If he does have to foreclose on you, he's taking back his $100,000 property and already has $40,000 of that. He could sell the property the next day for $70,000 and come out ahead on the deal.
Variations on this would include a land contract or contract for deed. Or use an Illinois-style land trust to protect both you and the seller.
In most of these scenarios, at some point you'd refinance the remainder. The investor loan probably would be relatively short-term. The seller loan might be short- or long-term. But you'd use that time to clean up your credit, so if/when you did refinance, you'd be able to do so at affordable rates.
Hope that helps.