Call the Polk County Bar Association Volunteer Lawyers Project if you don't think you can afford to hire a lawyer.
You need to call a lawyer to help you with this problem. If he can't buy you out (or won't) and you can't buy him out (or he may refuse), then you will need to bring an action to fix this mess.
Let this be a lesson to anyone that thinks about doing such a thing that you are walking into a mine field. Big problems are the rule.