First of all, I am not a lawyer and always recommend seeking the advice of legal counsel when it comes to contract disputes.
You have signed a legally binding contract. There is no three day right of rescission for Massachusetts real estate contracts like there would if you were buying a car etc. Unfortunately, if the contract was executed fully and properly and the broker does not want to release you, there isn't much you can do about it.
I would recommend taking the friendly approach and telling the agent that you made a mistake and would like to cancel the agreement. He will need to get permission from his owner/manager because your contract is actually with the owner/manager, not the agent. If the 'boss' doesn't agree, then you have a problem and you may want to consult an attorney. It also wouldn't hurt to call the 'boss' directly.
From the agent's point of view, he has put in time and effort to meet with you, price the property and sign the contract, and you obviously felt comfortable enough to want to do business with him initially. It may be good to give him the benefit of the doubt, unless you feel his email is simply too off-putting. I have found that emails can often be misunderstood or misinterpreted, especially those sent from mobile devices where the sender often is in a hurry, or is distracted. This may not be the case here, of course, so you must act accordingly.
Good luck, I hope you sell the property.