Like the man said; if you have a signed Contract, you must abide by it.
However, there is a reason why we use Escrow/Title Companies.
And, the money is not their's until and unless the Escrow Closes.
I would be suspicious of the Seller's motive.
Here in NH deposits are held in the listing brokers escrow account. It can also be held by teh title company. If you are writing an offer you as the buyer can request where you want it. if you already have a signed offer than you have to abide by what the contract states.
I am not sure "Why" the selling side has objections to placing he earnest money deposit with the title company, I have been placing the earnest money deposit with the buyers title company for over 15 years with No issues.
I agree with Laura Roskelly; typically earnest money deposit can be held by the buyer's broker or the title company chosen by the buyer. Please review the language in the contract which supports this practice.