You should contact the Title company. They are normally responsible for checking back taxes, utilities, violations and liens against a property before the sale. They will be able to explain why these outstanding bills was not detected.
Normally, all outstanding bills, which includes water and taxes are collected in full up to the date of closing and in some cases an additional amount is held in escrow. Therefore, I would suggest you discuss this matter with your attorney at the time of closing and the title company.