The seller is losing his house, his credit is probably ruined and you want to know if he will clean the pool. Weird things happen with short sales. If this is the worst thing that happens in this transaction and it closes so that you get the house that you want, count yourself lucky.
What does your agent say?
It is odd that the appraiser made the request... what's at stake for them? The negotiation is between you and the seller (or the seller's lien holder in this case). The appraiser should be an "independent" third party used only to assess value. Was this a typo on your part or is it really the appraiser that is asking the question? Perhaps it not transparent and making it difficult for the appraiser to assess the condition of the pool and therefore they need it clear to assign a proper value? If that's the case, you might ask if empty is acceptable rather than clear. The appraiser should be able to see the structural integrity and effective lifetime remaining in an empty pool.
Anyhow, if it isn't in writing, it doesn't exist. Get your time, effort, and money spent on refreshing this pool documented and protected in a contract (or addendum) and into the escrow office. Otherwise, it doesn't exist.
I think you have to decide exactly what you are willing to do (and spend) in order to complete the deal. It is your comfort level that counts here. You should be aware that, with a short sale, it is possible you could even lose the deal at the last minute, meaning that your contribution will go unrewarded. On the other hand, if the work gets you where you want to go, you will be happy you did it.
In other words, it is a tough decision, which it sounds like you have already figured out.
Not uncommon for buyer's to put in some money to the deal and take care of things they would never take care of on any other transaction during a short sale. However, I assume the house must be worth it for your above and beyond efforts. I assume the seller can't, won't, doesn't care, doesn't even live there anymore and somehow the listing agent has simply decided to let you deal with it. Weigh out your options, push back to be reimbursed. Some community have blight laws that push the sellers to take care of these issues. Whether it is sold as a short sale or a foreclosure (REO), this will need to be resolved. Have your agent work hard to have you reimbursed if at all possible.