Use a cleanr made specifically for acrylic windows such as Brillianize. Ask at your local home improvement center and they should be able to direct you to a product that will work. Spray on and wipe clean.
Acrylic has decent chemical resistance, but I wouldn't use anything too harsh on it. You can try Windex in a small corner to make sure that it doesn't attack the plastic. If you are worried, just use warm water and a soft sponge or rag or add a little bit of mild dish detergent to the water. Contact the manufacturer of the windows (if known) for the best answer on what they recommend. Good luck!