Check with your lawyer. Note: The seller is providing a reason. It may not make sense, and it might not hold up in court, but it IS a reason. But your lawyer would by far be the best person to answer.
As a practical, non-legal, answer: It's nearly impossible to force a seller to sell if he/she doesn't want to. Your lawyer will be able to provide advice on whether to seek damages. However--again, as a non-lawyer--the best thing to do would be to try to persuade the seller to change his mind.
Find below a link which will take you to a sun angle calculator giving you a basic understanding of how to site your home on the property in question. I'd invest in a good compass. I have compasses on hand as I show country property and acreage and this question always comes up. If you google Solar Orientation you'll get a wealth of information. We are coming into the golden age of solar, passive solar designs. Just north of my county we have one of the largest daily solar application villages in "Hopland,Mendocino County, California". They have alternative energy applications galore!