If the contract is the CAR contract, Page 5, Paragraph 16. States the days for walk through. If the seller refuses they are breach of contract. Your contract will detail the remedies the Buyer has in regards to seller in breach.
The Client's agents should be present. This walk through is for the benefit of both the seller and buyer. If, you the seller, will not be present, your agent should be present. It documents the condition of the home prior to closing. If the buyers can't get in, if repairs agreed to are not done, or damage occurred during the seller move out, a Buyer can request a delay or monetary hold back.
As frustrating as this process has been for you the seller. I suggest you focus on the end result of getting the house closed on time. And because you are concerned about the condition of the home, insist that your agent is present during the walk through.