It really depends. A buyer will be more confident knowing they truly are going to get the house and not confront an eviction issue, but short sales go sideways more often than regular purchases. I don't blame a short seller for waiting till the last minute to move both to save money and to ensure the deal is going to close.
If you have bank approval for the offer submitted and the buyer has completed their financing application and all signs point to closing, make your arrangements to move.
Moving prior to the lender's approval of the offer can cost you rent for months while the bank decides.
You are absolutely correct to have concern with this. By allowing someone to occupy the property after cloosing you entertain a host of possible problems that include the eviction process should they elect to make things difficult.
It's likely best to take a hard stand and insist that they are out prior to closing. No vacancy=no closing
Could have / Would have / Should have don't cut it ....