Your agent should have advised you not to close until the previous seller had vacated the property. It's possible, as James suggests, that some sort of rent-back was included in the sales documents. But, from your question, it appears as if there may not have been, or that it expires on May 30.
I'm not a lawyer, so this isn't legal advice. However, right now, you're a landlord, bound by all the rules, regulations, and laws that apply to landlords. And, yes, you have to evict your tenant. And you have to do it legally--with whatever notice is required, and so on. That's why you need a lawyer.
However, there may be an easier way to do it. It'll cost you, though perhaps not as much as a lawyer. Bribe your tenant. Offer her cash. $1,000 in cash to move out. With the $1,000, she can find a place to live. (Didn't she receive some money from the sale?) But you pay her only once she's physically vacated your house and given you your keys. But make the offer limited to x number of days--say 5 days. Or make it self-liquidating: $1,000 if she moves out on May 30. $900 if she moves out on May 31. $800 if she moves out on June 1. And so on.