I agree with my colleagues that your contract contains your answer.
I want to make a suggestion however. If you are getting interest as a result of that agents efforts, it may be a good idea to get them, or at least some agent involved to assist and manage the process from here for your benefit.
Getting an offer is only part of our job. Managing all the issues through the escrow period, inspections, appraisal etc are valuable services we provide beyond the marketing. Where you are off the market and if the buyer has no agent, you may be able to work out an arrangement that benefits everyone, especially you.