Yes you can decide you do not want an Open House.
Not every home is a good candidate for an open house.
I suggest having an honest discussion with your agent they may be able to take pre cautions that you will be comfortable with. An open house can be a great marketing tool, I myself sold 3 homes from the open house last year. Best of Luck !