Perhaps, as part of your purchase, you came and viewed the home 3 or 4 times. Some sellers don't mind you coming afterward, to show your daughter, who was away at college, or show to your mother-in-law who hasn't yet disapproved of the house. But some sellers don't want to allow any more showings, and unless you've written "seller will allow reasonable access to the home between now and closing"... they have zero obligation to allow you back into, what is still, their house.
Frustrating? Yes, a little... but unless you've had the foresight to request "visitation", your agent is correct.. there's nothing he can do, short of begging and pleading.
You will, of course, be allowed your walk-through, but that's typically only a day or two before closing, and that might be a bit late for you.