You are free as a purchaser to work with the owner directly. The protocol in this situation would be to work with the agent since you asked them to establish the appointment. The perception with many purchasers is the real estate agent is going to cost you more than purchasing a home directly from the homeowner. If the house is listed for $400,000 and you purchase the home with the real estate agent for $380,000, your purchase price is $380,000. If you purchase the home listed at $400,000 directly from the owner and dont work with a real estate agent for $380,000, the price is the same.
The transactions between home owners and home buyers are typically based on the recent sale prices of similar homes in the area. You are not buying the house for less money. The seller is netting more money after the sale because they did not pay a real estate commission. You paid the same money.
The important point to consider is this. With a real estate agent, you have a professional working on your behalf. A real estate agent is required by license law to work ethically. A real estate agent also has the experience and knowledge to guide you through the home buying process while protecting your interests. There is a strong value in the equation of purchasing a home with a professional and experienced real estate agent