If the original Realtor was not offensive, nor was there a reason not to work with that person, calling them regarding the home they introduced you to is the courteous, moral and ethical thing to do; they have already invested their time and efforts in helping you find the right place.
As far as any "trouble" goes, you may want to consult your Attorney. At the very least I might think your first broker can make a very strong case for procuring cause if a sale does occur.
But realize, all along, that the first agent and the seller's agent both work for the seller unless you hire the first agent as a buyer's agent, which I am not sure you can do after seeing the house as a seller's agent.
So although you should do the moral and ethical thing and go with the first agent, this doesn't mean you have to drop your pants, apply the vaseline, and squeal like a pig.