I have the feeling that you are adding something after the house is being built:
This really complicates things for the Builder, the paperwork, the separate purchasing, the follow-up with the Subcontractor; they really do not like CHANGES, and that might account for some of the $$.
You should have thought about it in the design stage, and you should be understanding about his position; he is not ripping you off, he is trying to get it done.
At this point, I would do it myself, later. But that's me.