Ed & Terry did a good job of answering your question, but I have to ask, for what purpose are you buying the home?
If you plan on living in it a long time then you should be more than fine, especially if you are building the home for yourself, meaning adding upgrades that appeal to you.
Now if you plan on building this home as a starter home and know your growing family will need something larger, then you may want to take a second look at what you are doing and seek some professional advice to ensure you are not overspending on an item that may not give you the returns you want or need. Yes, I know.. here comes the "Realtor" pitch but its true. An experienced agent may be of assistance here, but that decision is for you to make.
So as you can see, it goes back to the original question.
With all my buyers, including commercial property buyers we need to do some real life planning sometimes to decide how and where we look and invest. I honestly believe all the good agents do this. This is putting your needs (the client) first then making decisions based upon those needs. Without this information the fact that your house is the second largest is moot. But assuming this house is the one that fits ALL your needs for the long term. The short rule of thumb is NEVER over-build. So don't be the mansion surrounded by shacks, if you ever plan to recoup your investment.
Happy Home Buying!!