You have fallen in love with the house, but obviously not with the size of it, so this is not only a price point issue, or you would not have mentioned space at all. It's fairly quick and easy to have your agent or an appraiser evaluate whether the price you have agreed to pay is within a reasonable range for comparable properties. More difficult is evaluating the compromises that both you and other members of your family will have to make because of it's less than perfect size. That aspect doesn't have a price tag, it's a quality of life issue.
Paying less for a house doesn't make it any bigger, but perhaps you were more comfortable at the lower number because it meant that you could afford to pay for more family activities outside the home to compensate. If paying more for the house means fewer activities outside the home, that is something that needs to be worked through with each member of your family, to find out what is important to them, and where and how you want to spend your money as a family.
Another factor to consider is whether or not the house has any potential for expansion. If so, then even if you don't want to do the work now, or can't afford it straight away, it's only a matter of time until you can have your cake and eat it.