There are houses of less than two thousand square feet on more than five acres all over the Hamptons. I can think of several.
We still have farms here in eastern Suffolk County. And plenty of people neither want nor need a gigantic house.
Also, two big luxuries in the Hamptons are space and privacy, especially for people escaping New York City. Since this is a summer resort, people spend a lot of time outdoors. The generously sized estates allow for swimming pools, tennis courts, lawns for children and grandchildren to play upon, and gardens for flowers and vegetables.