As a life long NYC area resident and retired realtor, you need to visit the following:
East side of the Hudson River:
northern Westchester County (above I-287 - Cross Westchester Expressway / Westchester Ave.). Visit towns like Cortlandt, Mount Kisco, Yorktown, Bedford and Somers.
Putnam County - anywhere will fit your criteria. You MUST visit Clarence Fahnstock State Park - you will LOVE it!!!
Suffolk County (Long Island) - stick to the border towns with Nassau County to shorten the commute to Manhattan. You will love spending time along the Atlantic Ocean beaches!
West side of the Hudson River:
Rockland County - anywhere fits your criteria. Avoid the towns of Monsey & Spring Valley mainly due to the large Hasidic Jewish population and they control the public schools. The rest of Rockland is great.
New Jersey - not that knowledgeable so you will need to search on your own. Research income taxes as this may affect you.
Property taxes - the bad news. Get ready for sticker shock. The high tax amounts is to pay for the great school districts and residents vote on school budgets every year.