The train ride from Milford, Connecticut, to Grand Central appears to be 1 1/2 hours, which is further than you are looking for.
Use the two sites listed above and check which towns will work for you. Keep in mind that from border towns - Danbury, New Fairfield, Ridgefield, etc., you can catch the train in some of the New York stations, such as Patterson, Southeast and even Katonah will work from Ridgefield.
Ridgefield will offer a few homes for you to choose from, and it very much has that "Westchester" feel. Danbury and New Fairfield will offer many more homes for you to choose from, but check on those commute times.
And, yes, we have much lower taxes in Connecticut. We have a lot of people moving from New York to Connecticut, and commuting to Manhattan, because they can afford more house in our area.
I would recommend towns like Fairfield, Trumbull, and Monroe. Email me at firstname.lastname@example.org and I will set you up with a Listing Book account. With Listing Book, you can search for up-to-date Homes on the market, daily emails of homes that fit your criteria, the ability to save your favorites, free school and neighborhood reports, mortgage calculators, and much more. I look forward to hearing from you.