Remove the word "cheapest" from your post......and you have likely described the more expensive real estate areas in NJ. A scenic, tree-lined area with shops, parks, and convenient travel to NY comes with a price.
How long of a commute is acceptable? What is your real estate budget? With the answers to those two questions, your likelihood of relevant responses will increase.
Cheapest travel could mean Path, Midtown Direct on NJ Transit. You can also take the train from many other points, but you may consider a longer ride inconvenient. Alternatively, one can take ferry service to a number of points. See NY Waterways and Seastreak Ferry service route maps.
Shorter commutes and convenient travel to NY usually will be attached to hire real estate prices. Lower real estate prices are usually found further away from the commuter centers.
If travel options are your priority, look at the transit and ferry maps, and solicit feedback for towns along the routes within the commute distance and cost that is acceptable to you and define your real estate budget. Many people in Monmouth County commute to NYC every day. It's an hour by train and 40 minutes by ferry. For some, the lifestyle (a blend of shore, suburban, and country), along with more bang for your real estate dollar (compared to other parts of NJ) make it a perfect place. Others may prefer to be on the Midtown Direct train route.
Whether you direct your focus to Hudson, Bergen, Union, Middlesex, Essex, Monmouth, or other counties, there is likely to be a neighborhood that meets your expectations. Which areas fit your budget? Which areas are within an acceptable commute?