Yes, PA would be a far better choice than NJ, especially if taxes are a concern (and they should be). The commute is about an hour, and lots of people do it. It is perhaps likely that your husband can simply car pool or take a bus with loads of other people that commute from PA to Morristown every day. I worked very close to Morristown for several years and lived in NJ for 17 until recently - so I do know the area.
Here's some insight as to specific taxes and their implications for you:
State Income Tax. In NJ, your husband will be taxed at either 5.525 or 6.37 (above $75K annually)
In PA, your husband will be taxed at 3.07, regardless of his income. BUT, different towns can have this thing called an "earned income tax (EIT)". Some charge nothing, some charge 1% - it varies. Here is the proper link to find out exactly what the EIT is for every area in PA: http://munstatspa.dced.state.pa.us/Registers.aspx.
Many agents don't know this info or how to find it - so here you go!
Property Taxes. It is usually a fair statement that a home in PA has HALF the property taxes of a comparable home in NJ. NJ property taxes are VERY high, and continue to rise nearly every year. PA property taxes tend to be very stable.
You may wish to consider Stroudsburg, PA or East Stroudsburg, PA. Property values there are very reasonable, especially now. Please find below a link that will connect you to a pretty decent listing of homes that might be worth your consideration:
Please do feel free to ask follow-ups! My personal email is in my profile if you'd rather not communicate in a public forum.