Hmmm, PA sure is starting to sound good...
Yes, PA would be a far better choice than NJ, especially if taxes are a concern (and they should be). The commute is about an hour, and lots of people do it. It is perhaps likely that your husband can simply car pool or take a bus with loads of other people that commute from PA to Morristown every day. I worked very close to Morristown for several years and lived in NJ for 17 until recently - so I do know the area.
Here's some insight as to specific taxes and their implications for you:
State Income Tax. In NJ, your husband will be taxed at either 5.525 or 6.37 (above $75K annually)
In PA, your husband will be taxed at 3.07, regardless of his income. BUT, different towns can have this thing called an "earned income tax (EIT)". Some charge nothing, some charge 1% - it varies. Here is the proper link to find out exactly what the EIT is for every area in PA: http://munstatspa.dced.state.pa.us/Registers.aspx. Many agents don't know this info or how to find it - so here you go!
Property Taxes. It is usually a fair statement that a home in PA has HALF the property taxes of a comparable home in NJ. NJ property taxes are VERY high, and continue to rise nearly every year. PA property taxes tend to be very stable.
You may wish to consider Stroudsburg, PA or East Stroudsburg, PA. Property values there are very reasonable, especially now. Please find below a link that will connect you to a pretty decent listing of homes that might be worth your consideration:
Please do feel free to ask follow-ups! My personal email is in my profile if you'd rather not communicate in a public forum.
Although the commute to Morristown from PA is doable (some people even commute to NYC from PA) it is not an easy commute during rush hour. My husband use to commute to Morristown from where we live in Mt. Olive(Morris County) and it was usually at least 45 minutes to an hour, sometimes more if there was a traffic "event".
I live right on the border of Warren County where the taxes are a bit lower and you can get a little more for your money. I basically live in one of the most affordable towns in Morris County, but admittedly the taxes are quite high.
I'm sorry, I really can't help you from the PA side, since I am not licensed in that state and I don't have any experience with the commute but I thought it might help you to know that the commute from just one end of Morris County to the other during rush hour is long and quite frustrating since all the arteries going into Morristown at that time become severely clogged.
Good luck! I hope you find what you are looking for.