Yes, check with the town for both zoning and the building department. The basement has to meet certain safety criteria and must obtain a building permit and certificate of occupancy from the building department. Your parents may spend more in creating the finished basement than they will in rent in the short term since you mention they are riding out the market. I assume that means they waiting on prices to tick up before selling?
Don't forget, their property taxes will also increase. Will they be able to afford it even with a tenant?
Once they do go to sell, they will have a property that will be attractive to a smaller pool of buyers because fewer people want a multi family and even fewer than that want to pay higher taxes and insurance premiums.
Also consider whether or not your parents want to be landlords. It is NOT as easy as it may seem. What happens if the tenant doesn't pay the rent?
It's good that you are thinking this through.