(and size of septic system, which you can find out about at the town's health department)
You'll see how different agents have different opinions.
My suggestion is to have your agent run comps for both 3 bedroom and 4 bedroom homes. If there is a big price difference, I'd avoid calling it a 4 bedroom if you are buying it and state that in your lower priced offer. If there isn't a big difference in price, go with how you would use this room.