We turned out 3 bedroom home into a four bedroom with a den! Plus I still have a "normal size" dining room!
First, many capes have 4 areas on the first floor--a kitchen, a living room, a dining room, and another room. Often, that "other" room is used as a master bedroom. But yours doesn't have that? Just a kitchen, dining room, and living room?
I'd be hesitant to eliminate the dining room. On the other hand, most people do expect a bath next to the bedroom. That's why I say it depends on what's expected in your community? Are dining rooms important? Or number of bedrooms? If you add another bedroom, you'll now have a 4/1, which really cries out for a second bath.
Ideally, what you ought to consider is adding a bath (or at least a half bath) upstairs. That way, the two small bedrooms become usable, or you can turn those small bedrooms into a master suite. In most areas, 2 bath homes are far more desirable than 1 bath homes. Check with a Realtor who knows your area to determine the cost-effectiveness of adding a bath on the second floor.
Regarding what's classified as a bedroom, different jurisdictions have different definitions. Generally, though, a bedroom must also have a window large enough to crawl through. The size itself is specified by the jurisdiction--ie, 18" x 24"--but the concept is that in the event of an emergency the window could be used as an exit. Again, though, it varies by city and county. On the other hand, even if your basement room doesn't qualify as a bedroom (and thus couldn't be advertised as one), when you sell it could be staged however you wish--as a study, home-office, or bedroom.
Again, your best bet is to talk with an interview several Realtors who know your area and know what buyers are looking for.
As to the bedroom question, your local MLS probably has a definition for what qualifies as a bedroom and a local Realtor could help you. Our local "bedroom" definition means it has a closet and an egress window. Nonetheless, I have gone into homes with Buyers that supposedly were 4 bedroom homes and the Buyer has asked me where the 4th bedroom was because they did not view the room the Seller/Realtor was calling a bedroom as really a bedroom - Buyers are smart and I would recommend "under promise and over deliver" as opposed to the opposite. Creative wording in the marketing can convey the necessary image - good luck with your sale.