First, you will have no control over the other occupants. (You won't in a single family either, but there is usually more separation.) You will have to think about how you will get along, because decisions about when to fix the roof, paint the siding, or replace the driveway (for example) will have to be agreed on. If you buy a condo in a multi-family, make sure you read the condo documents carefully and understand how these decisions are made. If there are condo fees, who controls the spending? I once had a client who had trouble selling her unit because exterior repairs were needed, and the other owner refused to contribute. In another sale, my buyer rejected a beautiful unit because the unit next door had trash all over their side of the yard.
So if you are thinking of a two-family, your best bet is to buy the whole thing, and rent out ther other half. Then the advice below from Marybeth wiill be very important. Good luck!
My answer is going to sound a little Wizard of Oz'ish but you have the ability to answer your own question.
Others can offer their opinions, but the ultimate decision is, and should be, yours because it is a very personal one.
Whether you are buying a unit in a two family or a single family, you will be asking yourself the exact same questions.... Does this property "talk to me". Does this property feel like "home"? Is the property in a good location? Which school will my child be attending? Could there be a presence of lead paint that could potentially be hazardous to my child's health?
If you can be flexible with your town and location...and patient.... you will find the perfect home. What is most important is that you do not limit yourself by what you "hear". Go with your heart, a good buyers agent who understands what you are looking for, and you will soon be saying "There's no place like home!"