The biggest difference between a modular and a site built comes down to 2 things. 1 you may need a lot more supports down cellar as the modular is built in 2 halves and 2 halves do not ( to the best of my knowledge) have the strength a single beam would. That only means that you can't expect 2 separate beams to lock together with the structural ability of a one piece with no joint. and 2 you will find a connector in the wiring harness in the cellar where the 2 halves are put together.
Now a doublewide is very different. It is a trailer sitting on iron beams that have been known to rust away. Sitting on supports that can move, and often has floors made of particle board that is known to break.
A modular costs about $15-25k more here over a doublewide. It is worth it. Sometimes a ranch can be built cheaper on site, sometimes a house is much cheaper as a modular. What you need to look at are the specs. Are the floors made of 1 1/4" planking? Does it have anderson (with all the toys) windows? What kind of doors, how thick are the walls, what thickness drywall, what r-value insulation and so on.
For myself, If I was to buy new I would rather look at a quad-lock Insulating Concrete Forms ( or other mfgr) concrete poured in foam block walls built house. Very energy efficient, lower fire insurance, much quieter, and best of all, it will not rot or be eaten by bugs during your lifetime.