I'm not sure how brokers do it in New York or elsewhere, but the comp plans here at all the reputable mortgage companies I applied with last year are all nearly identical with very close rates (+/- 0.125%, perhaps it's because Chicago is such a competitive area (there are more lenders/banks than you can shake a stick at).
While I understand a lender could set their margins higher, it's suicide in a competitive market. Even when I worked at one of the biggest mortgage companies in the country, their rates were always competitive even though they had the most overhead of probably any company (lots of brick and mortar).
Outside of a mortgage company that's trying to shark you (usually any intelligent Realtor can figure this out), you're going to be looking at very similar pricing when talking to lenders.
Yes, there's always the flip side and someone could argue their point until the cows come home but consumers shouldn't come into this expecting to find sharks and waste precious time weeding them out, they're looking for an expert to give them sound professional advice who'll be reasonably available when they have questions.
Consumers who're reading this thread have some idea of where rates are (and probably know not to necessarily believe everything they see advertised) and if they smell a rat (or over-pricing), they're going to seek out a second opinion regardless of how well endorsed someone may be.
I have high producing realtors in areas outside of my local market sending me clients, not because I provide the absolute lowest rate (although it's always competitive) but because I give their clients the experience they're looking for in the biggest transaction the'll make in their life and the knowledge/expertise to back it up. This is what every loan officer should be doing for their clients and many of you already do this. This isn't an attempt at self promotion, I'm just making a point that the "lender" search should really be a "loan officer" search in our current market. No matter how much regulation there is in an industry, there's always going to be someone another lender can call a shark. I.E. I've seen discount (commission returns) Realtors do this to non-discount Realtors to elevate their standing as the "Agent of choice". While I don't want this to become an issue of what is or isn't fair with respect to compensation, my point is that given any industry, one could make a similar statement.
As I stated previously, find a professional, someone who's clearly proven themselves and go from there. If you're looking for specialized financing, find someone who specializes in it.
Best of luck in your search!
P.S. Notice I didn't put my company/contact info at the bottom? Trulia frowns upon (blatant) self promotion, lets clean up the forums guys, contributors on other sites make fun of you guys who post all your contact info here. If a consumer wants to contact you, they're smart enough to click on your picture and pull up your profile.