The answers below address a lot (no pun intended) of the issues associated with lot premiums. A great deal depends on the builder and development. In some areas, the variance in lot premiums is small, as there isn't a great deal of perceived (or otherwise) difference in the lots. They may vary slightly based on the size of the lot, but view, setting and other variables don't amount to that much and, therefore, the builder does not charge a great deal for the more "premium" lots.
In other areas, and with different builders, the lot premium may vary a bunch. If there are truly premium lots (back to open space or a golf course or other desirable area; mountain view, etc.) then the builder may charge thousands of dollars more for these lots. If, as stated in answers below, you are comfortable with a lot that isn't "premium" and your Realtor agrees that the lot isn't horrible, then you will probably be OK with purchasing a lot with no premium. However, keep in mind that, when you do go to resell this home, the homes on lots with truly amazing lots, will probably sell faster and at higher price. But then again, you are paying less for the home now.
Sorry for the ramble, but hope this helps.
Your Castle Real Estate