New York University is located, centrally, around Washington Square Park (West 4th Street between 6th Avenue and Broadway, roughly). If you are looking for convenience, Manhattan is the right choice. More specifically, Greenwich Village would be prime location. Depending on your budget, though, that may not be ideal. If you have a decent budget, anywhere between $1,500 and $2,500 a month, you will be able to find a great space in the Village. If you are working with a smaller budget, then I would recommend Brooklyn before Queens, for Brooklyn is much closer to NYU (you would be looking at approximately a 20-30 minute commute). New Jersey would force you to commute much farther (involving NJ Transit, driving, parking, subway, or some combination of all). Williamsburg is a thriving area for many college students and young adults, so the young population may be very enticing.
Short answer: stick to the Village or Williamsburg, Brooklyn for its convenience and young population.