There is generally no time limit but it will depend on what type of loan you currently have. Is it FHA or Fannie or Freddie backed? Will it make sense to change the term or type, perhaps FHA refinancing into a conventional.
An any case, check around with a few different lenders to compare pricing and their advice.
One group, that works in all 50 states, that we have had some very good results with is Emery FCU. They can be reached at 1-888-709-4555 on online at the web reference link below.