Mikhail, the key here is when you say "a legal 2 family". Does this imply that the one you are looking at is not a legal 2? In that case, if you are getting financing, it can definitely be a problem if tenants are still there. You would have to remove the stove in the kitchen before it is appraised, and hope you have an appraiser that doesn't note that it is an apartment.
The other issue is what your plans for the home are. Do you plan to have family members occupy the 2nd unit? You would definitely want a vacant home then. Trying to get a tenant out in NY is not an easy task, and is an expensive one. What theoretically might work, or what is the case in another state, may not work here. I had a deal once fall apart because the tenants wouldn't move out, and it dragged on for months. And, it was a single family where rooms were rented out, but they still couldn't get the people out.