You don't necessarily have to be debt-free. Lenders are looking at a combination of credit score, income, how much credit you have access to and how much debt you have relative to available credit among other things.
One of the things that really hurts people is credit card debt particularly if you continually carry a balance near the top of your credit limit. If that's your case, then I 100% understand why your lender is telling you this. I'd personally never buy a house if I had any amount of credit card debt. It's just such a huge waste of money to pay double-digit interest that I'd make getting rid of that a priority over buying a home. If you have the cash to pay it off, I'd say do it by all means.