That may be the case if when you purchased the house, you added the funding fee to your mortgage amount, have never paid anything additional towards the principal or refinanced the loan. It also depends on how long you have had the mortage. In the first five or so years, the majority of your mortgage payment is going towards interest not the principal of the loan.
The purchaser would still have to qualify for a mortgage with the lender. Another thing to watch out for is the fact that interest rates have dropped considerably. If your rate is higher than the current interest rates, there would be no incentive for the buyer to do that.