And you pay capital gains tax only on profits ABOVE: $250K for Single, $500K for Married couple.
You calculate profit as follows: Sale price - Purchase price = Profit
If the profit is higher than $250K or $500K, you do owe tax.
Pay it and be glad that you are lucky enough to actually be able to pay the tax - there are many others who instead face a short sale or a foreclosure!
NOTE: For tax advice always contact an accountant.