It is never too late to buy a home, it is never too late for anything. Plus 46 is the new 66, is it not? Lol :)
Anyhow, if you have a good downpayment and you qualifyand a bank is willing to give you a mortgage then I don't see why not. Good Luck, I hope you go for it.