Using software called Fidelity Agent, here are some possibilities, based on the PITI of $3500/mo, 5% interest for 30 years, conventional loan (of course, the interest rates may be less than this depending on when you apply for a loan, your credit score, etc). These are mere estimates just to give you an idea.
You should consult a lender for more exact figures as well as find out what loan programs suit your needs best.
What you can afford based on % down payment
If you want to try an FHA loan, 5% interest rate for 30 years
3.5% minimum down payment you can afford to buy $497,605
As for what's available in Lafayette under $660K.... Ther eare 17 properties under this price, ranging from a low of $324K for a 3 BR/1.5 condo to $649,230 for a bank-owned detached single family home with 3 BR/3 BA
Hope you find this helpful. If you haven't spoken with a lender yet so that you can be preapproved, that would be the first thing you may want to do. If you want some references, it will be my pleasure (as well as others' who respond) to give you names/numbers of responsive, responsible and resourceful lenders.
Let me know how I may help.