I used to live in San Jose and commuted to Santa Clara daily. Personally, I wouldn't make the commute from Gilroy to San Jose each day. Of course, I had a family and two small children. If you don't mind the commute and traffic, you can probably find a better value in Gilroy than in SJ. Also, if your work hours are flexible, then the commute might not be so bad.
When I was in college, I used to commute from San Jose to Lafayette for work daily (about 65 miles each way). I worked swing shift (3pm-midnight) so it only took about 1 hour each way, but I was single then and had nobody at home waiting for me each day.
Best of luck!