I commuted many years to this area when Hwy 101 was only 2 lanes. It is now 4 lanes, and so if you drive, you will have no problem getting into San Jose - commute times are typically at the speed limit all the way to San Jose. Once in San Jose, you'll have the regular potential back up areas you'd have regardless of where you are coming from. If you are driving alone, you'll see a slow down from north of Blossom Hill to Tulley. It will ease from Tulley to just North of 280/680 where it will slow down again until the Oakland Road exit. It is sometimes slow from just North of 880 to the airport. Then clear sailing. If you are two or more in the vehicle it is clear sailing though it will slow to 35 or 45 mph occasionally along the above mentioned areas. It is not worth it from a time perspective to take small roads from Gilroy, though it might be more pleasant, especially when the flowers are out. Choices are Monterey or Hale north of Morgan Hill (some take Santa Teresa to Morgan Hill). You can jump over to Watsonville Road and then Uvas and come into San Jose via Almaden Valley - but it will be VERY slow getting from Almaden to Carribean Dr. (slower than the whole commute from Gilroy along 101). The fastest way will always be 101 unless there is an accident. Once in San Jose, there are side roads that can be taken when 101 is completely blocked - they are a little complex - a little too complex to explain here - but they do work if 101 in San Jose comes to a dead stop. Going via 680 will normally not save you time.
The commute from the South Bay is one of the best around - I highly recommend it. The train service is a reasonable alternative and many high tech companies participate in free shuttles to and from their offices - check with your HR department. I know many Silicon Valley engineers who live even further south (i.e. Hollister) and who are happy with their commute.
Regards ... Wayne