If you can make the changes and leave a one car garage that may be in your best interest. Most clients do want a garage when searching for a home. An additional room is great if you can also put in a closet. By adding a closet you create another bedroom. That can help overcome the garage situation.
Due to our New England weather and the fact that most buyers value a garage for storage and protection of their car, I would recommend keeping at least one of the garages.
You could also talk to your builder to see if there is some way to repostition the garage such as building a new one either in front of your office addtion or as I did move them over . I added a family room where the one car garage was and added a 2 car garage.
I see you list Woodmont (which I assume you mean Milford) which is where I live also where your proximity to your neighbors can be as little as 5 feet so depending on the side and front yard one of these may be an option. Either you or your builder should be able to confirm your options and zoning regs at Town Hall.