It's so interesting that you would bring this up, as I was just talking to a friend who owns property in Manhattan Beach with the exact problem. Yes it is annoying, but buyers will not automatically disregard your home because of it.
I would think of it this way, given two homes that are exactly the same, they might pick the one without the problem instead. Unless...the one with the pole..is lower in price!
My friend is going to work with the city, and the neighbor to have that pole removed, but it will be expensive for her.