First if it's in your budget I would move the fence to your property line. If this is not feasible I would suggest visiting your neighbor and ask him to please not to cut your property. Remind him that those 3-4 feet of property belongs to you. By doing so you avoid yourself future property claims.
Best of luck.
Your Local Ecobroker Certified Agent
You need to make sure that you neighbor understands where your property line is and that he understand her shouldn't do anything on your property without your permission. What if you had wanted those trees! Hard to put back!
You need to remind him of the property lines and let him know that you will maintain your property or you can grant him permission to maintain your property.