You obviously need to CONFIRM where the property line is before moving forward. Sometimes there will be a " + " mark in the top of the curb at the street that designates the property line; however, you really need a plot plan to get a better idea of how the property lines are laid out (your lot may not be an exact square).
I do not discount the risk of an Adverse Possession issue arising; however, my understanding is that the pending â€œAdverse Possessorâ€ must mark off the property in some obvious way to show the area they are claiming; they have to use or improve the property; you have to pay all property taxes for a minimum of five years.
You can read more about this topic here: http://www.stimmel-law.com/articles/Adverse%20Possession%20-
If your neighbor is encroaching on your property, you should get started right away to straighten things out. If you let it go, after 5 yrs. the property becomes your neighbor''s by what they call "adverse posession". If the property lines are clearly marked, you may just nicely inform your neighbor where the property line is. I would then send him a letter confirming your conversation that he understands he has painted your fence and his 6 X 15 area (I think you said he put concrete down) is actually on your property. If you should end up in court later over a property dispute, you will need more than just your word that you had a conversation with him claiming your property. If he denies that the property is yours, you will probably need to hire a surveyor to establish your property lines.
Good luck to you.