Generally speaking the party wall agreement travels with the property meaning if it was properly executed when it was drawn up and then recorded with the county you get it whether you want it or not. It is a governing document to your ownership. If you are considering purchasing a property that has a party wall agreement you may want to consider having an attorney review it for you. The party wall is based on the idea of the wall that connects two properties. It can be very limited in scope and just talk about how you handle maintenance of the actual wall or it can be very broad and cover not only the wall but the roof, the yard, and water if the units only have one tap. It can also create rules like, "one side can not change the exterior of the home in anyway without the other side's written approval"- this can extend not only to paint color and trim- it can be construed to cover things like what color window treatments you have that show on the front of the property. A good party wall agreement not only spells out who does what it also spells out the legal remedies if one party or the other fails to meet their obligations and can allow on e homeowner to place a lien on the other's property. So be sure you know what you are signing up for if you purchase a property with a party wall agreement on it.
Good Luck with your new home purchase!