There are, in some locations, "voluntary home owner associations." Owners are not mandated to join, or pay fees. There are weakly enforced rules that exist in the best interest of the community but there is very little in the form of enforcement.
Your situation sounds like it could be one of these arrangements.
If this is the case, we suggest requesting copies of the association rules and by laws so you will know what guidelines you are NOT following....