If you're talking about a board member for a condo association or Homeowners Association, you need to look in your governing documents. You will have at least 3 documents:
2) Articles of Incorporation
If you do not have a copy of these documents and all amendments you can usually find a copy online at the clerk of circuit court for Broward County.
It usually spelled out specifically how a board member can be removed. For instance my community automatically allows for a board member to be dismissed if they miss 3 "consecutive" board meetings. So as long as a director shows up every 3rd meeting, the only way he can be removed is by a majority vote of the homeowners. Your docs will spell it out. Also the state statutes come into play and may override certain provisions in your governing docs. For example I believe condo laws have recently changed so that a Felon can no longer serve on a condo board. Also the "quorom" requirements for the annual meeting may be lower according to state statutes that your governing docs allow.
Here is an interesting web site about the problems of governing/living in a Condo or HOA:
Hope this helps.