If you defy the city, they will eventually put a lien of sub-standard condition on your property title.. and they will refer you to the District Attorney for prosecution... they will charge you all kinds of fines, etc. and make you comply after all those charges you incurred.
If you feel super-strongly about your case and you want to spend lots of money to prove you are right, then you can hire an experience attorney (if you contact me I can refer you to one) and fight the city from the beginning... but it will not be cheap, and you are likely to lose, since they have more money and staff attorneys. Do not ignore the issue as it will not go away.
Best of luck!
BTW, I am a local broker and general contractor, if you want a more specific answer contact my office directly.
Ron Escobar, MBA
Broker & General Contractor
DRE 01460645 CSLB 917824