So sad that this situation is making everyone miserable and on edge! No one needs this kind of stress in their lives.
Are you current on your license and permit with your city or county? That can make a big difference in how to approach such a situation.
Here we have public hearings where neighbors can have their say, before a permit is issued. This process makes it possible for everyone to feel heard, and not ambushed. All the various agencies get to investigate for potential impacts on the neighborhood, also considering public health and safety. We have some reasonable restrictions on our hosting, but they make total sense, so we don't consider it problematic.
That does not mean we will never have any problems here, but so far so good. By the time we'd had our public hearing we had learned so much, and the neighbors all know what we are doing. We even had one show up with a big truck to get a guest's car out of a muddy ditch. Yay!
Legal advice is good, imho, professional mediation may be the solution, as less adversarial. I hope this resolves well, and peace is restored to the neighborhood. Best wishes to you!