From what I'm reading here in the Community Center, many of us Super Hosts are feeling just like you.
Recently I counted all of my stars in every single category ('overall' plus the six sub-categories). Out of 70 reviews in all 7 categories, I had only 14 reviews out of all of the categories in which I did not have a five star review. Several of these less than five star reviews were from the same odd guest who did not read my listing or photo descriptions adequately and he thought my large sunroom was a 'tiny house' and was upset he had to sleep 'in the large cottage'. So he gave me 3's on nearly everything, but never said a word while he was here. So I have a supposed 93% five star rating, even though only 14 categories out of 482 possible categories were less than five stars. This confused me, so I called Airbnb one night to discuss.
The Customer Rep I spoke to couldn't figure the math out either, so he put me on hold and went to his manager. He eventually returned and told me this: It seems the only rating that 'counts' is the 'Overall' rating. So I had (at that time when calling Airbnb) 64 five star Overall ratings / reviews, and only 5 that were not 5 star (all five of these were 4 star reviews - even the 'tiny house' guy gave me a 4 star review overall, strangely. I had no 3 stars or less 'overall' reviews).
I did not realize that none of those sub-category reviews count towards our overall average. Also, Airbnb is not using a 'weighted average' to calculate our average, which makes no sense to me. At this point, given that 'declines' are also now counted against us, I am prepared to lose the Super Host status I worked so very hard for all this past year, during which time I sold out the Cottage every month and was likely Airbnb's top producer in my beach vacation rental neighborhood. C'est la vie. I will not have anyone staying on my property that clearly may break my rules or put me and mine at risk. No thanks. And I will still provide just as good an experience to whomever rents our Cottage, with our without my Super Host status, whether these guests are from Airbnb or not. Because it is MY home and MY rental and only I have the right (and ability) to determine who is appropriate for me to share it with, and I provide the best possible experiences to my guests because that is just who I am and how I do things. And so I will never be controlled by these tactics currently being used by the large booking sites that are designed to make owners treat their homes like commercial hotels via setting things up so that we do things their way or lose bookings / revenue and / or (in the case of Airbnb) our Super Host status. Because my property and my home are my sanctuary, and I will never forfeit my right to screen and vet potential guests who seek to stay here. Ever.