Inequity or glitch on ratings?

[Mandrake-And-Karen0]

We have 60 reviews from guests and of those 60 we've had 2-3 reviews that ranked us at 4.5-4.9.  Those reviews were during the first 2 years.  For the last 3 years we have gotten solid 5 star ratings.  We've earned every one of our 5 star reviews that we've gotten. We go above and beyond on cleanliness and the other areas of rankings that are within our control.  Did some research and found neighboring host properties that have lower rankings on cleanliness, location, communication, etc but their overall rating score is the same as or higher than our own.  Our rating score of 4.95 hasn't changed for 2 years despite our recent 3 year consecutive 5 star reviews.  Yeah, yeah, we know 4.95 is a good score.  That's not the point.  We just want to have a score that reflects guest reviews and our hard work and we want the criteria on ratings to be equitably applied to all hosts on the platform.  When I spoke with customer service we were given different answers ranging from they were sending a screen shot to their tech support area to their internal Superhost status award summary.  We were told to just wait until Airbnb updates the ratings every 3 months."  Well each 3 month Superhost review has come and gone for 3 years and our rating has not changed.  Interesting enough Airbnb shows our internal rating score as being 5.0 in the Superhost progress summary!

Wish I could get a simple explanation of how this rating system works.  To date we have not gotten a clear response.  Finally we were told by customer service that they couldn't explain how the ratings system works and they were closing our "ticket" on our concerns.  All active hosts are small business owners who get paid for renting out a room, apartment or house.  We should all have a clear understanding on how ratings work as they affect our businesses- for some in a more positive way than others.  Ratings can affect whether or not hosts get bookings and whether their listings are perceived as having higher or lower value.  It affects pricing and placement.  Why shouldn't we be informed on how the system works that has so much effect on our small businesses? Why don't customer service know how ratings work? Why can't they definitively explain this in a simple, clear way to Superhosts?

Airbnb's ratings-glitch or inequitable rating system? 

Anyone else have this same issue?

[Emilia42]

@Mandrake-And-Karen0 As Mike & Jane said, you need to back through your previous reviews. You received a 3 or 4 overall star rating at some point. 

Also, keep in mind that the Superhost process page displays your rating to the tenth decimal point (ex. 5.0 or 4.8) whereas the overall score on your listing measures to the hundredth decimal point (ex. 4.95 or 4.91)

In your case, 4.95 rounds to 5.0.

[Magic3]

This is not correct. The Superhost rating does measure to the hundredth of a percent. My status has been for the past year stuck at 4.95%. I have received nothing but 5 star reviews for the past year and I have over 850 reviews spanning 10 properties since 2018 when I first started. It's hard for me to believe that in 1 year I have not been able to move from 4.95 up one tick to 4.96.

Something in the algorithm must be wrong. I know it won't affect most guests if I'm 4.95, 4.96, 4.92, whatever. However, once I get the inevitable 4 star review (you can't please all the people all the time)  and I suddenly drop from 4.95 to 4.94 or lower, I'm not going to be too pleased about that. Not because guests care, but, because I care. It may seem trivial. But - if we are being rated and graded for all to see, it would be nice if that grade was actually accurately tabulated. Maybe go over even one more decimal point - maybe I'm actually a 4.959 rated host 🙂 

[Pat271]

You really do have to count up all the reviews and do the division yourself. That should produce the same result as the algorithm used to display the ratings to the guests. Everything else on the Insights/Quality/Overall Quality page is doing rounding, as @Emilia42 mentioned.

Like I implied before, it can be argued whether this is a good way to manage the ratings.

For instance, I think the reviews should be weighted. Hosts improve over time, and sometimes also degrade over time, so IMO the most recent reviews should carry more weight than the oldest reviews.

It’s also common practice in statistics algorithms to eliminate outliers. So if there is 1 very bad review amongst dozens of 5-star reviews, the bad review would simply not be included in the calculation.

You could also argue that the scores should be averaged to 1 or less decimal places, or that guests should be able to assign ratings more precise than just whole numbers. If they are going to display 2 decimal places, maybe guests should be able to assign 2 decimal places, although it might be a little daft to assign ratings like 4.73…

Anyway, the algorithm is what it is, and CS is unlikely to be able to spearhead any kind of change. It’s good to call them though, and write about it here, so that if enough people voice their dissatisfaction, it will receive more attention from the folks that can actually implement improvements. 

[Robin4]

@Mandrake-And-Karen0 

Whether we like it or not, it is accurate.... @Pat271  is correct with his method.

With my Airbnb hosting over the past 6 years I have been given 510 reviews.....

470 x 5 star reviews.

  36 x 4 star reviews.

     4 x 3 star reviews.

That gives me a total of 2,506 overall stars.

2,506 divided by 510 reviews gives me a rating of 4.91372

So my overall rating of 4.91 by that formula is correct! It hasn't got anything to do with what you have achieved over the past 2 years, it's a simple calculation based on your overall stats!

Once you get less than an overall 5 just twice, that will be reflected in your overall ranking for some hundreds of reviews!

When I say 'whether we like it or not'  what I mean is, of those 36 x 4 star reviews, 17 of those have been 5 stars in every category but, got a 4 star overall, so although it must be said that some of those 5 overalls that I got had the odd 4 star for one of the categories, that isn't relevant because it is the overall star rating that affects the ranking that is achieved.....but we live with it! 

Cheers.......Rob

[David11711]

I'll throw a wrench into this discussion. AirBNB currently has a known glitch that causes average ratings to miscalculate for some unknown reason. Our listing was impacted by this.

We have five reviews: four 5s and one 4. The basic mathematical average is (5+5+5+5+4) / 5 = 4.8. However, our listing is "stuck" at 4.7. AirBNB support says there is no way for them to correct the average manually while their engineering team investigates the issue.

Ironically, this was the second time we reported the issue to them on the same listing. The first time was after we received our third review. In this case, our first two reviews were a 5 and 4 (averages to 4.5). Our listing correctly showed 4.5 at the time. Then, our third review came in with a 5 score. That averages to (5+4+5) / 3 = 4.67 (rounds up to 4.7). However, our score still showed 4.5 on the listing. After contacting support the first time, they acknowledged the "bug" and manually updated our score from 4.5 to 4.7.

But now, with this second averaging issue where the score should be 4.8 instead of 4.7, they now insist they have no way of manually changing averages in the back end. Well that seems oddly false considering they already did it once, and now they suddenly have no way of updating the score a second time.

Overall, AirBNB support basically fails every time you give them something to fix, no matter how simplistic or low-hanging.

Just wanted to add this comment to the mix, as there can actually be an instance where AirBNB miscalulates an average score due to some "bug" that their "engineering team is diligently working on".

[Sandra856]

Hi @David11711 🌿

I don’t see any listings with only 5 reviews?

I don’t know if you know but the listing itself will show the average of all reviews given since the listings start (overall ratings). When you look at your super-host dashboard it will only show average of overall ratings given within the last year. But it is difficult to understand or help if it isn’t possible to see the listing. I can only see 2 listings with a lot of reviews.