Is the Review scoring system work correctly?

[Ryan66]

I call ALL HOSTS who have suffered fake reviews, and even what you perceive as “legitimate” rating.

READ THIS.

Not only do we as hosts have to deal with false guest reviews, Airbnb are ALSO cheating you out of your hard earned 5 star ratings and REFUSE to acknowledge it and fix it.

I am a superhost, and I actually feel panicked every time a gusts leaves me a review, because I am in such fear that they have the power to give me anything below 5 star which I require to keep my superhost status, even though I know everything was perfect, and delivered as promised.

Your status as a superhost is constantly on thin ice, not because you are a bad host, but because you are being cheated by the system.

Airbnb’s system, the thing you thought was there to help you, is in fact against you.

Here is the proof.

When a guest reviews you, they review you based on the SIX (6) sub-categories presented to them which are:

• Accuracy
• Check-in
• Cleanliness
• Communication
• Location
• Value

Each sub-category is rated out of 5 stars. With a total of 6 sub-categories, this amounts to a total of 30 stars, which is considered 100%. 

You’re FINAL rating is based off 5 stars, This means that each FINAL star has a value of 20%, which is the direct influence on your superhost rating. 

THUS:

Lets take a 4 star rating as an example.

In order to give a host a FINAL rating of 4 stars, the host would need to be rated at a minimum of 19/30 stars or a maximum of 24/30 stars.

(30stars minus 20% = 24stars or 80% out of 100%)

Anything above this, (25 to 30 stars) should be considered a 5 star FINAL rating.

To put the maths of the other FINAL star rating together for you, it SHOULD be as follows:

1 STAR final = 0 to 20% (0 to 6/30 sub-category stars)
2 STAR final = 21 to 40% (7 to 12/30 sub-category stars)
3 STAR final = 41 to 60% (13 to 18/30 sub-category stars)
4 STAR final = 61 to 80% (19 to 24/30 sub-category stars)
5 STAR final = 81 to 100% (25 to 30/30 sub-category stars)

-------------------------------------------------------------------------------------

Lets look at some example cases.

In order to be between 19 to 24 total subcategory stars (which would be a 4 star FINAL rating, a guest would have to rate you something like this:

• Accuracy : 4/5
• Check-in : 5/5
• Cleanliness : 2/5
• Communication : 5/5
• Location : 3/5
• Value : 4/5

4+5+2+5+3+4 = 23/30 stars. The above example review shows that you communicate well, (check-in & communication), you location isn’t the best (not always in our control) and you charge a little bit too much(debatable depending on guest’ s budget and time/season of booking). Mainly you could do better on cleaning the apartment. Not a train smash, just clean up better next time, or make sure that towel doesn’t have a stain on it. You still managed to get a 4 star, which is fair as you worked hard otherwise.

In most cases (on my listing) I get something more like

• Accuracy : 5/5
• Check-in : 5/5
• Cleanliness : 5/5
• Communication : 5/5
• Location : 4/5
• Value : 5/5

5+5+5+5+4+5 = 29/30 stars. Great, so I clearly expect a 5 star review as I achieved 29/30 possible stars.

NOPE!!!!!!!!!!! I got a 4 star FINAL RATING! Which caused major damage to my very fragile Superhost status.

Why is this? Well because Airbnb’s system is cheating you out of all your hard work and allowing the guest to OVERRIDE all the sub-category reviews with yet ANOHER final review by the guest.

Taking my review of 29/30 as an example, I did well in all sub-category fields except location, where he gave me 4 stars. Now in the guests mind, all he remembers is the 4 star for location (human nature) and thus makes his final rating a 4 overall! While by apartment isn’t an ocean view with an infinity pool, its certainly not the BEST location in the world, its certainly not the worst, but rather a general town apartment with buildings around it. But I also never promised that it was an ocean view location.

So I can accept a 4/5 star for location. But knowing that its not on a best view location, I also don’t charge as much as I would if it were. Thus the guest giving me 5/5 for value.

I also am a very organized host and strive to deliver the best communication possible. Again I got 5/5 for that.

I always make sure the place is spotless, 5/5 well deserved.

My co-host was there on time for checking and everything went smoothly. 5/5 again.

My photos and listing details, represent exactly what the guest got, again 5/5

But now you allow the guest to give yet another “OVERALL” rating which then makes all my other strong 5/5 sub-category scores null and void.

Literally throwing them into the trash, even though I did brilliantly in them. It only remembers the bad and punishes you for it, HARD!

This is not fair, Airbnb are stealing your hard earned reviews and stabbing you in the back.

---------------------------------------------------------------------

PROOF:

Don’t believe me? Go to your https://www.airbnb.com/progress/ratings page and select the “Filter by overall rating” to 4 or below (anything except 5 stars)

Now, take any of your ratings, let’s say you filtered 4 stars and calculate up the total sub-category stars you earned. Then compare that to these scoring brackets:

1 STAR final = 0 to 20% (0 to 6/30 sub-category stars)
2 STAR final = 21 to 40% (7 to 12/30 sub-category stars)
3 STAR final = 41 to 60% (13 to 18/30 sub-category stars)
4 STAR final = 61 to 80% (19 to 24/30 sub-category stars)
5 STAR final = 81 to 100% (25 to 30/30 sub-category stars)

If you achieved from 25 to 30 sub-category stars, you should have gotten a 5 star rating and NOT a FALSE 4 star.

I did this with my TEN, 4 star ratings and was in absolute disbelief that out of ten 4 stars reviews, NINE (9), were in fact supposed to be 5 stars!

Yes you read that right, 90% of my so called 4 stars are incorrect. Some guests even gave me 30/30 sub-category stars, but then gave me a 4 star overall rating - This is insane!

WOW Airbnb, are you seriously not seeing the VERY BIG issue here?

I then even went to my two 3 star reviews, which actually added up and that came up to 23/30 and 28/30 sub-category stars. Meaning that the first  is supposed to be a 4 star and NOT a 3 star, and the second a 5 star and not a 3 star.

In fact the first was falsified and by doing by the sub-categories, we can see evidence to this, more of that further down this post.

I have put up examples of all these reviews to show you as proof here. https://drive.google.com/drive/folders/1UI2ezQx9LqJ5MEJKxtXUobNAh3OSwWnd?usp=sharing

THE SOLUTION:

Its simple really: REMOVE THE OVERALL RATING

Ill repeat this as Airbnb have a knack of conveniently “ignoring” certain parts of what you write to them:

REMOVE THE OVERALL RATING - Your sub-category ratings should AUTOMACTICALLY be handling all of this, NOT THE GUEST.

He/she, already left this sub-category reviews based on the 6, INDIVIDUAL area of service delivered to them by the host. The rest is up to your system to calculate and give the Host the final rating, NOT THE GUEST.

--------------------------------------------------------------------------

WHAT ELSE WILL THIS SOLVE:

Well Airbnb, I don’t know if you have noticed, but your Facebook page is an absolute DISASTER with guests complaining about fake and false reviews, left right and centre.

Now please enlighten us as to why you think this is and why you still don’t seem to see an massive problem here?

Is it because 90% of all hosts are bad? I think not

Is it because rating a host based on something that wasn’t promised in their listing (let’s say aircon as an example) is fair? I think not.

Is it because a guest should be allowed to review you, even if the never stayed at the apartment – I think not

Is it because your rating system is broken and stealing our hard earned – YES! AS PROVEN.

So actually LISTENING for once and removing the very BROKEN OVERALL RATING, will not only make the reviews fair, but also big a big help in filtering FAKE reviews.

How you ask?

Example 1:

This overall final rating option opens the flood gates to guest, being able to just angry, impulsive 1 star a host, because they weren’t happy with a small thing that ticked them off, let’s say a dusty desk, or a drop in internet for a few hours, even though they gave them a much higher sub-category ratings for the rest. This is an immediate indication of a false review. An overall rating is too much power for some (actually most) people and they abuse the power.

Example 2:

Now if a written review says something like: “The apartment had no aircon”, but they gave you a good a 5/5 for value, isn’t this a massive contradiction? I am offering them my place without aircon at a very affordable rate, (which they KNOWINGLY accepted, before booking). If they wanted aircon they should have chosen a place with aircon, most likely PAYING MORE for that listing, compared to mine. Which they don’t want to do. I am aware it doesn’t have aircon which is why my rate is lower. Aircon is a not only a massive upfront cost, but also a big running cost for electricity. I simply cannot provide aircon and the low price point together. I would be running my AIRBNB BUSINESS AT A LOSS. If the guest is happy with the value, then you CANNOT complain about a luxury item, that should be costing them, far more per night. Again a FALSE REVIEW.

CONCLUSION

So Airbnb what are you going to do about this? I have tried to communicate this with your support staff multiple, painstakingly times and just get throw the same copy, paste terms and condition link pages in my face, that have ABSOLUTLYE NOTHING to do with this issue.

Yet you feel you have ‘solved” this HUGE issue.

Before signing off, I strongly encourage ALL HOSTS to take this to airbnb’s front doorstep and make them know how unhappy we are about being ripped off of out hard earned reviews.

We spend hundreds of thousands, if not millions on our listings, taking out risky bond payments to buy properties or for massive renovation costs. We invest a huge about of our time to make sure its all up and running to perfection, only to be stabbed in the back by the system we put our trust into.

Enough is enough. If you are a superhost you deserve to stay that way. If you are a host wanting to become a superhost, then you deserve to be given your reviews FAIRLY, in order to achieve that. The sub-category rating system if FAIR. NOT the overall rating system.

We need to stand together. Facebook isn’t enough, the FB staff are just here to pawn you off with their copy paste reply of “we are sorry to hear that, PM us” with ZERO result in fixing this massive problem at hand.

Flood their  https://www.airbnb.com/help/feedback  with what I have written above. We DESERVE our hard earned ratings and FAIR REVIEWS and WILL NOT ACCEPT them being stolen from us any longer.

I want my 10 x 5 star reviews that you stole from me. I worked hard for them I deserve them.

End of story.

Aircon

I currently have NINE. Yes you read that correct, 9, false 4 star reviews out of my total of 10. These are all victim to these falsely calculated ratings by your broken system. That is 90% 4 stars that should be 5 stars! WOW! Even my only 3 star should in fact be a 4 star.

I recommend ALL guests look at your 4 star, even 3 star and below and see what the total sub-category ratings for those were and see just how much airbnb’s system is stabbing you in the back, for your hard deserved work

What i also see is that you think is a clever idea to put "accepted reservations" in as a requirement for superhost?

Do you actually know how many messages is get from people "booking" fake nights to ask me questions or to promote services? PLENTY

Now once again i get penalised for something THEY are doing incorrectly.

There should be NO “would you recommend this guest” option for hosts.

[Judy29]

I hear you loud and clear and feel the same frustrations.  My latest review was five star on all six questions and yet it came through on the review as a four star.  I did loose my superhost and when competing with 27 other accommodations (most of which have superstar status), it is difficult to up the quality of the accommodations when already jumping through hoops to make sure the house is bleached germ free and vacuumed till the new carpet is sucked off the floor, lawn maintenance is prestine and there is fresh fruit and bottled water left for the guests.

My current guest has had major difficulties getting connected to our hi speed wifi.  I called the provider to trouble shoot at the onset, had a technision come immediately the next morning to fix the issue.  Nothing was wrong.  So it must be something with the guests  system set up and has to be reset to become compatible.  I even offered her our top accommodations which is almost twice the space with many more luxury features...and her systems compatible to the wifi.  She politely refused to move.  Im waiting for her to leave now (which is one day earlier than booked and she wants to be refunded for her final day).    As a host, have I deserved a less than five star? 

[Syl11]

Thanks so much Ryan for your detailed examples and showing this so clearly, its so great that finally someone can explain it with the percentages. If this gets addessed it will finally stop the upset superhosts and its all the more positive PR blublic relations exercise for all involved.

I know this will get addressed.

Cheers and enjoy hosting! thanks

[Patricia1008]

Ryan, your examples clearly demonstrate that the review process and assignment of " stars ' status for Superhost designation is neither objective nor statistically accurate.  The rather arbitrary method of evaluating and assigning standards of excellecence upon which hosts income from bookings is based opens a whole can of worms for Airbnb and for guests who themselvest rate far less than 5 stars. Loss of Superhost status ( and income )to statistically skewed evaluation methodology can spell major liability for Airbnb.  Each guest and each host is different.  I just discharged/vacated a guest from the nether world after graciously extending her stay an extra day so she could locate other accommodations and not just end up evicted with her multiple huge suitcases tossed on the street.  Yes, I got paid a whopping $41 for the extra day which entailed over 4 hours of dealing with her issues with a case manager, missing two mornings of work, staying up well past midnight dealing with her issues, extra housekeeping expenses, stinking cooking burned food odors throught my house, burned food in pots and pans lefts unwashed, and a nearly new bedspred with melt marks where a laptop had been left running on the bed, failure to observe house rules and me finally having to threaten to get the police and charge her with tresspass if she didn't leave by checkout time.  What sort of rating do you suppose I will get from this guest ?  Do you think her evaluation of my accommodations will cause me to loose my Superhost status? I've documented all of my communications with her and Airbnb. Her abherrant behavoir caused me additional stress and expense.  This experience caused me to block off my rooms to recover, be hesitant about accepting instant bookings at all and caused me to substantially raise my prices to keep out the riff Raff who may think they can sqwat here, do it hatever they choose to damage my home, my health and not pay for their stay here.  The Airbnb case manager told me she was going to PAY the one night extension for the guest who kept claiming her check had not been c edited to her account a week after it was deposited, but slipped up mid week indicating her bank showed her balance of $1000 but the check had still not been credited ?? 

So now I'm wondering what Airbnb might decide to do to punish me for not wanting to keep this rude trroublesome guest any longer than necessary having given her the required 3 day notice to vacate before having her physically removed.  

[Alexa45]

Personally I think the answer is an Uber style model where guests are in a "SuperGuest" program and host reviews of them impact their ability to book on the site and affords 'SuperGuests' some advantages as well.  This would level the playing field, having both parties invested in accurate reviews.   Would also help hosts avoid some of the more 'troublesome' guests.  

I also agree that an 'olympic' approach to scoring dropping the highest and lowest and calculating from there makes sense.    It's not a level playing field when host status reviews are quarterly, but the ratings on which those are calculated are lifetime.  

I'm going to try and get over losing my status in this last round of reviews - and just keep on doing what I'm doing.  

[Leonard1]

SuperGuest program, that is a good idea.

[Clint-and-Kendall0]

Hi Ryan,

Thanks for your hard work and great research. I too agree that it does seem to be a bit unfair that the very last questions that guest get asked is their overall rating, because as you pointed out, it's human nature to think that if you just got done leaving a 4 star rating any of the categories whatsoever, then you must have to leave a 4-star overall rating.

However, I should point out (which you very likely may already know), Airbnb does point out on many listings if you have been 5-star rated for specific sub-categories, as in the screenshot below that I took from a fellow host's listing.

So at least those subcategory ratings don't get completely wasted and can be beneficial to the host.

I think possibly the best solution would be to move the overall rating to the beginning of the rating process instead of the end.

Thanks so much for helping to educate all of us on how the review process works. It definitely was an eye-opener for me and I learned some things that I didn't fully understand before.

[Carl-and-Pandora0]

@Clint-and-Kendall0

The subcategory ratings aren't always accurate. On our listing it states that 95% of guests have rated us five stars for location when in fact we have 100% five star ratings for location. Called ABB about this and after going back and forth and back and forth their official answer ended up being "I don't know". 

[Jennifer976]

One of my recent guests gave me 5 stars on everything and i got 4 on overall?  How does that work?  Does the guest choose the overall rating and maybe she made a mistake or does Airbnb calculate it?

[Lawrene0]

The guest chooses the overall rating, @Jennifer976. Sometimes they give five stars in all the categories, but choose four for the overall rating because of something fair like the host was grumpy, or something unfair like the weather was bad or there were mosquitoes outside. How was your weather that week? It is often that - a rating of their holiday in general. Irksome, but probably not a mistake. 

[Jennifer976]

Weather was great and she left really nice public and private comments.  I did reply asking her what I can do to improve but didn't hear back??

[Sarah977]

@Jennifer976  @Lawrene0   Guests are now (in some areas of the world but not others, according to posts here) being shown a screen which doesn't have star ratings for the overall experience attached to it at all. Instead it just has text and check boxes. So a guest doesn't realize that that translates into Overall star ratings for the host. 

"Exceeded my expectations" (would be a 5* for us)

"A bit better than my expectations" (4*s)

"Met my expectations" (3*s)

As you can see, a guest who gives you 4*s overall has said that your listing was a bit better than their expectations. Nothing wrong with that, except where airbnb insanity is concerned. Unbelievably untransparent and sneaky.

[Lawrene0]

I have seen that screen in the past few months as a guest, @Sarah977, but, at least here, I still had to enter a star rating. It was an "extra" screen. My dilemma, when I first saw it, was what to say. I thought if I said better than expected, the host would get a chiding email for not being accurate. But that's not the case. Better than expected is the right answer 🙂

[Fred13]

I am trying to guess the IQ of the 'brainchild' that would think 'A bit better...' is a predictable evaluation. Airbnb needs to get rid of the 'children' working there and constantly coming up with these childish ideas. 

[Edgar110]

Hello,

im new to AIRBNB its my first year, but I totally agree with you on this, the rating system is broken!

I just got 5 stars in all categories except on location i got a 4, then my final score is... 4 stars!? Really? This is so unfair and i feel depressed over this. 

- Accuracy 5

- Check in 5

- Cleaning 5

- Value 5

- Location 4

- Communication 5

final score = 4 stars 

this is sad 

[Magdalena54]

Agree, this is quite annoying. On top you are always subject to the host perception which is quite subjective. I host a private room in Geneva which is one of the most expensive cities in the world.

The cost of renting or buying an apartment here is of course not comparable with most other cities in the world, and the rate I charge for a private room is less than 5 times the cost for a hotel room.

I get reviews saying their experience in my apartment has been the best in Geneva, top of the top, perfect treatment, perfect cleanliness, etc but I had a couple of people rating the value for money in less than 5 stars, every other detail of the place (cleanliness, accuracy, etc being 5 stars in all).

I'm sorry, but the cost of living in Geneva is higher than you would expect in your hometown, and other rooms in Airbnb which may be cheaper look like a dumpster, that's why you rented mine. How is it fair that you give me a less than 5 star review on that account ??