Is the Review scoring system work correctly?

Ryan66
Level 10
Cape Town, South Africa

Is the Review scoring system work correctly?

I call ALL HOSTS who have suffered fake reviews, and even what you perceive as “legitimate” rating.

 

READ THIS.

 

Not only do we as hosts have to deal with false guest reviews, Airbnb are ALSO cheating you out of your hard earned 5 star ratings and REFUSE to acknowledge it and fix it.

 

I am a superhost, and I actually feel panicked every time a gusts leaves me a review, because I am in such fear that they have the power to give me anything below 5 star which I require to keep my superhost status, even though I know everything was perfect, and delivered as promised.

Your status as a superhost is constantly on thin ice, not because you are a bad host, but because you are being cheated by the system.

 

Airbnb’s system, the thing you thought was there to help you, is in fact against you.

 

Here is the proof.

 

When a guest reviews you, they review you based on the SIX (6) sub-categories presented to them which are:

 

  • Accuracy
  • Check-in
  • Cleanliness
  • Communication
  • Location
  • Value

 

Each sub-category is rated out of 5 stars. With a total of 6 sub-categories, this amounts to a total of 30 stars, which is considered 100%. 

 

You’re FINAL rating is based off 5 stars, This means that each FINAL star has a value of 20%, which is the direct influence on your superhost rating. 

THUS:

Lets take a 4 star rating as an example.

 

In order to give a host a FINAL rating of 4 stars, the host would need to be rated at a minimum of 19/30 stars or a maximum of 24/30 stars.

(30stars minus 20% = 24stars or 80% out of 100%)

Anything above this, (25 to 30 stars) should be considered a 5 star FINAL rating.

 

To put the maths of the other FINAL star rating together for you, it SHOULD be as follows:

1 STAR final = 0 to 20% (0 to 6/30 sub-category stars)
2 STAR final = 21 to 40% (7 to 12/30 sub-category stars)
3 STAR final = 41 to 60% (13 to 18/30 sub-category stars)
4 STAR final = 61 to 80% (19 to 24/30 sub-category stars)
5 STAR final = 81 to 100% (25 to 30/30 sub-category stars)

 

 

-------------------------------------------------------------------------------------

 

Lets look at some example cases.

 

In order to be between 19 to 24 total subcategory stars (which would be a 4 star FINAL rating, a guest would have to rate you something like this:

 

  • Accuracy : 4/5
  • Check-in : 5/5
  • Cleanliness : 2/5
  • Communication : 5/5
  • Location : 3/5
  • Value : 4/5

 

4+5+2+5+3+4 = 23/30 stars. The above example review shows that you communicate well, (check-in & communication), you location isn’t the best (not always in our control) and you charge a little bit too much(debatable depending on guest’ s budget and time/season of booking). Mainly you could do better on cleaning the apartment. Not a train smash, just clean up better next time, or make sure that towel doesn’t have a stain on it. You still managed to get a 4 star, which is fair as you worked hard otherwise.

 

In most cases (on my listing) I get something more like

 

  • Accuracy : 5/5
  • Check-in : 5/5
  • Cleanliness : 5/5
  • Communication : 5/5
  • Location : 4/5
  • Value : 5/5

 

5+5+5+5+4+5 = 29/30 stars. Great, so I clearly expect a 5 star review as I achieved 29/30 possible stars.

 

NOPE!!!!!!!!!!! I got a 4 star FINAL RATING! Which caused major damage to my very fragile Superhost status.

 

Why is this? Well because Airbnb’s system is cheating you out of all your hard work and allowing the guest to OVERRIDE all the sub-category reviews with yet ANOHER final review by the guest.

 

Taking my review of 29/30 as an example, I did well in all sub-category fields except location, where he gave me 4 stars. Now in the guests mind, all he remembers is the 4 star for location (human nature) and thus makes his final rating a 4 overall! While by apartment isn’t an ocean view with an infinity pool, its certainly not the BEST location in the world, its certainly not the worst, but rather a general town apartment with buildings around it. But I also never promised that it was an ocean view location.

 

So I can accept a 4/5 star for location. But knowing that its not on a best view location, I also don’t charge as much as I would if it were. Thus the guest giving me 5/5 for value.

I also am a very organized host and strive to deliver the best communication possible. Again I got 5/5 for that.

I always make sure the place is spotless, 5/5 well deserved.

My co-host was there on time for checking and everything went smoothly. 5/5 again.

My photos and listing details, represent exactly what the guest got, again 5/5

 

But now you allow the guest to give yet another “OVERALL” rating which then makes all my other strong 5/5 sub-category scores null and void.

Literally throwing them into the trash, even though I did brilliantly in them. It only remembers the bad and punishes you for it, HARD!

 

This is not fair, Airbnb are stealing your hard earned reviews and stabbing you in the back.

 

---------------------------------------------------------------------

PROOF:

 

Don’t believe me? Go to your https://www.airbnb.com/progress/ratings page and select the “Filter by overall rating” to 4 or below (anything except 5 stars)

 

Now, take any of your ratings, let’s say you filtered 4 stars and calculate up the total sub-category stars you earned. Then compare that to these scoring brackets:

 

1 STAR final = 0 to 20% (0 to 6/30 sub-category stars)
2 STAR final = 21 to 40% (7 to 12/30 sub-category stars)
3 STAR final = 41 to 60% (13 to 18/30 sub-category stars)
4 STAR final = 61 to 80% (19 to 24/30 sub-category stars)
5 STAR final = 81 to 100% (25 to 30/30 sub-category stars)

 

If you achieved from 25 to 30 sub-category stars, you should have gotten a 5 star rating and NOT a FALSE 4 star.

 

I did this with my TEN, 4 star ratings and was in absolute disbelief that out of ten 4 stars reviews, NINE (9), were in fact supposed to be 5 stars!

Yes you read that right, 90% of my so called 4 stars are incorrect. Some guests even gave me 30/30 sub-category stars, but then gave me a 4 star overall rating - This is insane!

 

WOW Airbnb, are you seriously not seeing the VERY BIG issue here?

 

I then even went to my two 3 star reviews, which actually added up and that came up to 23/30 and 28/30 sub-category stars. Meaning that the first  is supposed to be a 4 star and NOT a 3 star, and the second a 5 star and not a 3 star.

 

In fact the first was falsified and by doing by the sub-categories, we can see evidence to this, more of that further down this post.

 

I have put up examples of all these reviews to show you as proof here. https://drive.google.com/drive/folders/1UI2ezQx9LqJ5MEJKxtXUobNAh3OSwWnd?usp=sharing

 

 

THE SOLUTION:

 

Its simple really: REMOVE THE OVERALL RATING

 

Ill repeat this as Airbnb have a knack of conveniently “ignoring” certain parts of what you write to them:

 

REMOVE THE OVERALL RATING - Your sub-category ratings should AUTOMACTICALLY be handling all of this, NOT THE GUEST.

He/she, already left this sub-category reviews based on the 6, INDIVIDUAL area of service delivered to them by the host. The rest is up to your system to calculate and give the Host the final rating, NOT THE GUEST.


--------------------------------------------------------------------------

 

WHAT ELSE WILL THIS SOLVE:

 

Well Airbnb, I don’t know if you have noticed, but your Facebook page is an absolute DISASTER with guests complaining about fake and false reviews, left right and centre.

 

Now please enlighten us as to why you think this is and why you still don’t seem to see an massive problem here?

 

Is it because 90% of all hosts are bad? I think not

Is it because rating a host based on something that wasn’t promised in their listing (let’s say aircon as an example) is fair? I think not.

Is it because a guest should be allowed to review you, even if the never stayed at the apartment – I think not

Is it because your rating system is broken and stealing our hard earned – YES! AS PROVEN.

 

So actually LISTENING for once and removing the very BROKEN OVERALL RATING, will not only make the reviews fair, but also big a big help in filtering FAKE reviews.

 

How you ask?

 

Example 1:

This overall final rating option opens the flood gates to guest, being able to just angry, impulsive 1 star a host, because they weren’t happy with a small thing that ticked them off, let’s say a dusty desk, or a drop in internet for a few hours, even though they gave them a much higher sub-category ratings for the rest. This is an immediate indication of a false review. An overall rating is too much power for some (actually most) people and they abuse the power.

 

Example 2:

Now if a written review says something like: “The apartment had no aircon”, but they gave you a good a 5/5 for value, isn’t this a massive contradiction? I am offering them my place without aircon at a very affordable rate, (which they KNOWINGLY accepted, before booking). If they wanted aircon they should have chosen a place with aircon, most likely PAYING MORE for that listing, compared to mine. Which they don’t want to do. I am aware it doesn’t have aircon which is why my rate is lower. Aircon is a not only a massive upfront cost, but also a big running cost for electricity. I simply cannot provide aircon and the low price point together. I would be running my AIRBNB BUSINESS AT A LOSS. If the guest is happy with the value, then you CANNOT complain about a luxury item, that should be costing them, far more per night. Again a FALSE REVIEW.

 

 

CONCLUSION

 

So Airbnb what are you going to do about this? I have tried to communicate this with your support staff multiple, painstakingly times and just get throw the same copy, paste terms and condition link pages in my face, that have ABSOLUTLYE NOTHING to do with this issue.

Yet you feel you have ‘solved” this HUGE issue.

 

Before signing off, I strongly encourage ALL HOSTS to take this to airbnb’s front doorstep and make them know how unhappy we are about being ripped off of out hard earned reviews.

We spend hundreds of thousands, if not millions on our listings, taking out risky bond payments to buy properties or for massive renovation costs. We invest a huge about of our time to make sure its all up and running to perfection, only to be stabbed in the back by the system we put our trust into.

 

Enough is enough. If you are a superhost you deserve to stay that way. If you are a host wanting to become a superhost, then you deserve to be given your reviews FAIRLY, in order to achieve that. The sub-category rating system if FAIR. NOT the overall rating system.

 

We need to stand together. Facebook isn’t enough, the FB staff are just here to pawn you off with their copy paste reply of “we are sorry to hear that, PM us” with ZERO result in fixing this massive problem at hand.

 

Flood their  https://www.airbnb.com/help/feedback  with what I have written above. We DESERVE our hard earned ratings and FAIR REVIEWS and WILL NOT ACCEPT them being stolen from us any longer.

 

I want my 10 x 5 star reviews that you stole from me. I worked hard for them I deserve them.

 

End of story.

 

 

Aircon

 

I currently have NINE. Yes you read that correct, 9, false 4 star reviews out of my total of 10. These are all victim to these falsely calculated ratings by your broken system. That is 90% 4 stars that should be 5 stars! WOW! Even my only 3 star should in fact be a 4 star.

 

I recommend ALL guests look at your 4 star, even 3 star and below and see what the total sub-category ratings for those were and see just how much airbnb’s system is stabbing you in the back, for your hard deserved work

 

What i also see is that you think is a clever idea to put "accepted reservations" in as a requirement for superhost?

 

Do you actually know how many messages is get from people "booking" fake nights to ask me questions or to promote services? PLENTY

 

Now once again i get penalised for something THEY are doing incorrectly.

 

There should be NO “would you recommend this guest” option for hosts.

72 Replies 72

This is totally true because if a guest has a poor rating, they simply get someone else in their party to book or create another account and start fresh.  Hosts will far more likely take a new account versus a guest booking a property with no history.

Hi Ryan,

I do agree with you. There is no explanation for guests on how the ratings work.

They will give you all 5-star ratings except maybe 4 in one category and then think the overall rating now has to be 4/5.

A guest did exactly this and then rebooked. I questioned him and he said it is confusing!

 

 

 

 

 

A lot of guests press 4* accidentally... 


@Dimitrios-and-Andreas0 wrote:

A lot of guests press 4* accidentally... 


OR that is what they say when you call them out on it.

Murdoch0
Level 2
England, United Kingdom

"Guest could, in fact, give you 1 star overall and give you 5 star in each category. " Indeed so, @Roberta0 - and that's perhaps the central point of Ryan's well-argued post claiming that the review marking system is broken.

 

It makes no sense to request ratings for specific elements deemed relevant and then permit a kind of super rating which trumps the individual ratings. Ratings are largely, not entirely of course, subjective and if something doesn't quite chime - location, for instance - then the guest may give that four stars, perfectly reasonably, but then carry that sense of perhaps slight dissatisfaction forward and think, "Well, it was pretty good but shame about the view so I think 4* is about right" when, in this example, the numeric value is actually 29/30, or 97% rather than 100%, and not the 80% allocated in practice.

 

In your example, if a guest feels that each of the six sub-categories are absolutely first class then if s/he marks the whole experience down to abysmal with a single star rating what is that meant to indicate? And as that overall rating is ordinarily the only one a prospective guest will see do you really think it gives a proper sense of the experience to be expected. And what if it were the other way round - one star for every sub-category but a five star overall rating? Would you, as a guest, feel that the 5* overall was merited? That is, would you feel it entirely right to mark the unit 100% rather than the 20% the individual ratings aggregated to?

@Ryan66

 

You are right and many hosts have made similar comments about the failings of the rating system. There isnt even any proper guidance given to guests about how to score their visit. Guests should at least AS A MINIMUM have an idea of what 5,4,3,2,1 should be guided as.

 

If I have 3 guests staying in my apartment they pay more per person, than if 4 people stay..as the cost of the apartment is the same for up to 4 guests. The 3 guests may view it as less value for money than the 4 guests, even though the apartment,amenities,comfort,services,support etc are exactly the same. Many Superhosts , you will notice, are giving away free, tea,coffee,wine,flowers,milk,airport pick ups and goodness knows what else..just to get their coveted 5 stars... good value for their guests but bringing down their earnings and making them work harder and harder.

 

Location is judged in an illogical way too...as most hosts have pointed out....guests dont really know how they are supposed to be scoring that.

 

I was recently given a 3 star overall rating (my first ever) because i didnt have cable TV, even though I dont tick to say I do. The sub-category ratings were averaged at 4.5... ridiculous 

 

The rating system is primitive and a large number of Superhosts are fully aware of its failings and realise that, at present, it is like trying to push an elephant uphill

 

 

 

 

Hi Roberta,

No, I do not think you are correct with your calcuations. But you have discussed a real problem - that with the rating system of AirBnB.

 

I think the BIG problem is not with calculations, than with the knowledge of guests what and how it is to be rated. Many, many guests think that 5-stars means you have to provide everything as a 5-star HOTEL! And that is the big problem! I had a guest he gave me 4-star for cleanliness, I asked him why - he told me that he gave me very, very good rating, why I should bother at all, it is not 1-star, he told me, it is 4, therefote it is almost 5, you have only to improve A LITTLE as to be as a 5-star hotel! Things like this happened several times with my rating recently - as though as much I provide, as much they want and for lower and lower price!

 

Guests cannot understand that one star less is very important - and I decided from now on to explain

to everyone how important to me is each star and that these stars have nothing to do with the stars hotels have - that EACH one of the rating - these 6 points you mentioned - have to be rated considering VELUE FOR MONEY. Am I right, I am not sure, but I will be pleased to hear other opinions as well.

 

 

Huma0
Level 10
London, United Kingdom

@Lilly0

 

Yes, you're right. Many guests don't understand how inportant that one star difference is and why should they when Airbnb are misinforming them, telling them 4 stars is good and telling us we'll be delisted if we fall below 4.7? It's all a nonsense really.

 

I think the review/rating system might be the most commonly dicussed topic on this forum. It's certainly the one that causes the most stress to hosts.

Hi Ryan,

I do not think your calculations are telling the right thing with the rating system. A week or two ago I wrote here in one of the discussion rooms - that AirBnB has to provide guests with two rating systems. The first one should be made from AirBnB - rating the flats on design, amenities provided and a few other things.   A rating similar to the hotel rating system, but not exactly like it. For example - I cover everything needed to be a 5-star hotel, I looked through the list of things such a hotel must have and found that I am even much better in all but one point - my sq feet of the bathroom are not the needed for a 5-star hotel. And hotels do not provide kitchen amenities, and much more things, flats on AirBnB must have - on the other hand 5-star hotels have many things provided which flats on AirBnB cannot afford - massage, room service, breakfast served in bed, conference rooms and more and more things. And guests rating should come second, so they can do their bookings and ratings knowing in advance what rating that property was given by the system. You see - this is the difference with the hotels - here you should be able to book a 5-star AirBnB on the price of a 2-star hotel or even 1-star.

 

I think that the important thing is for the business AirBnB to understand its proper place - and to defend it with all its srenth, ambition, witt and so on. And this place is NOT TO BE THE CHEAP HOTELS ALTERNATIVE! Once they come to properly articulate the idea of that business - they will be free to do it without being non stop the target of juridical suits on the part of the hotel business.

Jaci0
Level 4
Toronto, CA

Yes, I agree that the overall rating at the end means different things to different people.

For example, the Airbnb accommodation can be rated specifically for

  • Accuracy
  • Check-in
  • Cleanliness
  • Communication
  • Location
  • Value

However, the "Overall experience" is a psychological experience and depends on many possible things out of the hosts' control.  Anything from a week of rainy weather or outside transportation or disagreements with a fellow travel..can come to mind when asked for "Overall Rating. 

Having to design research questionnaires and paradigms from my past profession as a psychologist using an "Overall Rating" is the least accurate and most self-interpreted question.

 

Hosts and their Accommodation are the "product being sold" and we should be listened to as virtually eveyone here is really trying to provide excellent accommodations for their guests. Airbnb should treat us a very important commodity without which there would be no business.

 

 

Howard58
Level 3
Santa Marta, Colombia

I have concurred with most of this thread and your reply has concislely summed up the whole feeling of frustration and sense of impotence that we as hosts (& SuperHosts in my case) feel all the time in the face of guest reviews and the indifference that AirBnB show us.

 

I have maintained a great string of reviews and yet have been left feeling victimised and penalised (true or not, it doesn't matter) by the attitudes of AirBnB when there is an issue with a single guest. Be it chalenging them over drug use, abusive or just plain rude behaviour. You should be allowed to enforce your house rules without feeling that AirBnB are not behind you when the guest gets to try and destroy your reputation with crap reviews simply because they did not like being asked to comply with the very rules that they agreed to when they booked.

 

What are we to do? Ignore it!? That is certainly the feeling that I got from AirBnB when they "investigated" my intervention (non-violent) following a husband beating his wife in one of my rooms. 

 

Thanks for putting it so well and lets hope that AirBnB actually start listening. Sadly that will not happen until they have serious competition and we can vote with oiur business. But for now they have no real competition and get to value the higher commision that they get from guests over hosts.

 

😞

Jaci0
Level 4
Toronto, CA

PS: I lost my Superhost becasue I had guests come and not not heed many of my directions. The communications were difficult as they spoke another language. The arrived with 3 people after booking for only 2. They disobeyed house rules and I had water damage in my home.

I called AirBnb for help and it was very wishy washy advice and in the end they only agreed I should take the higher ground and leave no review.No one returned any calls for insurance after the guests left.

 

(I wish I knew how to get that unfair review removed?)

 

 

 

Hi Jaci,

Often times its the person you are dealing with at Airbnb that determines how your particular situation gets resolved. If you dont get the results you feel you are deservimg with one person take it to another 

person who has more authority and usually more experience in resolving conflict.  I have found this out through experience.

I am about to lose my Superhost I had a guest who I knew would be difficult, I chose not to leave a review yet his 2 star review was made public thereby denying me the opportunity to respond with my rating. It has pulled down my sequence of 4 and 5 star ratings which got me to SuperHost status, I am now at the point where to restore my average I need a sequence of 5 star ratings to restore Superhost status, all because of one malicious person who broke my house rules and did not like it when I mentioned this to him. What is the point of house rules if Airbnb's policy is to always take the guests's viewpoint. Rogue guests should not be able to use ratings as a method of taking revenge for a perceived sleight after they treat someone else's home like a hotel room.  I am thoroughly disappointed that whenever I have contacted Airbnb support I get some vague non-committal response that basically says 'we're not going to do anything'.

@Howard58re no alternative,  booking.com appear to have set up a rival service

Rebecca160
Level 10
Albuquerque, NM

@Ryan66You are correct in that we should be getting rated on all six points that are reviewd and that the Overall experience either should be eliminated entirely, or drop to  being one of seven categories that hosts are ranked upon. (ie., there would be 35 total possible stars divided by seven to get the average rating for a host.) 

I do not know why Airbnb is not listening. I do not know why there scoring system is getting even more ridiculous after July 1. I do believe that many hosts will lose their Superhost status and will leave the platform, unless they change.