Is the Review scoring system work correctly?

[Ryan66]

I call ALL HOSTS who have suffered fake reviews, and even what you perceive as “legitimate” rating.

READ THIS.

Not only do we as hosts have to deal with false guest reviews, Airbnb are ALSO cheating you out of your hard earned 5 star ratings and REFUSE to acknowledge it and fix it.

I am a superhost, and I actually feel panicked every time a gusts leaves me a review, because I am in such fear that they have the power to give me anything below 5 star which I require to keep my superhost status, even though I know everything was perfect, and delivered as promised.

Your status as a superhost is constantly on thin ice, not because you are a bad host, but because you are being cheated by the system.

Airbnb’s system, the thing you thought was there to help you, is in fact against you.

Here is the proof.

When a guest reviews you, they review you based on the SIX (6) sub-categories presented to them which are:

• Accuracy
• Check-in
• Cleanliness
• Communication
• Location
• Value

Each sub-category is rated out of 5 stars. With a total of 6 sub-categories, this amounts to a total of 30 stars, which is considered 100%. 

You’re FINAL rating is based off 5 stars, This means that each FINAL star has a value of 20%, which is the direct influence on your superhost rating. 

THUS:

Lets take a 4 star rating as an example.

In order to give a host a FINAL rating of 4 stars, the host would need to be rated at a minimum of 19/30 stars or a maximum of 24/30 stars.

(30stars minus 20% = 24stars or 80% out of 100%)

Anything above this, (25 to 30 stars) should be considered a 5 star FINAL rating.

To put the maths of the other FINAL star rating together for you, it SHOULD be as follows:

1 STAR final = 0 to 20% (0 to 6/30 sub-category stars)
2 STAR final = 21 to 40% (7 to 12/30 sub-category stars)
3 STAR final = 41 to 60% (13 to 18/30 sub-category stars)
4 STAR final = 61 to 80% (19 to 24/30 sub-category stars)
5 STAR final = 81 to 100% (25 to 30/30 sub-category stars)

-------------------------------------------------------------------------------------

Lets look at some example cases.

In order to be between 19 to 24 total subcategory stars (which would be a 4 star FINAL rating, a guest would have to rate you something like this:

• Accuracy : 4/5
• Check-in : 5/5
• Cleanliness : 2/5
• Communication : 5/5
• Location : 3/5
• Value : 4/5

4+5+2+5+3+4 = 23/30 stars. The above example review shows that you communicate well, (check-in & communication), you location isn’t the best (not always in our control) and you charge a little bit too much(debatable depending on guest’ s budget and time/season of booking). Mainly you could do better on cleaning the apartment. Not a train smash, just clean up better next time, or make sure that towel doesn’t have a stain on it. You still managed to get a 4 star, which is fair as you worked hard otherwise.

In most cases (on my listing) I get something more like

• Accuracy : 5/5
• Check-in : 5/5
• Cleanliness : 5/5
• Communication : 5/5
• Location : 4/5
• Value : 5/5

5+5+5+5+4+5 = 29/30 stars. Great, so I clearly expect a 5 star review as I achieved 29/30 possible stars.

NOPE!!!!!!!!!!! I got a 4 star FINAL RATING! Which caused major damage to my very fragile Superhost status.

Why is this? Well because Airbnb’s system is cheating you out of all your hard work and allowing the guest to OVERRIDE all the sub-category reviews with yet ANOHER final review by the guest.

Taking my review of 29/30 as an example, I did well in all sub-category fields except location, where he gave me 4 stars. Now in the guests mind, all he remembers is the 4 star for location (human nature) and thus makes his final rating a 4 overall! While by apartment isn’t an ocean view with an infinity pool, its certainly not the BEST location in the world, its certainly not the worst, but rather a general town apartment with buildings around it. But I also never promised that it was an ocean view location.

So I can accept a 4/5 star for location. But knowing that its not on a best view location, I also don’t charge as much as I would if it were. Thus the guest giving me 5/5 for value.

I also am a very organized host and strive to deliver the best communication possible. Again I got 5/5 for that.

I always make sure the place is spotless, 5/5 well deserved.

My co-host was there on time for checking and everything went smoothly. 5/5 again.

My photos and listing details, represent exactly what the guest got, again 5/5

But now you allow the guest to give yet another “OVERALL” rating which then makes all my other strong 5/5 sub-category scores null and void.

Literally throwing them into the trash, even though I did brilliantly in them. It only remembers the bad and punishes you for it, HARD!

This is not fair, Airbnb are stealing your hard earned reviews and stabbing you in the back.

---------------------------------------------------------------------

PROOF:

Don’t believe me? Go to your https://www.airbnb.com/progress/ratings page and select the “Filter by overall rating” to 4 or below (anything except 5 stars)

Now, take any of your ratings, let’s say you filtered 4 stars and calculate up the total sub-category stars you earned. Then compare that to these scoring brackets:

1 STAR final = 0 to 20% (0 to 6/30 sub-category stars)
2 STAR final = 21 to 40% (7 to 12/30 sub-category stars)
3 STAR final = 41 to 60% (13 to 18/30 sub-category stars)
4 STAR final = 61 to 80% (19 to 24/30 sub-category stars)
5 STAR final = 81 to 100% (25 to 30/30 sub-category stars)

If you achieved from 25 to 30 sub-category stars, you should have gotten a 5 star rating and NOT a FALSE 4 star.

I did this with my TEN, 4 star ratings and was in absolute disbelief that out of ten 4 stars reviews, NINE (9), were in fact supposed to be 5 stars!

Yes you read that right, 90% of my so called 4 stars are incorrect. Some guests even gave me 30/30 sub-category stars, but then gave me a 4 star overall rating - This is insane!

WOW Airbnb, are you seriously not seeing the VERY BIG issue here?

I then even went to my two 3 star reviews, which actually added up and that came up to 23/30 and 28/30 sub-category stars. Meaning that the first  is supposed to be a 4 star and NOT a 3 star, and the second a 5 star and not a 3 star.

In fact the first was falsified and by doing by the sub-categories, we can see evidence to this, more of that further down this post.

I have put up examples of all these reviews to show you as proof here. https://drive.google.com/drive/folders/1UI2ezQx9LqJ5MEJKxtXUobNAh3OSwWnd?usp=sharing

THE SOLUTION:

Its simple really: REMOVE THE OVERALL RATING

Ill repeat this as Airbnb have a knack of conveniently “ignoring” certain parts of what you write to them:

REMOVE THE OVERALL RATING - Your sub-category ratings should AUTOMACTICALLY be handling all of this, NOT THE GUEST.

He/she, already left this sub-category reviews based on the 6, INDIVIDUAL area of service delivered to them by the host. The rest is up to your system to calculate and give the Host the final rating, NOT THE GUEST.

--------------------------------------------------------------------------

WHAT ELSE WILL THIS SOLVE:

Well Airbnb, I don’t know if you have noticed, but your Facebook page is an absolute DISASTER with guests complaining about fake and false reviews, left right and centre.

Now please enlighten us as to why you think this is and why you still don’t seem to see an massive problem here?

Is it because 90% of all hosts are bad? I think not

Is it because rating a host based on something that wasn’t promised in their listing (let’s say aircon as an example) is fair? I think not.

Is it because a guest should be allowed to review you, even if the never stayed at the apartment – I think not

Is it because your rating system is broken and stealing our hard earned – YES! AS PROVEN.

So actually LISTENING for once and removing the very BROKEN OVERALL RATING, will not only make the reviews fair, but also big a big help in filtering FAKE reviews.

How you ask?

Example 1:

This overall final rating option opens the flood gates to guest, being able to just angry, impulsive 1 star a host, because they weren’t happy with a small thing that ticked them off, let’s say a dusty desk, or a drop in internet for a few hours, even though they gave them a much higher sub-category ratings for the rest. This is an immediate indication of a false review. An overall rating is too much power for some (actually most) people and they abuse the power.

Example 2:

Now if a written review says something like: “The apartment had no aircon”, but they gave you a good a 5/5 for value, isn’t this a massive contradiction? I am offering them my place without aircon at a very affordable rate, (which they KNOWINGLY accepted, before booking). If they wanted aircon they should have chosen a place with aircon, most likely PAYING MORE for that listing, compared to mine. Which they don’t want to do. I am aware it doesn’t have aircon which is why my rate is lower. Aircon is a not only a massive upfront cost, but also a big running cost for electricity. I simply cannot provide aircon and the low price point together. I would be running my AIRBNB BUSINESS AT A LOSS. If the guest is happy with the value, then you CANNOT complain about a luxury item, that should be costing them, far more per night. Again a FALSE REVIEW.

CONCLUSION

So Airbnb what are you going to do about this? I have tried to communicate this with your support staff multiple, painstakingly times and just get throw the same copy, paste terms and condition link pages in my face, that have ABSOLUTLYE NOTHING to do with this issue.

Yet you feel you have ‘solved” this HUGE issue.

Before signing off, I strongly encourage ALL HOSTS to take this to airbnb’s front doorstep and make them know how unhappy we are about being ripped off of out hard earned reviews.

We spend hundreds of thousands, if not millions on our listings, taking out risky bond payments to buy properties or for massive renovation costs. We invest a huge about of our time to make sure its all up and running to perfection, only to be stabbed in the back by the system we put our trust into.

Enough is enough. If you are a superhost you deserve to stay that way. If you are a host wanting to become a superhost, then you deserve to be given your reviews FAIRLY, in order to achieve that. The sub-category rating system if FAIR. NOT the overall rating system.

We need to stand together. Facebook isn’t enough, the FB staff are just here to pawn you off with their copy paste reply of “we are sorry to hear that, PM us” with ZERO result in fixing this massive problem at hand.

Flood their  https://www.airbnb.com/help/feedback  with what I have written above. We DESERVE our hard earned ratings and FAIR REVIEWS and WILL NOT ACCEPT them being stolen from us any longer.

I want my 10 x 5 star reviews that you stole from me. I worked hard for them I deserve them.

End of story.

Aircon

I currently have NINE. Yes you read that correct, 9, false 4 star reviews out of my total of 10. These are all victim to these falsely calculated ratings by your broken system. That is 90% 4 stars that should be 5 stars! WOW! Even my only 3 star should in fact be a 4 star.

I recommend ALL guests look at your 4 star, even 3 star and below and see what the total sub-category ratings for those were and see just how much airbnb’s system is stabbing you in the back, for your hard deserved work

What i also see is that you think is a clever idea to put "accepted reservations" in as a requirement for superhost?

Do you actually know how many messages is get from people "booking" fake nights to ask me questions or to promote services? PLENTY

Now once again i get penalised for something THEY are doing incorrectly.

There should be NO “would you recommend this guest” option for hosts.

[Huma0]

@Edgar0

Don't be mislead. Those stars you see under each guest review are confusing, because if a guest doesn't rate a category, it will show up as five stars when it is not, and when it doesn't count.

So, for the example you give above, this guest left you 4 stars overall and hey left you 4 for location. As for the other categories, they might have left you 5 for each of them, but more likely they just didn't rate them.

It's a stupid glitch in the Airbnb system. It will automatically show a non-rating as 5 stars, leaving hosts totally confused as to why that guest left 4 stars overall. My experience is that most guests do not bother to fill in all the star categories anymore (especially since Airbnb made the review process more detailed for them). They only bother to rate the subcategories they're not totally happy with and leave the others blank, which to us look like 5 stars, but really aren't.

[Ron125]

 The rating problem can be easily fixed by only making hosts responsible for things in their power and calculating the stars based on math and not based on the whim of a client. This is particularly clear experiences where the comment is all, perfect perfect perfect then a 4 rating. To take it a step further, looking at the numbers, there is a specific demographic of person who gives the 4 ratings disproportionately than any other, so that gives me an incentive to discriminate when members of that demographic want my services. I also do a photography experience. I have a 4.88 rating, 4 ratings again from the '4' demographic. I applied to do the identical Experience in a different city closer to my home and it was rejected because it does not meet the AIRBNB quality standards. Go figure.

[Victoria567]

What will fix these reviews written by rogue guests, is an honest factual review of such guests from fellow hosts,  because I fail to understand, how come, some many hosts are increasingly hosting them?

So hosts must have, previously written a review for such guests, as they seem to be bouncing along from unsuspecting host to unsuspecting host!

What would really help other hosts, is the halt of the vague review of poor guests, from fellow hosts.

No facts, just the bland ‘ best suited to a hotel’...Er Hello?

Fellow host......I’ve just had the guest I would NOT want to host again and they got a 5* star, review and I’m a 5* host and these guests have just given me a terrible review!.....so WHAT THE HECK, is going on?

Other host.......Oh it couldn’t be that bad, just chalk it down to experience as it WILL make YOU a much better host. I suggest that you don’t cane this guest, just write “best suited to a hotel” as EVERYONE surely knows, what this guest did in your family home.

Everyone  on the air bnb platform can look at this negative review left by this guest and sympathise with you.

Fellow host.......but that’s so vague, how’s that going to STOP this guest bouncing around air bnb from unsuspecting host to host?

Other host.........it’s ONLY a single negative review, for goodness sake! It soon gets buried anyway, but most  important of all, I DONT want to risk my STATS!

Er....Hello?

[Mr8]

Man you are amazing with how well you articulated this. You should write a book!!

[James-And-Tanya0]

The trust and safety department is disgraceful.  They only communicate through email, and so it is a very one sided  style of communication.  They are currently threatening to close my case if I don't agree to a minimal settlement.   Be aware if you ever have damage. 

[Victoria567]

Hi@James and Tanya

What is really disgraceful are the vague reviews written for such rogue guests by fellow hosts.

Did you look up this guests previous reviews?.....someone must have given them stellar reviews!

What would really help other hosts, is the halt of the vague review of poor guests, from fellow hosts.

Poor vague guest reviews with No facts, just the bland ‘ best suited to a hotel’...Er Hello?

Fellow host......I’ve just had the guest I would NOT want to host again and they got a 5* star, review so WHAT THE HECK, is going on?

Other host.......Oh it couldn’t be that bad, just chalk it down to experience as it WILL make YOU a better host. I suggest that you don’t cane this guest, just write “best suited to a hotel” as EVERYONE surely knows, what this guest did in your family home.

Fellow host.......but that’s so vague, how’s that going to STOP this guest bouncing around air bnb from unsuspecting host to host?

Other host.........it’s ONLY a review, for goodness sake! It soon gets buried anyway, but most  important of all, I DONT want to risk my STATS!

Er....Hello?

[Robin4]

@Ryan66 @James-And-Tanya0 @Mr8 @Ron125

Ryan has gone to a lot of trouble here and although I have sympathy for his cause, he has unfortunately got it entirely incorrect!

The guests 5 star overall rating is not in any way linked to their sub category ratings!

A guest can give a 5 in every category and still give the host an overall of 4 or even 3!

The guest can give a 4 in sub categories and still give an overall of 5! I have had both!

I think  most of us would be aware of that, we have all had reviews like this......

Absolutely nothing wrong in any of the sub categories, so by Ryan's argument that overall 4 would not be possible. But it is, the guest was satisfied with everything, just didn't think it was perfection!

Ryan is right in that we do need change, but we have to go about it from the perspective  of making something wrong, right........ not trying to prove that something that is right, is wrong!

Cheers......Rob

[Ron125]

Unnecessarily complicated. Thet simply let the guests give grades for a list of qualities. They average the grades. They have some categories that the grade does not contribute to but it is kept secret. If a guests sees a location before booking then their grade for location  should not count toward the average. I had a guest give me a 4 because my house was not a 5 star hotel and he actually wrote that. Or just create an appeal process. If you are not happy with a grade, appeal. Other hosts get to vote and if you get a predetermined percentage of support, the grade is removed. I think you guys are really complicating things unnecessarily

[Huma0]

@Rob0

Not necessarily so. If the guest didn't rate any of the subcategories (and they don't have to if they don't want to), they will misleadingly show up as 5 stars under the review, when they are not. When the guest does not rate a category, your stats in that category neither go up nor down. For some reason, Airbnb sees fit to show the host that it's 5 stars in each category when it is actually none.

In these cases, the 4 star overall would make more sense as you have no idea what the guest didn't like if they didn't rate the other categories nor give you feedback.

[Corrie8]

@Ryan0 from Cape Town

You have a point but maybe to get Air B&B to action or actually see the point you are making, you should keep it short by simply saying

"REMOVE OVERALL RATING...IT IS FLAWED"

Sadly, I doubt that they will take the time to read your whole claculation explanation. 

On your point about "being just a host"  I can add that without Host Air B&B would not exist and therefore they should maybe look after their hosts a little better!

Just my 5 cents worth.

[Sorab1]

I see your point Ryan.  I'm with you!

[Ned-And-Laura0]

Ok, here are my thoughts on the review system.  First the reason you can give an overall rating seperate from the sub categories is becasue the sub categories don't cover their overall experience.  Say for example the AC was rattling all night and didn't cool very well or the bed squeaked or water from the tap tasted funny or the shower didn't get very hot.  So many things can go into the overall experience that are not covered by the sub categories.  We ALL get the 5 stars for all the categories and still get the 4 star overall.  Rather then blame the system, maybe after the 4 star review is posted, politely ask the guest what you could better.  I always do that and have gotten some very good feedback.  Sometimes it's things I can't control, but sometimes it's things I can easily fix.  Once somebody gave me 4 stars overall and they said it was becasue there was no outlet near the bed to charge their phone and the nightstand light was burned out and the hair dryer didn't work.  Three things I could easily fix and never would have been aware of if I didn't ask.

Also, I feel as hosts we put WAY to much thought into the reviews and stars because really the potential guests don't care that much.  I have said it a dozen times, so long as your space is booking and money is coming in, nothing else really matters.  We are not here for some existential validation of our lives, we host becasue we want to make money.  Guests book based on location and price.  They look that the photos, maybe read the description and briefly scan the last couple reviews (which are the only ones shown unless they click to see more which nobody does).  And the stars are not even tied to a specific review, only the average is shown at the top.  And I promise you, guests don't care if you have a 4.7 or a perfect 5 star rating.  Honestly I feel that the most guests don't even know what super host is.  I was a guest for years before becoming a host and I never even noticed the little badge next to the photo and had no clue superhost even existed.  I booked on price, location and photos.  So just keep doing the amazing job that you do and I promise you, the reviews will take care of themselves.

[Zappa0]

I dont know about you guys, but I breeze through those star ratings and click the submit button. 5 5 5 5 5 and thumbs up. If you do it on a cell it is click...swipe left...click ...swipe left. Very easy to accidently click the wrong star and hit submit. 

[Huma0]

@Ryan66while I understand and agree with the overall jist of your argument, there is one point to consider (apologies if someone else has already mentioned this as it's quite a long thread so I might have missed it), but are you sure that all of your 4 star overall reviewers gave you 5 stars in almost every category?

Airbnb do something very misleading when showing hosts their stats on the Ratings page. A guest does not have to fill in the star ratings in the sub-categories if they don't want to and if they do, they don't have to fill them all in. However, if a guest does not fill them in, they will show under the review as 5 stars, when in fact they are not at all, they are non-ratings and do not affect anything.

Or, say a guest gives you 4 stars overall, 4 stars for Location and then doesn't fill in any of the other star categories, it will show up as 4 stars overall, 4 for Location and 5 for everything else, when they never actually gave you those 5 stars for everything else. I hope that makes sense!

So, it's obvious when a guest scores you 4 stars or less in a subcategory, but impossible by looking at the stars below the review to tell if they left 5 stars or just didn't rate that category. The only way to be sure is to keep track of your pecentages for each category and check them after each review, but even then, you have to wait for them to update (wait for the number of reviews to go up by one) to tell.

I have started doing this. What I find really frustrating (and I'm surprised other hosts don't mention it more on the forums) is that many, many guests will only rate the one category they are disatisfied with, but not bother to rate the others, e.g. they will give 5 overall and 4 for something like Location, but not bother with the rest. I'm sure the guest doesn't see this as a problem. Perhaps they assume that by giving you 5 stars overall, any category they don't rate will automatically be 5 as well. That is not true.

If guests only rate the one (or maybe two) categories they aren't totally happy with but don't bother to rate the ones they are perfectly happy with, over time, this inevitably drives down the category ratings even though that is not an accurate reflection of the guests' experience.

[73宿0]

@Ryan66 I am with you, totally agreed! 

Your explanation is so clear that it lifts up my pain to think again about airbnb's ugly " overal experience "

I have received many 4 stars overal experience, yet perfect in 5 stars sub category.

This kills my super host status, when I spoke to airbnb reps, they told me it is base on freedom of will from different cultrue, some people think 4 stars is very good, but some people think 5  stars is the way to be GOOD, they can't control it!!!

So what's the point of superhost 4.8 when the reviewers are not based on the same thinking manner???

This review system is absoulutely broken to the best.

Remove overall experience rating is needed!
combine the sub listing stars into overal experience is the way to go.