Review Update inspired by our Community Center Hosts

[Laura_C]

At the end of last year, we announced several updates outlining new policies to help address irrelevant and biased reviews, a topic many of you understandably have expressed concern about. Thank you for all the feedback you gave in response to this, I've been reading all your comments and additional ideas on what you would like us to focus on next.  

Although only a few weeks have passed since then, your suggestions are already making a difference. 

One feature we've just launched (yes! It is already live!!), is adding a prompt to the overall rating section of the review process for guests:

If a guest gives 5 stars for all subcategories ie. Cleanliness, Accuracy, but then selects 4-star as an overall rating, a prompt flagging this to the guest will appear.

It was 100% your comments in these threads that inspired us to ensure this work happened. Shoutout to our hosts who brought this to our attention: @Sarah977, @Paola4, @Susanna0, @Laura108, @LuisCarlos1, @Aaron79, @Mariana58, @Justo6, @Rodrigo569, @Elmari0, @Melodie-And-John0, @Dee51, @Brian1595, @Peggy137, @Heidi313, @Ela22, @David64, @Anne8553, @Keita5

Of course, there were many other suggestions given and this is just the beginning of our work to make improvements in this area. Please do continue to share your ideas across the Community Center and we look forward to providing you more updates throughout the year.

~ Laura

[Paul1255]

@Laura_C  I hope you are well!

I recently took a trip away and stayed in an Airbnb as a guest. When it came to review time when I opened the review prompt email I had to do the following:

-click on 1 of 5 stars,  each labelled as Terrible; bad; OK; good and great.

That click on one of the stars then takes me out of the email and onto the Airbnb platform to complete the rest of the review.

That is what decided the "overall" star rating for the trip and this is why we see problems with guests ratings of the sub categories differently to their overall experience- they might not even realise they have given a overall star rating in the first place.

But the real issue that needs to be fixed- and I am not sure why a problem has to continue to be a problem for years until a solution is found- is that star rating expectations from Airbnb on the guest side don't match up to the host side. Airbnb essentially give different meanings to the star ratings system depending on if you're a guest or a host:

Guest:

1*  terrible 

2* bad

3* OK

4* good

5* great

Host:

1* terrible/bad

2* terrible/bad

3* terrible/bad 

4* terrible/bad

5* OK/good/great

Is there anything in progress to address this disparity in the star rating system- and the cause of so much host stress and anxiety from threats of being delisted, to losing Superhost status?

Paul. 

[J-Renato0]

It has been a long time since this issue was raised. (4* good for the guest, bad for the host)

The thread has 11 pages. 

It was in May 2017 - Almost 3 years ago !

https://community.withairbnb.com/t5/Hosting/So-why-is-4-Stars-Good-for-Guests-but-Bad-for-Airbnb-Hos...

[Paul1255]

Thanks for highlighting this @J-Renato0 

Perhaps it's confirmation that there won't be a change any time soon....

Best wishes

Paul.

[Emilia42]

@Laura_C 

I 100% agree with Paul. @Lisa723 once wrote on this forum "When you give a host a 4 star review you are essentially casting your vote to have them removed from the platform." (direct quote may not be word for word)

That really stuck with me and it's true. A host who consistently gets 4 star reviews for being 'good' risks a 5 day suspension with every pending review. Hosts are constantly encouraged and made to 'educate' their guests on the truth behind the review system. To a point which I think is getting a little out of control. I would really like to see the process change altogether and the stars eliminated.

[Paul1255]

It’s so true @Emilia42 

My stay as a guest was “good” for sure, but not in a million years would I have considered rating it anything less than “great” for the reason you mention- rendering the system in its current form useless.

[Emilia42]

@Paul1255 I do find the system useless! As a guest, I have been gravitating back to hotels because the rating system is consistent. Whether I check into a 3 star or 4 star hotel I know what I am getting. On Airbnb, I have had a few mediocre stays with 5 star hosts (but of course I would always rate them 5 stars myself.) I am not confident that just because a host has 5 stars means that I am going to sleep through the night.

[Alex-And-Rhea0]

@Emilia42 That's good point-about hotels being consistent, and Air Bnb are not hotels.

We have almost never received a 5-star review from a Hong Kong resident unless they were of European extraction, I feel that that the star system confuses people from a different culture, for example, the Peninsula Hotel in Hong Kong or The Mandarin would be considered 5-star hotels so my humble ski chalet is .... To top it all there is no grovelling Concierge waiting for a tip nor an insouciant front desk manager, nor a baggage handler to carry your bags for you.

In other words, it's basically self-service.

So why do people travel and use Air Bnb?

Because it's new, adventurous, independent and economical?

I think  English as a first language people come to our ski chalets primarily because of our good reviews which they read at least the latest 4 to 5 reviews.

[Helen350]

@Emilia42 @Paul1255 @Laura_C I wouldn't mind having a permanent 4.0* rating = good, or even a 3* =OK, IF Airbnb's threshold for suspension / delisting were below 3.0! IF a system existed where 3* really was known to be 'OK' by guests, hosts & above all Airbnb staff/exec, then there would be room for distinguishing levels within OK, & highlighting the truely exceptional.

- My place is simple & basic, in the spirit of the original Airbnb, but because of that it's cheap, & I hope, clean enough. 3* would be fine & appropriate in a sane rating system, where it is the high end luxury places that get 5*. (Of course,as it is now, I still need 5* in order to present myself as adequate....) 

[Rowena29]

@Laura_C 

I couldn't agree more with @Emilia42 and @Paul1255 

I have told many people ( not just guests - friends - associates who use airbnb) how damaging anything less than a 5 star can be and UNILATERALLY they have all been shocked.  They have all responded, "but that's ridiculous!" " " i had no Idea!" etc etc.   In many many people's minds, ( especially outside usa) full stars  is something that you only assign to a truly extraordinary experience - the creme de la creme - the pinnacle of perfection.   

To routinely give - and expect - full stars undermines and devalues the excellent and exceptional. 

Personally I would also like to see a system where the stars expire after a year - similar to how ebay seller ratings work. At least there if you've had a couple f unreasonable, ridiculous ratings, you know if you continue to offer great service they will be a blip and you can retrieve your previous stellar status. There's something to aspire to, instead of something to continually resent.  Currently, it just takes one person to naively/unknowingly/maliciously tank your stars and it's a permanent reminder and blight on your ranking until the end of time. It's galling, especially if  it happened early on in your hosting and especially if the reasoning was petty or trivial.  I bet your colleague who got marked down on stars for not having bakeware in her listing on her very first hosting  would agree!

[Laura_C]

Hey @Paul1255, @Rowena29  and @Emilia42 , thanks so much for your thoughts.

Reviews and stars systems are incredibly difficult to get right. While it would be way easier to eliminate them (and trust me, sometimes I wish we could), we know that it would actually decrease the confidence that guests have in booking a home, so our hosts overall would have less bookings. So what we need to do is to make them as good as possible - fair for hosts, and helpful and relevant for guests.

I connect regularly with the team that works on reviews (Paul, I forwarded them your comment also) - they are definitely aware of the concern here around 4 being represented as 'good' for guests but 'not good' for hosts. Airbnb Experiences has been working with a different description system so we're closely following that to see if it might be a better solution, and simultaneously thinking if there's something else we can do to address this issue (eg notes during the review flow to guests to help them understand what a big deal a 4 or below is for a host). There have been lots of ideas on this topic here on the Community Center that I've ensured have been collated and sent to the Reviews team.

Thanks again for your thoughts and for reaching out!

~Laura

[Lisa723]

@Laura_C why is it "incredibly difficult" to get right? Maybe Airbnb could learn something from other systems that don't seem to find it so incredibly difficult, e.g. Amazon, Yelp, and Vrbo/HomeAway?

I think your answers here are disingenous. It is obvious to anyone with a brain that Airbnb has its own reasons for intentionally misaligning the meaning of ratings as presented to hosts vs the meaning of ratings as presented to guests, and those reasons can't possibly have anything at all to do with actually helping guests identify listings that are a good fit for them.

[J-Renato0]

Hello @Laura_C 

In addition to Lisa723, I would say that booking dot com and expedia dot com has great systems of reviews. 

Unfortunatelly Airbnb does not here many suggestions ... or maybe ignore. Great suggestions that could help to improve in real terms the review system.

1-Why keeping reviews written more than 3 years ago? Lots of things may have been changed. Review must reflect an actual situation of the host and the property.

2-In terms of statistics, a good hotel is 7.5 to 9 in terms of reviews mark (scale 1 to 10). 10 is the exceptional, not the normal.

3-I already have suggested that Airbnb should discard absurd results. I have suggested it more than twice. Once some Airbnb  staff person mentioned that they were about to consider that. 

Example, in a batch of 10 reviews, discard the worst and one of the best to compute the superhost status, if the current system is maintained. One of the best results would be discarded to offset the measurement, and make the result more real.
This is what is done in terms of statistics to obtain results closer to reality.
In Olympic gymnastics competitions this method is used. In order to obtain more real statistics, isolated and very absurd results are also discarded in the fields of science and statistics. These are called points far outside the bell curve.

Thanks for your time and for given us the oportunity offer some ideas to improve the review system.

[Alex-And-Rhea0]

@Lisa723 Whew! Succinct!! I agree wholeheartedly with your rhetoric.  It is our business and we are striving to do the best we can and we deserve better than what Air BnB has come up with -so far .

[Huma0]

@Laura_CI totally agree with the comments already made here by other hosts. By far the biggest problem with the review/rating system is the discrepancy between what the guests are told the stars mean and what they actually mean to hosts. That needs to be fixed and ASAP. Hosts have been calling for this for years and what they are asking for is logical and reasonable. Why can't it be done?

While the prompt you mention is not a bad thing, I think an important point is being missed here. Sure, a lot of hosts on the CC have lamented about those guests that leave 5 stars in each category, but 4 stars overall. It seems so unfair.

However, I believe that in most cases, this is a misapprehension. What people don't seem to realise is that when a guest DOES NOT RATE ONE OR ALL of the individual categories, on the host's side, it will show that the guest left them 5 stars for that category.

Yes, that's right. Let's break it down.

- Guest A leaves 5 stars overall and 5 stars in each category. Host will see that he/she left 5 stars overall and in each category.

- Guest B leaves 5 stars overall and 5 stars in two of the categories but doesn't rate the rest of them. Host will see 5 stars overall and in EVERY category.

- Guest C leaves 4 stars overall but doesn't rate any of the categories. Host will see 4 stars overall but 5 stars in every category and be perplexed.

Why would the guest leave 5 stars in each category and then 4 overall? Well, they probably DIDN'T but that's how it will appear (and this really should be fixed. It can't be that difficult to do. Just show the guest didn't rate the category rather than show it as 5 stars).

It's not that I dislike the idea of prompting the guest to reconsider their 4 star rating, but I really think that the problem this fix is addressing is a very small one. I am certain very few guests leave 5 in each category and 4 overall (never happened to me in hundreds of reviews). It's just that on the Airbnb system, it displays as such to the host.

Sorry to keep banging on about this, but no one seems to get it. Just because it LOOKS like the guest left you 5 in each category, that does not mean that they actually did.