Making reviews more fair for hosts

[Airbnb]

 Many of you have asked us how Airbnb can protect hosts from one-off bad reviews. When this question came up at the most recent Host Q&A, we told you we were working on ways to make the review process more fair for hosts. Specifically, we made 2 promises:

1. We committed to launching a tool to detect outlier reviews—or one-off bad reviews. A common example is when there’s a discrepancy between the overall rating and the category ratings provided by a guest (when a guest gives a host 5 stars for cleanliness, accuracy, and the other categories but a 2-star rating overall, for instance).
2. Based on your feedback, we also committed to exploring ways to help guests better understand that the location category rating is meant to be objective. The location category rating doesn’t impact your overall rating (or Superhost status), but we know it’s important to you, and we want to make sure the whole system is as fair as possible.

Today, we’re excited to announce two improvements to the review process that directly address these issues. Since these changes have been introduced, we’ve already noticed a tangible uptick in more accurate, fair reviews for hosts, and we hope they solve some of your pain points. Here’s what’s new:

One-off review alerts

We’ve added a step to the review process for guests when they give a host an inconsistent overall rating. For instance, the guest may have given 4-star or higher ratings for all the categories (cleanliness, accuracy, etc.), but then give an overall rating of less than 3 stars.

The new pop up screen asks guests: “Is this right?” And goes on to explain that they rated their overall stay lower than they rated it in specific categories. It gives guests an option to either change the rating or ignore the alert.

This new alert has led to higher overall review ratings for hosts. Since we launched, we’ve seen a 2.8% drop in 3-star reviews and a 3.9% drop in 2-star reviews. While these percentages may seem small, they’re driving real improvements in the accuracy of our review system, and hosts are benefitting.

Location, location, location

We’ve heard from you that the location rating can be particularly frustrating because some of you have experienced guests dinging you in this category, unexpectedly, after great stays. This category is tricky. It gives valuable information to prospective travelers, which we don’t want to lose. At the same time, we hear your concern that you’re being graded for something you can’t control: guests’ opinion of your location. This opinion is inherently subjective—one person’s “rustic rural retreat” may be another’s “too far from public transportation.” So we made it more clear in the review process that guests are rating the accuracy of your location description, rather than the location itself.

Now, when a guest goes to rate you in the location category, if they give you less than 3 stars, they see an explanation: “Was the listing’s location not described accurately?” So far, this has led to a 0.8% increase in the average rating for location.

While we were working on this, we also made similar improvements to the value category. If a guest gives you less than 3 stars there, they’ll see this message: “What would have made this listing a better value?” This has led to a 0.25% increase in the average rating for value.

These changes were designed to begin to address your concerns around unfair reviews, and to help make sure that guests understand what ratings mean. We still have a journey ahead of us to keep making the review system better, and you’ll continue to see updates from us on this throughout the year. Thank you for hosting!

[Suzette24]

This is great! A guest just gave us a bad review because we dont have a TV in the apartment (stated clearly in our profile) and there's no daily housekeeping to clean up after her and layout daily fresh linen and towels (our cleaning fee is $40!) We were so concerned but when we look at our ratings we could tell AirBnB rated her review as an outlier. Grateful!

[Anthony608]

On only my third guest I had an unfair review and it was sparked by her finding a cleaning cloth left under the bed prior to her check-in.  The only way in which the guest would have found that cloth was if she deliberately crawled under the bed looking for an issue.  For this, I received a “3" in cleanliness. 

On feedback, she also falsely stated that she had been given the wrong address for the property – I think she might have been mad she had to walk down a street to the bus stop instead of the bus stop being literally outside the front door.  I didn't pursue this, since overall it was 4 stars from her, but this was still my first taste of a somewhat unfair review.

[Sarah977]

@Airbnb   So you are only doing this for reviews that are 3 stars or less WHY?  Are you aware that guests are marking down on accuracy because the place was better than they expected? Yes, some of your guests are that clueless. And then the host gets punished in their ratings for being too good?

If you are only going to prompt guests who leave 3 stars or less, then you need to stop rating hosts with anything less than 5 stars as failing.

What in the world is wrong with you people?

[Julia1]

Like others I welcome the move in the right direction but really Airbnb do need to reconcile the gulf between telling guests that 3* is OK and telling us we need much more.  Also the changes in the Location rating i.e. challenging low ratings has resulted in less that 1% changing their rating - well whoopy doo that is tiny and obviously more work on this is required e.g. why not ask if location was as described which would be a lot fairer. 

[Anonymous]

What exactly is the "valuable information" that the star ratings for Location provides to prospective guests? If the property's position on Google Maps is displayed correctly, guests need only an internet connection to find out just about anything they want to know about the location before booking.  If there is something objectively inaccurate about the listing's depiction of the location, there is already an Accuracy rating.

Similarly, I can't see the usefulness of a Value rating when viewing a listing, since I don't know what nightly price the previous guests paid. 

"Be objective" rings hollow to me in the context of reviews, since the very nature of reviews is subjectivity. Perhaps it's time to admit that much of the data collected in the form of star reviews is not actually helpful to anyone.

[Marc-and-MJ0]

A great initiative !! We have been saved from the 'one-off' bad guest leaving an unfair review by the outlier review function.

Looking forward to more iniatiatives in this direction from @Airbnb3 

Marc & MJ

Short Stay Property Management

Melbourne

****

*[Link hidden in line with the Community Center Guidelines]

[Katie193]

@Marc-and-MJ0

I would love to know how you made that happen and where the outlier review function is! I have an outlier review preventing me from superhost status this time around - 4.7 instead of 4.8 because of her 2. First 2 in 4 years and it was retaliatory.  How did you get Airbnb to remove it? We don't seem to have that function here in the US.

[Sarah977]

@Katie193  They weren't referring to a function available to hosts. If you read the previous posts in this thread, what they are talking about is a pop-up that appears to guests when they have marked high stars for all the subcategories but the overall rating doesn't jive at all with that.

[Giedre-and-Andre0]

No @Sarah977

Its "...less than 3 stars..."

So its for 1*  and 2** only.

This will not help (at least me) at all!

Here my recommendation for fair reviews  @Airbnb 

- Was the location representation correct in listing description:   1 - 2 -3 -4 - 5

- If a guest was not moving into his booked place (no show or leaving before getting into the place): NO star rating should be possible for guest and host. Guest and host can leave a written review and a reply to given review.

- If a guest tries to "sneak" extra people into the space: Host should be able to cancel,  NO star rating should be possible for guest and host. Guest and host can leave a written review and a reply to given review.

- If host or guest needs to end a booking during the stay: NO star rating should be possible for guest and host. Guest and host can leave a written review and a reply to given review.

- If there is a discrepancy between category rating and overall rating, a popup window should ask for explanation

(Starting at 2 stars or less, is doing  NOTHING for me! In fact, as of your own statistic: It will NOT doing anything for 99.2% of your Hosts)

- Hosts are in a huge disadvantage reg. star ratings as they need  to get 5 to 8     5*****  to recover from one 3***

- Guests should be blocked from booking (same as hosts) if they not leaving minimum  50% reviews.

- "Compliments": Should be posted in total instead of last/recent guests.

EXAMPLE:  73 guests said this place was sparkling clean, instead of 12 recent guests said this place was sparkling clean.

I understand, the "compliments addition" was created to put extra pressure on hosts to have a even higher quality standard but it feels like "simple 5 stars" are not good enough and there are guests who not doing this extra rating.

[Geri26]

As per my new thread ' Guest Revenge Review' one recent person left 1* rating and I suspect reported me (I cdon't know how to check if he has) and now both my listings are under threat.   I feel sick about the whole experience and interaction with both the person and Airbnb

[Jeff602]

With my 2 cents, the rating system aren't set up in a way where hosts like me can

be successful and probably won't know what it's like to be a superhost. More needs to be done with the rating system beyond the initial post above. Plenty of my guests say they had nice stays at my place, only to later get less than 5 stars (mainly 4 stars) in the overall category. Personally there seems to be multiple 1-offs with the ratings I'm getting including some recent ones where the guests communicate nothing after I ask them what can be done to make their stay better. Again way more needs to be done.

[Yulianna0]

I wonder why the stars are so important? Written review shows personal experience and personality of guest and host, so it can be useful for next guests and for hosts. Let people write! And for sure, our guests have to be clearly informed, that they are valuing not their dreams and expectations but only things that are in the description. 

[Jayne34]

I'm still waiting for the Airbnb customer rep to contact me after I reported a 1 star review from a guest I had to cancel after his first night due to his belligerent behavior and nonstop vulgar telephone calls at 2 in the morning! What he wrote was all loes, including that I was scamming guests by telling them I was canceling and giving a refund then didnt do it. Something verifiable as a lie by Airbnb and clearly against their rules of facts must be accurate. 

[Heather315]

It's good that you are taking steps to address these issues, all of which I've experienced as a host. However, when I called to ask you to adjust three reviews where the guests gave me 5 stars in every category, but then gave 4 stars for the overall review, and you refused to do so, that was not good customer service on your part. In each case I contacted the guest throught the app messages and each one replied that it was a mistake. In cases where the guest clearly states it was a mistake, you should fix it. I'm still rather pissed about this, you should know. 

[Judy461]

To me the overall score needs to be all the other scores averaged. A separate score unrelated to the others makes no sense.