Making reviews more fair for hosts

[Airbnb]

 Many of you have asked us how Airbnb can protect hosts from one-off bad reviews. When this question came up at the most recent Host Q&A, we told you we were working on ways to make the review process more fair for hosts. Specifically, we made 2 promises:

1. We committed to launching a tool to detect outlier reviews—or one-off bad reviews. A common example is when there’s a discrepancy between the overall rating and the category ratings provided by a guest (when a guest gives a host 5 stars for cleanliness, accuracy, and the other categories but a 2-star rating overall, for instance).
2. Based on your feedback, we also committed to exploring ways to help guests better understand that the location category rating is meant to be objective. The location category rating doesn’t impact your overall rating (or Superhost status), but we know it’s important to you, and we want to make sure the whole system is as fair as possible.

Today, we’re excited to announce two improvements to the review process that directly address these issues. Since these changes have been introduced, we’ve already noticed a tangible uptick in more accurate, fair reviews for hosts, and we hope they solve some of your pain points. Here’s what’s new:

One-off review alerts

We’ve added a step to the review process for guests when they give a host an inconsistent overall rating. For instance, the guest may have given 4-star or higher ratings for all the categories (cleanliness, accuracy, etc.), but then give an overall rating of less than 3 stars.

The new pop up screen asks guests: “Is this right?” And goes on to explain that they rated their overall stay lower than they rated it in specific categories. It gives guests an option to either change the rating or ignore the alert.

This new alert has led to higher overall review ratings for hosts. Since we launched, we’ve seen a 2.8% drop in 3-star reviews and a 3.9% drop in 2-star reviews. While these percentages may seem small, they’re driving real improvements in the accuracy of our review system, and hosts are benefitting.

Location, location, location

We’ve heard from you that the location rating can be particularly frustrating because some of you have experienced guests dinging you in this category, unexpectedly, after great stays. This category is tricky. It gives valuable information to prospective travelers, which we don’t want to lose. At the same time, we hear your concern that you’re being graded for something you can’t control: guests’ opinion of your location. This opinion is inherently subjective—one person’s “rustic rural retreat” may be another’s “too far from public transportation.” So we made it more clear in the review process that guests are rating the accuracy of your location description, rather than the location itself.

Now, when a guest goes to rate you in the location category, if they give you less than 3 stars, they see an explanation: “Was the listing’s location not described accurately?” So far, this has led to a 0.8% increase in the average rating for location.

While we were working on this, we also made similar improvements to the value category. If a guest gives you less than 3 stars there, they’ll see this message: “What would have made this listing a better value?” This has led to a 0.25% increase in the average rating for value.

These changes were designed to begin to address your concerns around unfair reviews, and to help make sure that guests understand what ratings mean. We still have a journey ahead of us to keep making the review system better, and you’ll continue to see updates from us on this throughout the year. Thank you for hosting!

[Jennifer1696]

In regards to guests not leaving a review....I use another platform that requires each party to review before they can book again or as a host before they can accept another booking, reviews are required. Can @Airbnb2 implement this requirement in fairness to both parties?

[Ralph88]

We've been a "Host" since Sept 2018 and have found everything about AirBnB to be good.  We also appreciate what looks like a dedicated effort on AirBnB's part to improve things on an ongoing basis for both the Host and Guest.  However, reading over 800 of the above posts, I realize that the overall concern (and it's ours also) is about reviews from guests who have an "ax to grind".  I think it's human nature to feel defensive or want vengence when corrected or asked to change our behavior.  

I might suggest that any overall negative review (more than two threes) be kept between the giver and receiver allowing them an opportunity to work out the things that were problematic.  Airing one's dirty laundry in public helps noone!  I might also suggest that if any Host or Guest receives more than two or three negative reviews within a certain period, that AirBnB investigates the issues and then if necessary  post the reviews or take punitative actions against the receiver of the maxed out number of reviews.  

It lookes to me that the majority of gross violations from both sides of the asile come from 'newbys' in both hosting and especially the guest catagorys.  I suggest that AirBnB implement a more stringent vetting effort with both parties to insure that they know & understand the implications of their reviews as well as there overall expected conduct when implementing their affiliation to AirBnB.  

It is said that "one rotten apple spoils the bushel".  The only remedy to this situation is to remove the "rotten apple" as quickly as possible!  

[Cherie40]

I think no matter what hosts do, there will be guests who feel that nothing is perfect and so even if their time was fabulous and they loved everything, they simply refuse to give 5 stars.  

I do find that meeting my guests face-to-face 99% of the time gets me those great ratings that I love.  And I have stayed at an Airbnb that had a great way of saying that while four stars are great ratings for movies, they're not great ratings for Airbnb hosts.  I thought that was very well put.  

I will say, this experience of hosting people from all over the world has given me a greater love for people.  I give people overall a 4.5 star rating.  And I've learned things about their part of the world that I would never have known.  I wish Airbnb had been around 30 years ago, I would have made it a full-time career. 

---

@Airbnb wrote:

 Many of you have asked us how Airbnb can protect hosts from one-off bad reviews. When this question came up at the most recent Host Q&A, we told you we were working on ways to make the review process more fair for hosts. Specifically, we made 2 promises:

1. We committed to launching a tool to detect outlier reviews—or one-off bad reviews. A common example is when there’s a discrepancy between the overall rating and the category ratings provided by a guest (when a guest gives a host 5 stars for cleanliness, accuracy, and the other categories but a 2-star rating overall, for instance).
2. Based on your feedback, we also committed to exploring ways to help guests better understand that the location category rating is mean to be objective. The location category rating doesn’t impact your overall rating (or Superhost status), but we know it’s important to you, and we want to make sure the whole system is as fair as possible.

Today, we’re excited to announce two improvements to the review process that directly address these issues. Since these changes have been introduced, we’ve already noticed a tangible uptick in more accurate, fair reviews for hosts, and we hope they solve some of your pain points. Here’s what’s new:

One-off review alerts

We’ve added a step to the review process for guests when they give a host an inconsistent overall rating. For instance, the guest may have given 4-star or higher ratings for all the categories (cleanliness, accuracy, etc.), but then give an overall rating of less than 3 stars.

The new pop up screen asks guests: “Is this right?” And goes on to explain that they rated their overall stay lower than they rated it in specific categories. It gives guests an option to either change the rating or ignore the alert.

This new alert has led to higher overall review ratings for hosts. Since we launched, we’ve seen a 2.8% drop in 3-star reviews and a 3.9% drop in 2-star reviews. While these percentages may seem small, they’re driving real improvements in the accuracy of our review system, and hosts are benefitting.

Location, location, location

We’ve heard from you that the location rating can be particularly frustrating because some of you have experienced guests dinging you in this category, unexpectedly, after great stays. This category is tricky. It gives valuable information to prospective travelers, which we don’t want to lose. At the same time, we hear your concern that you’re being graded for something you can’t control: guests’ opinion of your location. This opinion is inherently subjective—one person’s “rustic rural retreat” may be another’s “too far from public transportation.” So we made it more clear in the review process that guests are rating the accuracy of your location description, rather than the location itself.

Now, when a guest goes to rate you in the location category, if they give you less than 3 stars, they see an explanation: “Was the listing’s location not described accurately?” So far, this has led to a 0.8% increase in the average rating for location.

While we were working on this, we also made similar improvements to the value category. If a guest gives you less than 3 stars there, they’ll see this message: “What would have made this listing a better value?” This has led to a 0.25% increase in the average rating for value.

These changes were designed to begin to address your concerns around unfair reviews, and to help make sure that guests understand what ratings mean. We still have a journey ahead of us to keep making the review system better, and you’ll continue to see updates from us on this throughout the year. Thank you for hosting!

---

[Mandi-S-Just-Business-Ro0]

As hosts we can no longer see photos before booking. I get that. But is there any truth to the rumour that we can no longer see past reviews, profile, or how they reviewed others?

[Anonymous]

I'm glad there are some changes regarding Rating Location but I'm afraid it's not good enough. "...guests are rating the accuracy of your location description, rather than the location itself" - this is great, yes! but only under 3 stars? why not 4 stars?  And quite honestly, they are not rating accuracy! if accuracy of location was being rated, I would only get 5 stars because my description of location is on spot! I say exactly how far it is from Metro, Trams and shops. There are no surprises so just because they can't afford accommodation on Stephansplatz, doesn't mean they should rate my location less than 5. I know it doesn't influence the Superhost Status, but it just seems really unfair. Once I got a note from Airbnb saying something like "well done.!Your last 2 guests gave 5 stars to location..." It's a funny statement - how can anyone improve on Location 😄 So yeah, this location rating is still not fair in my opinion. I hope you'll find a better solution soon. Thank you very much for all the effort.

[Martin684]

It's great you are attempting something even though small.

This does nothing for disgruntled guests who give a very low score on everything even though most guests give you high scores.

I am a member of a large group that hosts in our area and we have also noticed guests would contact you after arriving complaining for no reason, then asking hosts to reduce price or bad review will be issued.

These people are holding Hosts hostage. I have had this happen twice on this platfrom.

[Paul1477]

Thank you for those changes.

[Icardo0]

Ja this is nice but when will you solve the problems with the Airbnb app in the Huawei A-pad and I-phone. I can definitely not log in into my account from my New Huawei phone and my I-pad This is embarrassing for your company

[Iva67]

This is ridiculous!!!! I am right out PISSED OFF about YOU guys NOT DOING YOUR JOB!!!

This is IDIOTIC TO SAY THE LEAST. 

How a host can come to the house, trash the house and make an illegal party then write a review that doesn’t reflect anything and l CAN’T DO ANYTHING ABOUT IT since they did not oftend me ( they just trashed the house, had police over and made me look like an neighbour that doesn’t care ) and what’s even WORSE IS THAT YOU ARE STILL NOT DOING ANYTHING ABOUT IT! - DO YOUR JOB and stop bs- ing everyone this IS OFFENSIVE FOR ANYONE THAT HAS ANY COMMON SENSE. SHAME ON YOU

[Mary140]

A guest said the  closet smelled like cat urine,  it was a first... But had ignored my stating please hand wet towels in bathroom  or plastic hanger.,  just not draped over a metal pole. I had already did my review. 

[Heather697]

Whilst I really appreciate all the effort Airbnb are making to improve the fairness of the reviews and realize how complicated it is, I would like to comment on another factor. If the host has consistantly good star ratings for a category, such as punctuality, then one guest leaves one star, thus lowering the overall rating, I feel this should also ring some alarm bells. This happened to me...and was extremely unfair, as the guest in question missed her flight, told us two hours after she was due to arrive and gave us her new eta. However, it was around Christmas time and we were delayed by impossibly heavy traffic. We phoned her and apologized, asking her to go and have a coffee nearby for the 20 minutes or so that we would take. (And this is a guest I gave a discount to, found her a cot, did some shopping for, etc etc...). Despite all this, she left the lowest rating for punctuality and now I do not qualify for the category of business trips. 

I feel that a one-off bad rating such as this should also be investigated.

Many thanks.

[David---Milly0]

Wow....we thought we were the only ones experiencing outlier reviews! 😞 We didn't realise how many of us are being penalised out there due to the lack of robust scoring functionality current endured. 

Our issues currently experienced:

- When we survey our Guest's, those that are not Host's themselves do not understand the importance of SuperHost status, how it is earned, the difference between SuperHost vs Airbnb Plus... let alone why SuperHost status is so important to Hosts (as well as to the benefit to the Guest's). 

- Airbnb support services (stupidly) cannot override a review even when BOTH THE GUEST & HOST AGREE that a score (Overall Rating!) was rated low or accidentally given! This is unfair and leaves a bad taste in everyone's mouth even after a 100% perfect stay (And where the individual categories were all 5-stars!). Airbnb response to our query on this was 'we are not able to alter the star ratings of reviews, as this process is fully automated by the system.' Come on, this is not automated Airbnb!! This is a manual input from either the Host or Guest (depending on who is reviewing). 

- The number of categories is not the same when comparing the Host and Guest rating system. Airbnb Host's are rated by Guest's on 6 individual categories (ex-Overall Rating category) VERSUS Guests are rated by Airbnb Hosts on 3 individual categories (ex-recommendation category). Where is the consistency?  

- Interpretation of 'Location' category should be 'location description as per listing'. The current vagueness of this category is not clear enough to our guests when we survey them.  

- Interpretation of 'Value' is too subjective and unnecessary. Again, the vagueness of this category is not clear enough to our guests when we survey with them 

Our suggested solutions:

- The OVERALL RATING should be robust and AUTO-CALCULATED based on a WEIGHTED AVERAGE SCORE of key INDIVIDUAL CATEGORIES and not manually scored by the Guest. I.e. When the individual categories are all rated 5-stars, the Overall Rating cannot be anything below 5-stars.  This eliminates the 'freestyling' currently experienced.

- There should be an OVER-RIDING FUNCTION available where a rating has been INCORRECTLY/ACCIDENTLY RATED by either the Guest or Host. This function would only be available when the Guest and Host both agree that the rating needs to be updated. The rule would be you can only provide a rating that is higher than what was originally rated. The override would be triggered by the originating party requesting (challenging) the change. 

- The number of categories and its definitions should be updated, let alone equal in numbers (when comparing the number of individual categories available for Airbnb and Guest users). 

Airbnb should realise that fixing this issue quickly is to the benefit of Airbnb - The more (hard earnt) SuperHost's rated there are (...and not 'paid' Airbnb Plus's status), the better the quality of accommodation available for our wonderful guests! 

[Timea3]

That is a good thing, because I also had some sad experiences with this.

I had a guest who said the soap is missing, while the soap was there. After a private talk I understood he was talking about shower gel, which we do not provide, so why to look for it and why to say it was missing? All this, after he - also a host - told me the price is very small and I should ask for more money.

I also had a guest who:

1. Asked me to lower the price for her! I could not do that, it goes against my visions, but I did offer her free coffee whenever she wanted. (And she did ask for coffee 2 times).

2. She also did check in earlier than it would have been possible and left later then the usual checkout time.  

3. I also gave her a lot if infos about prices, salaries and so on in our area, speaking to her very sincerely; as she was planning to move here. 

And after all this, she gave us 3 starts for VALUE. I found it particularly unfair - mainly because she was, most probably, relating to the prices from her home country, not to the prices in Switzerland! ... 

3. She was going to an interview to a place which is at 7 (seven!) minutes drive with a direct bus that leaves right in front of our flat.

Still she gave us 3 stars for the location. I was like... WTF? I was assuming she did choose us because it was close to where the interview took place. 

I was thinking about this bad review for days... 

[Donna598]

Thank you Airbnb for at least looking at these issues.  Like some people have already said, it's a start but,  it does not really fully cover some of the varied issues that arises for many hosts although, I accept that this is difficult to achieve.  So for example, where the hosts description states a max number of people for example 4 people, and a guest makes a reservation for 4 people but then attempts to have 7 people stay when there is naturally only room for 4 people and they have only paid for 4 people, where is the support from Airbnb?  My expectation as a host is that Once Airbnb have been informed of the issue, they should contact the guests immediately and inform them that they cannot do this!  Why should there be any confusion.  What is stated in a host’s property details is what they consider acceptable.  The guests can make their decision with Airbnb Customer Service as to whether they stay or leave but why should the host lose out financially or being left with aggressive guests?  Surely when rules are broken by thoughtless guests, this should not be the fault of the host.  If the guest decides to leave then that's their decision, they tried to break the rules!   If they decide to stay with the correct number of people, then leave a terrible review which is totally contrary to all previous reviews for that host, than surely Airbnb has the right to refuse to accept this review.  I don't think good hosts would ever put themselves in a position where they would jeopardise their good name/hosting skills.  For there own peace of mind, Airbnb needs to have a little bit more faith and trust in their hosts and maybe start employing inspectors to randomly check out hosts and their properties and not just take a hosts description at face value. Seeing, is after all believing!

On the subject of inconsistent ratings, if a guest rates everything  say a 4 or 5, and then          gives an overall rating of perhaps 3, a way to avoid this is that the “Airbnb system”       

(perhaps a computerised programme) should generate the average overall, not the   

guest.  I believe this would be a fair way to achieve a more accurate rating rater than a                 number based on the guests input.

[Emma-and-Mark0]

Whilst I agree with many of the comments made I don't agree that auto-calculating the overall score is a good idea. There are many occasions when I have stayed somewhere and would rate my overall experience as 5 but some individual elements lower than 5. Airbnb can't possibly ask all of the questions that might make someone's overall experience more than the sum of it's parts.... e.g. view from the window, decor, eco-friendly touches etc... Likewise I have stayed in places where I cannot fault the cleanliness, hosts etc... but the overall experience isn't the best due to individual elements I haven't been asked to score.  Whenever I've rated somewhere I've booked through a website that does auto-calculate the overall score I find it really frustrating and am tempted to change my ratings on individual elements to get the overall score I think best reflects my stay (I've never done this but the temptation is there). I would be gutted if a guest of mine scored me down on cleanliness to be able to score our place an overall 4 out of 5 than 5 out of 5 for example.