Making reviews more relevant and useful for our community

[Airbnb]

Many of you rely on reviews to grow your business—they highlight your amazing hospitality. They’re also a helpful way to get specific feedback on what you’re doing well as a host and where you can improve. But there are times when you feel like a review may be misleading or irrelevant to future guests, and we know that can be painful both personally and professionally.

It’s tricky. Guests and hosts need to be able to share what they feel is important about their experience, so we don’t want to limit what people can and can’t say on the platform. However, it’s also essential that reviews reflect a guest’s stay and that they provide useful information to future hosts and guests. As part of our ongoing journey to get this balance right, here are some of the changes we’re making:

Introducing our updated Review Policy

With our updated Review Policy—effective December 11, 2019—we’ve addressed two types of reviews that we know can be frustrating: irrelevant reviews and biased reviews. The updated policy clarifies our expectations and ensures our customer support agents are equipped to remove these types of reviews.

By updating our policy to cover irrelevant and biased reviews, we’re strengthening our commitment to building a community of trust. These review updates build upon our existing Content Policy (which outlines the kind of content that is never allowed on Airbnb) and our new Guest Standards Policy, which will introduce a system for tracking bad guests. Under our updated Review Policy, guests and hosts who repeatedly leave certain kinds of biased reviews may be removed from the platform.

Let’s look at irrelevant reviews and biased reviews in a little more detail:

Reviews that are irrelevant

This policy covers situations where a review contains information that’s irrelevant to you as a host or your listing—and isn’t useful to future guests.

Here are a few examples:

• A guest accidentally leaves a review on your profile that was meant for another host
• A guest never checks into your space (for a documented reason unrelated to you as the host or your listing) and leaves an irrelevant review about their experience. For example, a guest never shows up, due to a canceled flight, but leaves you a review that complains about a dirty couch.

Under the updated policy, both of those reviews would be removed because they contain only irrelevant info.

There are also times when a guest comments on issues outside of your control, or unrelated to the service you provided. Those types of reviews may be removed if they only contain irrelevant content that isn’t useful to future guests. Here are a few examples of irrelevant content that could result in the removal of a review:

• A guest leaves a comment about your appearance
• A guest leaves you a bad review because they were frustrated by public transit in your city
• A guest leaves a review about the type of people in your neighborhood

These comments have nothing to do with your listing or the service you provide as a host and aren’t useful to future guests. So, under our updated Review Policy, our customer support agents would be empowered to remove both the review content and star rating.

Reviews with biased information

Our community benefits most when reviews share an unbiased view of the member’s experience. Our updated Review Policy covers the removal of reviews with inappropriate bias—this may include situations where the reviewer is attempting to extort the person being reviewed, has a conflict of interest, or competes with the person being reviewed. Here’s a closer look at three types of biased reviews that will be removed under our updated policy:

• Extortion: This is when a guest tries to use reviews to get something they want—for example, if they threaten to leave you a bad review unless you give them late checkout. Any attempt to use reviews or review responses to force a person to do something they aren’t obligated to do is a misuse of reviews, and we don’t allow it. People who use Airbnb also aren’t allowed to tie positive reviews to promises of compensation.
• Conflict of interest: We appreciate how much hard work goes into earning a positive review. As a result, we will not allow hosts to unfairly boost their ratings by accepting fake reservations in exchange for a positive review, using a second account to review their own listing, or providing something of value (like cash or a late checkout) in exchange for positive reviews.
• Competition: We’ll remove reviews written by hosts of competing listings or experiences where that person (acting as a guest) leaves negative reviews for competitive listings, or where we determine the content is intended to dissuade others from booking those listings or drive business to other listings.

Guests and hosts who repeatedly violate our updated Review Policy may face consequences, including account suspension and removal from the platform. We’ve also invested in retraining our customer support agents and improving our workflows, so hosts will receive better support in instances when irrelevant or biased reviews happen. As with our new Guest Standards Policy that tracks bad guests, our updated Review Policy includes warnings and education that can lead to suspension or removal of people who repeatedly leave biased reviews.

Updating our Review Policy is another important step in our journey of supporting guests and hosts like you who rely on relevant, useful reviews.

Your top questions, answered

What kinds of reviews will be removed by this updated Review Policy?

A healthy review system is one that respects and protects our community’s genuine feedback. For that reason, we take the removal of any review very seriously and only remove reviews that clearly violate Airbnb’s Review Policy. You can read the updated Review Policy in full, but, in short, this means a review is only removed if:

• The review is in violation of Airbnb’s Content Policy,
• The review is biased, or
• The review is irrelevant to the author’s experience on Airbnb

Guests and hosts who repeatedly violate our updated Review Policy may face consequences, including account suspension and removal from the platform.

What kind of documentation should I have under the updated Review Policy?

We can’t emphasize this enough: Always aim to communicate with guests through the Airbnb platform. If conversations happen off-platform, be sure to keep a record of those conversations too. That way, if you ever need to report a review to our agents, they’ll have—at their fingertips—the information they need to make the right decision. That said, even when you don’t have this preferred documentation, we encourage you to report any reviews that violate our Review Policy because we may be able to identify other evidence or patterns of behavior regarding that guest. 

Will all irrelevant review content be removed?

If Airbnb determines that the review contains no relevant information about a host or guest or listing, the review will be removed. Reviews that contain mostly irrelevant information are also subject to removal, but only where the relevant information does not meaningfully inform community members.

Where a review contains information that is unrelated to an experience as a host or guest, or is focused on something beyond the control of the person being reviewed, our team will determine the relevance of the review by considering how useful it is to our community of hosts and guests. To do this, we’ll look at two things:

• Does the review recount the reviewer’s experience and provide their personal perspective?
• Is the review helpful to other members of the Airbnb community? Does it provide essential information about a host or guest, listing, or experience that would help others make more informed booking decisions?

What’s the difference between extortionary and retaliatory reviews?

It’s considered extortion if a guest attempts to use reviews (or review responses) to force a host to do something they aren’t obligated to do. So, for example, if a guest threatens to leave a bad review if you don’t allow them to bring additional guests, that review would be extortionary and would be removed under the updated policy.

Then there are times when a host may feel that a negative review is made in retaliation. This is when, for example, a host doesn’t allow the guest to bring additional guests, and the guest goes on to leave a review about how inflexible their host was, or even writes a negative review about cleanliness or location. However, without evidence of a threat to leave a negative review, this would not be considered extortionary and would not be removed under the updated policy. If this happens, we encourage hosts to use their public response to politely address the issue.

Why aren’t you removing all retaliatory reviews?

While we understand how frustrating it can be when you receive a review that feels retaliatory, we don’t have a crystal ball to tell us what a person’s true motivations are. So, without a documented threat to leave a negative review or other evidence of a biased review, Airbnb won’t intervene. Here’s why:

• As a marketplace, we often don’t know the truth about what took place, and our review system is a critical feedback mechanism for our hosts and guests. That means we want as many reviews as possible to remain intact—so our community can put them to good use.
• Most guest reviews contain honest feedback about their experience and useful information for hosts and guests.

To reiterate, as outlined in our updated Review Policy, Airbnb can—and will—intervene where there’s evidence of a threat, promise of action that’s dependent on the review, or other conflict of interest and/or competition. Additionally, we will continue to intervene when a guest leaves a review that violates our content policies—including discriminatory content or a violent threat.

For more information, read the updated Review Policy or learn more about how we moderate disputes of our Review Policy in our Help Center.

How we’re tackling inconsistent reviews

Sometimes, a negative review is less about the guest’s experience in your space and more about them not understanding how reviews or the platform work—they can even be the result of an honest mistake. Earlier this year, we built a tool to help address these types of reviews by automatically detecting inconsistencies, and then interrupting the flow to give guests a chance to correct them. So, for example, if a guest gives you 4 or 5 stars in every category (cleanliness, accuracy, and so on) but then a 1-, 2-, or 3-star rating overall, a pop-up will ask if they’re sure about their overall rating.

Similarly, if a guest leaves a low rating for something like location or value—two categories we know can be interpreted differently by guests—a clarifying question will appear.

• For location, we’ll ask if the location was accurately described in the listing
• For value, we’ll ask what would’ve made the stay a better value

These interruptions force guests to think a little more about the rating they’re giving, which they can then go back and correct. As a result, we’re already seeing more consistency between the category scores and overall scores. Improvements like these help ensure that guests’ ratings align with their experience—better ratings are more useful to guests and reward the hard work of hosts.

There’s still more to come

Reviews are the backbone of our community—they help hosts grow their businesses, and they help guests gain the confidence they need to make the booking. We have a dedicated team doing lots of thinking around how to make our entire review experience better for hosts and guests. We’ll continue to improve the review system over time—please keep sending us your feedback about improvements that you’d like to see. In the meantime, we’re excited about these changes and hope you are too.

[William304]

Wonderful that you are making reviews more relevant.  Suggest you make it easier to find the place for host to enter a review

[Morag0]

I feel that the value category needs to be removed or re-worded. Perhaps 'was this listing good value compared to comparable listings in the area'

I live and host in Sydney CBD where a 2 bedroom apartment is going to cost you $1million which is reflected in the listing price. A hotel room is $200+ per night and a room in a nice central apartment is ave $100-120 per night. Guests, especially from overseas, are not aware of the cost of property and give comments like 'was very expensive'. Value is subjective and is based on someone's experience /ideas of what things should cost. 

[Chris1154]

I agree that the value is so subjective it shouldn't be part of the review system. Just had four guests stay in my villa over Christmas and gave me a 4 for value. There are very few places in all of Siem Reap that will accommodate that many people in a house and let alone charge what I charged over one of the biggest holidays of the year!

[Mange-Ram0]

Suggestion.

Overall Review rating should be calculated on the basis of weighted average. To quote :

A guest stays for 5 days and he gives a rating of 5 star

Another guest stays for one day and he gives a rating of  4 star

Overall Rating should be calculated like this :

5 x 5 = 25

4 x 1 = 4

Total 25+4=29

Average = 29 divided by 6 = 4.83

[Steven65]

Retaliatory reviews. So i have a guest that checks out, and has damaged furniture and left the place in a complete mess. I leave the guest a 1 star review and would not host him again. I am automatically emailed by Airbnb. "Would you like to request money from this guest". Well, yes i would. But as soon as the guest is aware of my action; he then posts a retaliatory and false review, right? This announcement does not appear to solve or tackle this common issue. Can Airbnb please respond. 

[Adele326]

I am pleased that the review policy has been reviewed.  what about when a guest gives 5 for all the categories but an overall 4? this doesn´t make sense. 

I have also had one guest giving an incredibly low score of 1, for a one night stay while another guest gave a consistent 5 for a 3 month stay in the same listing! The guest provided no evidence for the low rating, just assertions - ´The place was not clean´.  Rather than ´I found a thick layer of dust.....` I found hairs in the shower! etc.  A pop up that will prompt guests to give specific examples and complaints will trigger the ´honest´feedback we are seeking. 

A prompt when people are giving negative reviews asking whether they had raised the specific complaint with the host. I encourage guests to communicate their concerns to me.  Guests who have given negative reviews in all cases did not raise the concerns with me directly. I find this disturbing. 

[Steven65]

@Freddy95 I wasn't aware the compensation request also has a 14 day deadline. Clearly deterring hosts from making a claim. 

[Maria11945]

Some guests don't bother to read the description of my place and then they leave bad reviews about things that are made clear by me. Example 1: You are free to smoke in our house, we also smoke. They don't see it, they give us low rating on cleaningness and complain about the smoke. Example 2: No kitchen access (there is even a sign in the room and one outside the kitchen). They don't see it, they go in, then they complain about unwashed dishes in the sink. Example 3: We rent a room in our apartment, not a whole space (that's why I don't want people messing up the kitchen). They don't see it, they come here, they want to leave and they demand their money back, after my husband and I have spent a whole day cleaning and preparing for them, they write a bad review. I wish there was more clarity on the site, maybe images-indications, visible beyond doubt to everybody, because it feels unjust and frustrating.

[Jacqueline524]

This is all very nice but the issue your STILL need to address is superhost status for those of us that are in highly seasonal locations.  I get three to 4 months of stays - tops, but your evaluations cover mush of the time when NO ONE comes to our area to stay and thus my spot is EMPTY!

I keep losing my "superhost" status and gaining it back AFTER the season is done.  why not just leave it alone??????!!!!!!!!!!!!!!!!!!!!!!!!!

[Vural0]

This guy stays at my place, steals towels and when I raise this, he denies it, leaves a terrible review rating all 1 and accuses me of poor service. Eventually, I prove it, he admits and I get a refund but now Airbnb tells me that his review didn’t violate their policy! Something has to change immediately. Your content policy only covers major issues. How come your Case Managers can tell me that his experience was genuine?

[Shannon524]

I understand that Airbnb is a business with the top priority of pleasing and booking guests, but honestly, even though Airbnb has been good for us (we started 11 months ago), the amount of stress that the fear of getting a 4 or lower star review really bogs me down. The review system is COMPLETELY broken. For example, so many people view reviews differently. Our house caters to older people who like finer things. For many of them, though, leaving a 4-star review would be considered "well above average" which I think it accurate for our house. It's not typically what you would call a 5-star place when compared to a mansion or a Ritz or other "5-star experiences," but if they do not rate it as 5 stars, it's a strike against me.

As a host, I would love to feel like an asset to Airbnb...not easily dispensible cattle.

[Nancy1249]

I just got a four star rating this week from a guest for location, a guest who had RAVED to me about how much she loved the place and left a terrific review, so I asked politely what I could have done to improve the location info. She said, "Oh, I'd only give five stars to a place on the beach." I have never claimed to be on the beach or within walking distance. I say two miles from the beach, five minutes by car. 

This rating was made after the mid-December change in policy and the stars are still up there as four. I have good other stats so I hope this doesn't affect my Superhost status but I'm wondering why the new system didn't catch this.  

[Sarah977]

@Nancy1249  None of the individual category ratings are counted when assessing Superhost. The only rating that is counted is the "overall" which isn't an average of the category stars, but a separate rating the guests star.

[Patrick1443]

I'm glad you guys are working on the review policy, however, I feel like all the new amendments are painfully common sense, and I'm surprised they took this long to address. Of course a rival host trying to besmirch your listing should be removed.

I would like to see changes made more to the actual metrics than just common sense stuff like don't racially profile or write bogus reviews. For instance, if someone is consistently leaving 3 star reviews, maybe their review should be weighted differently instead of allowing them to tank a host's listing that's new and has only had a few guests. Or throw out the best and worst review, or weight reviews more strongly that have occurred over the past 3 months. Maybe Airbnb does this, but I haven't read anything about it. It seems like the simple five star system that everyone uses could use some upgrading because of its inherent flaws, especially when five stars are expected and 4 stars is generally unacceptable.

[Chris1154]

I have two questions based on this updated info. 

1. My villa has been reviewed with mostly 5's and a couple 4's and...….. one 1! There is no way for me to overcome that in the averages to raise my overall host score without having the house booked constantly. As that is not the case how as a host will I ever be able to overcome that one really bad guest/review? There was no direct communication with me why they checked out immediately nor what I could do for them or if they could have a refund. They then proceeded to trash the place in the review and complain to me in the private message that I did not refund them. It is the only one in two years and I still cant get the score up to super host. I think that one score should be allowed to be dropped in a case like this that obviously is not in character with the experience 99% of the guests are having.

2. Now that those pop ups are available to guests in the review process questioning their possible score (location and value) will hosts be able to receive the feedback that they provide so it might be addressed? 

Thanks for some steps in the right direction from the host side.