Making reviews more relevant and useful for our community

[Airbnb]

Many of you rely on reviews to grow your business—they highlight your amazing hospitality. They’re also a helpful way to get specific feedback on what you’re doing well as a host and where you can improve. But there are times when you feel like a review may be misleading or irrelevant to future guests, and we know that can be painful both personally and professionally.

It’s tricky. Guests and hosts need to be able to share what they feel is important about their experience, so we don’t want to limit what people can and can’t say on the platform. However, it’s also essential that reviews reflect a guest’s stay and that they provide useful information to future hosts and guests. As part of our ongoing journey to get this balance right, here are some of the changes we’re making:

Introducing our updated Review Policy

With our updated Review Policy—effective December 11, 2019—we’ve addressed two types of reviews that we know can be frustrating: irrelevant reviews and biased reviews. The updated policy clarifies our expectations and ensures our customer support agents are equipped to remove these types of reviews.

By updating our policy to cover irrelevant and biased reviews, we’re strengthening our commitment to building a community of trust. These review updates build upon our existing Content Policy (which outlines the kind of content that is never allowed on Airbnb) and our new Guest Standards Policy, which will introduce a system for tracking bad guests. Under our updated Review Policy, guests and hosts who repeatedly leave certain kinds of biased reviews may be removed from the platform.

Let’s look at irrelevant reviews and biased reviews in a little more detail:

Reviews that are irrelevant

This policy covers situations where a review contains information that’s irrelevant to you as a host or your listing—and isn’t useful to future guests.

Here are a few examples:

• A guest accidentally leaves a review on your profile that was meant for another host
• A guest never checks into your space (for a documented reason unrelated to you as the host or your listing) and leaves an irrelevant review about their experience. For example, a guest never shows up, due to a canceled flight, but leaves you a review that complains about a dirty couch.

Under the updated policy, both of those reviews would be removed because they contain only irrelevant info.

There are also times when a guest comments on issues outside of your control, or unrelated to the service you provided. Those types of reviews may be removed if they only contain irrelevant content that isn’t useful to future guests. Here are a few examples of irrelevant content that could result in the removal of a review:

• A guest leaves a comment about your appearance
• A guest leaves you a bad review because they were frustrated by public transit in your city
• A guest leaves a review about the type of people in your neighborhood

These comments have nothing to do with your listing or the service you provide as a host and aren’t useful to future guests. So, under our updated Review Policy, our customer support agents would be empowered to remove both the review content and star rating.

Reviews with biased information

Our community benefits most when reviews share an unbiased view of the member’s experience. Our updated Review Policy covers the removal of reviews with inappropriate bias—this may include situations where the reviewer is attempting to extort the person being reviewed, has a conflict of interest, or competes with the person being reviewed. Here’s a closer look at three types of biased reviews that will be removed under our updated policy:

• Extortion: This is when a guest tries to use reviews to get something they want—for example, if they threaten to leave you a bad review unless you give them late checkout. Any attempt to use reviews or review responses to force a person to do something they aren’t obligated to do is a misuse of reviews, and we don’t allow it. People who use Airbnb also aren’t allowed to tie positive reviews to promises of compensation.
• Conflict of interest: We appreciate how much hard work goes into earning a positive review. As a result, we will not allow hosts to unfairly boost their ratings by accepting fake reservations in exchange for a positive review, using a second account to review their own listing, or providing something of value (like cash or a late checkout) in exchange for positive reviews.
• Competition: We’ll remove reviews written by hosts of competing listings or experiences where that person (acting as a guest) leaves negative reviews for competitive listings, or where we determine the content is intended to dissuade others from booking those listings or drive business to other listings.

Guests and hosts who repeatedly violate our updated Review Policy may face consequences, including account suspension and removal from the platform. We’ve also invested in retraining our customer support agents and improving our workflows, so hosts will receive better support in instances when irrelevant or biased reviews happen. As with our new Guest Standards Policy that tracks bad guests, our updated Review Policy includes warnings and education that can lead to suspension or removal of people who repeatedly leave biased reviews.

Updating our Review Policy is another important step in our journey of supporting guests and hosts like you who rely on relevant, useful reviews.

Your top questions, answered

What kinds of reviews will be removed by this updated Review Policy?

A healthy review system is one that respects and protects our community’s genuine feedback. For that reason, we take the removal of any review very seriously and only remove reviews that clearly violate Airbnb’s Review Policy. You can read the updated Review Policy in full, but, in short, this means a review is only removed if:

• The review is in violation of Airbnb’s Content Policy,
• The review is biased, or
• The review is irrelevant to the author’s experience on Airbnb

Guests and hosts who repeatedly violate our updated Review Policy may face consequences, including account suspension and removal from the platform.

What kind of documentation should I have under the updated Review Policy?

We can’t emphasize this enough: Always aim to communicate with guests through the Airbnb platform. If conversations happen off-platform, be sure to keep a record of those conversations too. That way, if you ever need to report a review to our agents, they’ll have—at their fingertips—the information they need to make the right decision. That said, even when you don’t have this preferred documentation, we encourage you to report any reviews that violate our Review Policy because we may be able to identify other evidence or patterns of behavior regarding that guest. 

Will all irrelevant review content be removed?

If Airbnb determines that the review contains no relevant information about a host or guest or listing, the review will be removed. Reviews that contain mostly irrelevant information are also subject to removal, but only where the relevant information does not meaningfully inform community members.

Where a review contains information that is unrelated to an experience as a host or guest, or is focused on something beyond the control of the person being reviewed, our team will determine the relevance of the review by considering how useful it is to our community of hosts and guests. To do this, we’ll look at two things:

• Does the review recount the reviewer’s experience and provide their personal perspective?
• Is the review helpful to other members of the Airbnb community? Does it provide essential information about a host or guest, listing, or experience that would help others make more informed booking decisions?

What’s the difference between extortionary and retaliatory reviews?

It’s considered extortion if a guest attempts to use reviews (or review responses) to force a host to do something they aren’t obligated to do. So, for example, if a guest threatens to leave a bad review if you don’t allow them to bring additional guests, that review would be extortionary and would be removed under the updated policy.

Then there are times when a host may feel that a negative review is made in retaliation. This is when, for example, a host doesn’t allow the guest to bring additional guests, and the guest goes on to leave a review about how inflexible their host was, or even writes a negative review about cleanliness or location. However, without evidence of a threat to leave a negative review, this would not be considered extortionary and would not be removed under the updated policy. If this happens, we encourage hosts to use their public response to politely address the issue.

Why aren’t you removing all retaliatory reviews?

While we understand how frustrating it can be when you receive a review that feels retaliatory, we don’t have a crystal ball to tell us what a person’s true motivations are. So, without a documented threat to leave a negative review or other evidence of a biased review, Airbnb won’t intervene. Here’s why:

• As a marketplace, we often don’t know the truth about what took place, and our review system is a critical feedback mechanism for our hosts and guests. That means we want as many reviews as possible to remain intact—so our community can put them to good use.
• Most guest reviews contain honest feedback about their experience and useful information for hosts and guests.

To reiterate, as outlined in our updated Review Policy, Airbnb can—and will—intervene where there’s evidence of a threat, promise of action that’s dependent on the review, or other conflict of interest and/or competition. Additionally, we will continue to intervene when a guest leaves a review that violates our content policies—including discriminatory content or a violent threat.

For more information, read the updated Review Policy or learn more about how we moderate disputes of our Review Policy in our Help Center.

How we’re tackling inconsistent reviews

Sometimes, a negative review is less about the guest’s experience in your space and more about them not understanding how reviews or the platform work—they can even be the result of an honest mistake. Earlier this year, we built a tool to help address these types of reviews by automatically detecting inconsistencies, and then interrupting the flow to give guests a chance to correct them. So, for example, if a guest gives you 4 or 5 stars in every category (cleanliness, accuracy, and so on) but then a 1-, 2-, or 3-star rating overall, a pop-up will ask if they’re sure about their overall rating.

Similarly, if a guest leaves a low rating for something like location or value—two categories we know can be interpreted differently by guests—a clarifying question will appear.

• For location, we’ll ask if the location was accurately described in the listing
• For value, we’ll ask what would’ve made the stay a better value

These interruptions force guests to think a little more about the rating they’re giving, which they can then go back and correct. As a result, we’re already seeing more consistency between the category scores and overall scores. Improvements like these help ensure that guests’ ratings align with their experience—better ratings are more useful to guests and reward the hard work of hosts.

There’s still more to come

Reviews are the backbone of our community—they help hosts grow their businesses, and they help guests gain the confidence they need to make the booking. We have a dedicated team doing lots of thinking around how to make our entire review experience better for hosts and guests. We’ll continue to improve the review system over time—please keep sending us your feedback about improvements that you’d like to see. In the meantime, we’re excited about these changes and hope you are too.

[Steven532]

Interesting issue.

My view is that unless the host has said something wrong about the public transport in the area, then your feedback has nothing to with the host and it is entirely the responsibility of the guest to do their own research - just like I do for wherever I go as a guest. Your host is providing accommodation, not planning and services for your whole trip - usually.

The reviews that said it was a dark and lonely street are interesting. Potentially fact based and very important if the host had said otherwise but again, yes you should do your research - Google maps is very easy to research with, as are many other tools.

[Deirdre11]

So I should Google "will bus drivers in Madeira be able to understand my pronunciation of Portuegese place names?"? If I book at a place I need to know if it is easy to get in and out of. Why should it be wrong to comment on that? And as for using Google Maps, you don't get the address until after you've booked, so that's pointless.

[Lynnie1]

I use clues in the pictures and the location on the two maps and can usually find the place and 'cruise' the street on Google street view.

[Deirdre11]

That's a pretty tedious exercise if you're narrowed it down to the final five and you're booking places in Madeira, Poland, Strasbourg, Hungary and Romania as I did recently.

[Sarah977]

@Deirdre11  An Airbnb review is meant to review the accomodation and the host, not other aspects of your trip, like transportation, etc. You can post those sorts of reviews and comments on places like Trip Advisor. 

Would you want your host to review you as to whether you left the place clean, communicated well, and other aspects of how you are as a guest, or would you want them to talk in their review about things which are irrelevant to your suitability as a guest, like the style of clothing you wear, your accent, or whether your luggage was nice and new or old and ratty?

If a host states that it's a 5 minute walk to the nearest bus stop, and it turns out to be 15, that's a relevant thing for a guest to mention in a review. Whether local taxi drivers can understand your pronunciation is a personal matter and has no place in an Airbnb review.

[Deirdre11]

What's wrong with people having as much information as possible? Then they can make the best choice for their trip and we get the guests who are going to be most happy with our place. I'm long over the delusion that people are coming to ski in Hakuba, Japan because they want me to be their new best friend. Their only two questions are "where's the key?" and "what's the wifi password?". I'm an irrelevant part of the whole experience.  Location is of prime importance to them and they often comment on it in their reviews. I used to have another place that was much more isolated, it was great for back-country skiers though, and I tried my dammdest to attract that type of guests, I even had a heading "For powder-hounds only". And then I'd getting bookings from a family with three little kids and Nana along as babysitter. Someone in the family, I'm guessing it was Dad saw himself as a back-country powder guy, even if the rest of them couldn't ski. Is it really my job to tell them "No, this is not suitable for you"?

Finally, finally!, in a review, somebody wrote "it's too far from the restaurants for the kids to walk". I was so glad somebody finally said it and bookings from families backed off. Now personally, I don't think you go skiing in Japan for the restaurants but it seems an important consideration for a lot of people. I can truthfully say "Restaurants are ten minutes walk away" in my listing but should I add, "except not for little kids who are tired after a long day's skiing and it's uphill on the home stretch anyway"?

If you're the family who read that review and decided to go somewhere else, wouldn't you be pleased that you had that information?

I really don't feel good having guests who aren't happy, even if they do like my clothes. And they should, I make a big effort.

[Manasui0]

@Deirdre11 

My villa has a ski in and ski out amenities.. but the only problem is .. my villa is located n Bali..

I thought it would be funny..

since no body complain.. it give a +1 amenities..

I mean for argument sake, if there is any.. you can literally ski in and out of it.. 

if there is any snow around,... hahahhahahahahhahaha

[Claudia2242]

I think it's appropriate to note potential difficulties.  Our property is on a wooded lake.  There's no beach for wading into the water, so I state that.  I also note that the property is hilly and may not be suitable for those who have trouble walking up and down uneven, hilly ground.  I've had guests in their 40's say it was too strenuous to walk up the hill and others decades older not complain at all.  It all depends on the person and their perception.  As hosts, we try to be as honest as possible so people can judge accurately for themselves.

[Lana-and-Jamie0]

I concur with your comment about irrelevance.

As to "sketchy"...that is incredibly subjective.

One person's "sketchy", is another person's "kitchy". One person's "dangerous", is another person's "lively, and social". My neighborhood is up-and-coming, for example - however some may describe it as being a run-down-dump, since they don't see the upward movement of the construction and renovation happening in our historical, urban neighborhood on a one night stay - especially if driving in from the North, and at night. It's one of the most walkable neighborhoods in the region, and certainly in a transitional stage, but many historical buildings have been maintained and renovated, and many others are actively being renovated. 

A person walking down the block looked at my guest once (he actually told me), and he responded to this interaction in his review as "staying in the ghetto", which personally, sounds extremely biased and shortsighted - to say the very least.

That is exactly what happened. 

That review was willingly removed by CM without hesitation. 

So, it is appreciated that location isn't valid in cases like this.

[Dwight34]

Aloha Mark,

Good points.

As for, "Also, I'm not sure why guests should be prevented from commenting on the neighborhood, there certainly are sketchy type areas that appear dangerous or run down and that would be information that future guests would want to know."

It is an illegal practice called Steering (Fair Housing Act 1968).  From RE for Dummies.

"Steering is guiding, encouraging, or inducing people in some way to move to or stay away from a certain area or neighborhood, and it’s illegal. Overt steering is easy to understand and avoid. Subtle forms of steering in the name of being helpful, like saying, “This is a good neighborhood,” or “You wouldn’t be happy here,” also should be avoided. Self-steering is prohibited also."

Mahalo for your kind attention.

Dwight

[Stacey23]

Yes! This! Exactly. It's dog-whistle racism. It's ridiculous that guests are allowed to judge neighborhood in this way that is illegal under any other context.

[Steve308]

Not sure I agree totally on your view. Example. I have stayed in a hotel (not Air B&B) that turned out to be a very dangerous area.....I want to know that beforehand and if you (I) are traveling on business you don't have the time to determine what you're getting into. I have also stayed in an Air B&B in what seemed to be a safe area. It wasn't. It's not easy, sometimes not possible, to determine safeness but it's reasonable to be fair about what you're getting into.

[Rosabeth0]

True! 

[Maggie46]

Totes agree about the "retaliatory reviews...." so frustrating and seems pretty easy to see through- Air bnb has evolved, changed of course, and guests are well, expecting a different experience- more like  "hotel" i'm finding more and more, as opposed to staying in someone's home.  And, at least where I live, in Burlington, VT many properties are rented out or bought and used uniquely for Air bnb short term rentals - so it IS a different experience than staying in someone else's home.  I wonder if it's time for a new Air bnb Category:  REALLY staying in someone else's home OR staying in a home someone has rented or bought to Air bnb out...

[Suzanne529]

Hello! I totally agree with this. We have self check in and check out- and a local manager who is on call for any guest issues. Although our apartments are accurately described as self check in and check out I have had bad reviews saying it wasn't a true Airbnb experience. When obviously if you are booking a self contained apartment for 11 people, it isn't the same as staying in someone's spare room.  A separate category- of self-contained apartments or units- would be a great idea.