Airbnb Answers: Protecting you from one-off bad reviews

[Airbnb]

You asked: Can Airbnb protect hosts from one-off bad reviews?

Reviews are so important. They not only impact the success of your business, they’re also really personal. We know you put a lot of thought and care into your hospitality and that it’s frustrating when you receive a review that’s uncharacteristically low—be it a mistake, a misunderstanding, or an unfair assessment. You’ve raised concerns about this, and we want you to know that we hear you and we’re taking this issue really seriously. We’ve invested and will continue to invest a lot of thought and effort into how we can make the review system more fair.

The good news is, we’re already making a lot of progress. Here are some updates we can share with you now, below, and we’re committed to keeping you in the loop as we roll out more changes.

A tool to detect outlier reviews

At the last Host Q&A, in June, 2018, we said, specifically, that we’d look into outlier reviews. To be clear, one-off low reviews can be considered outliers when a host has otherwise great review scores, but a single guest leaves a bad rating that seems out of place. Here’s a solution we’ve come up with: We’re working on building new tools that will automatically detect when outlier reviews like this occur— and give us an opportunity to correct them. So let’s say a guest gives you five stars for cleanliness, accuracy, check in, and so on— for each of the sub categories— but then a two-star rating overall. The new tool will flag this and prompt the guest to correct the overall rating. We’re developing this new tool now, and you can expect to see it soon.

Research into how those reviews affect Superhost status

In addition, we’re doing research to see how a single outlier review can impact a host’s ability to gain or retain Superhost status. The ultimate goal is to find ways to make outliers less impactful for terrific hosts and to make sure the review system is fair.

Location, location, location

Finally, we’re looking at the Location rating and how that can affect a host’s standing. It’s worth noting that Location isn’t used in calculating a host’s overall rating. But you’ve told us that you’re frustrated when you get a low rating in this category since where your home is located is out of your control. And we hear that.

This is a tricky one. It’s a hard concern to address because judging the quality of a home’s location is so subjective. Some guests may love that your space is off the beaten path; others might be disappointed it’s not closer to public transit. We’re doing a lot of research into how we can continue giving travelers the information they need and want when they’re making booking decisions and, at the same time, make sure hosts aren’t affected by something that’s out of their control.

So we’re looking into new ways to collect and highlight guest impressions of your home’s location information. While we don’t have a concrete announcement on this yet, you can expect to hear more details soon, perhaps at the next Host Q&A. In the meantime, please know we’re working on this. Keep sending us your feedback on how we can make the review system as fair as possible.

View this and other answers from the Host Q&A here.

[Salem2]

I think another point to make is that many guests don't understand the impact of a negative review. If they did they might pay more attention to it.

[Ping-And-Daryl0]

@Christabel0.This proves my point about human behavior. Some people just don't give perfect 5 stars. AirBNB development or whoever dreamed up the idea of 5 stars as the norm are kidding themselves. Impossible "fake" statistics!

[Ryan66]

Your status as a Superhost is constantly on thin ice, not because you are a bad host, but because you are being cheated by the system.

Airbnb’s system, the thing you thought was there to help you, is in fact against you because of Airbnb’s lack common sense.

Removing location as a star rating isn’t going to solve anything because the system is broken on larger scale, that being the final OVERALL RATING that the guest gets given (and abuses), overrides the location ratings anyway.

This is causing you to lose MANY 5 star reviews that you should have actually gotten, thanks to this very broken system that Airbnb simply cannot seem to understand is broken for some reason.

Here is the proof.

When a guest reviews you, they review you based on the SIX (6) SUB-CATEGORIES, presented to them, which are:

• Accuracy
• Check-in
• Cleanliness
• Communication
• Location
• Value

Each SUB-CATEGORY is rated out of 5 stars. With a total of 6 SUB-CATEGORIES, this amounts to a total of 30 stars (6 x 5 = 30), which is considered 100%. 

Although, guests also get given a pointless but very powerful 7th review option, which is the OVERALL RATING and this is where the problem lies.

The OVERALL RATING overrides the SUB–CATEGORY RATING as the FINAL REVIEW on your profile, making the SUB–CATEGORY RATINGS have no influence on the FINAL RATING, which is a MAJOR PROBLEM, as guests can wreck havoc using it as well as completely lie.

For example, I have had guests give me 5 stars in all SUB-CATEGORIES (30/30 starts), but then give me a 3 or 4 star in the OVERALL RATING (for some unknown reason) and then I sit with a 3 STAR FINAL REVIEW when in fact I achieved perfect results in all my SUB–CATEGORIES.

Thanks to Airbnb’s broken system, all the 5 starts I received in the SUB–CATEGORIES aren’t used to rate my service, only the one off OVERALL RATING is seen as “important”

This is NOT OK Airbnb!

------------------------------------------------------------------------------------

FINAL REVIEW

Forgetting about the SUB–CATEGORIES for a moment, let’s have a look at your FINAL REVIEW RATING, which is based off 5 stars.

This means that FINAL REVIEW star has a value of 20%, which is the direct influence on your Superhost rating of 4.8/5 STARS, which is a 96% rating. 

• 1 FINAL REVIEW STAR = 0 to 20%
• 2 FINAL REVIEW STARS = 21 to 40%
• 3 FINAL REVIEW STARS = 41 to 60%
• 4 FINAL REVIEW STARS = 61 to 80%
• 5 FINAL REVIEW STARS = 81 to 100%

I will use this to PROPERLY calculate the SUB-CATEGORIES to give us a FINAL REVIEW RATING, without the need or the currently useless 7th OVERALL RATING option.

Here is how:

So, knowing that we have a TOTAL of 30 STARS in our SUB-CATEGORIES, lets assign them as percentages to be used as the FINAL REVIEW RATING as follows:

• 1 FINAL REVIEW STAR = 0 to 20% (0 to 6 sub-category stars)
• 2 FINAL REVIEW STARS = 21 to 40% (7 to 12 sub-category stars)
• 3 FINAL REVIEW STARS = 41 to 60% (13 to 18 sub-category stars)
• 4 FINAL REVIEW STARS = 61 to 80% (19 to 24 sub-category stars)
• 5 FINAL REVIEW STARS = 81 to 100% (25 to 30 sub-category stars)

So what this is basically saying is that if you only achieved anything between the minimum of 19, to a maximum of 24 SUB-CATEGORY stars, you will get a FINAL REVIEW RATING of 4 STARS.

If you achieved 25 to 30 stars you get a 5 STAR FINAL REVIEW RATING and so forth.

-------------------------------------------------------------------------------------

Lets look at some example cases.

In order to be between 19 to 24 total subcategory stars (which would be a 4 star FINAL rating), a guest would have to rate you something like this:

• Accuracy : 4/5
• Check-in : 5/5
• Cleanliness : 2/5
• Communication : 5/5
• Location : 3/5
• Value : 4/5

4+5+2+5+3+4 = 23/30 stars. The above example review shows that you communicate well, (check-in & communication), you location isn’t the best (not always in our control) and you charge a little bit too much (debatable depending on guest’ s budget and time/season of booking). Mainly you could do a better job on cleaning the apartment. Not a train smash, just clean up better next time, or make sure that towel doesn’t have a stain on it.

You still managed to get a 4 star, which is fair as you worked hard otherwise.

------------------

Anything above this, (25 to 30 stars) should be considered a 5 star FINAL rating, which is the case in most of my rating.

I get something more like:

• Accuracy : 5/5
• Check-in : 5/5
• Cleanliness : 5/5
• Communication : 5/5
• Location : 4/5
• Value : 5/5

5+5+5+5+4+5 = 29/30 stars. Great, so I clearly expect a 5 star review as I achieved 29/30 possible stars.

NOPE!!! I got a 4 star FINAL RATING!

This caused major damage to my very fragile Superhost status.

Why is this?

Well because Airbnb’s system is cheating you out of all your hard work and allowing the guest to ALSO rate you using the useless 7th OVERALL RATING option, making all the hard work you put into get 5 stars in your SUB-CATEGORY stars, absolutely pointless.

Taking my review of 29/30 as an example, I did well in all sub-category fields except location, where he gave me 4 stars, because he expected a ocean view (when I never promised one) as my apartment is clearly listed in town with a mountain view.

Now in the guests mind, all he remembers is the 4 star for location (human nature) and thus also gives his OVERLALL RATING a 4 STAR, which makes my FINAL RATING, a 4 overall which is NOT FAIR.

• My photos and listing details, represent exactly what the guest got - 5/5
• I also am a very organized host and strive to deliver the best communication possible. Again I got 5/5 for that.
• I always make sure the place is spotless, 5/5 well deserved.
• My co-host was there on time for checking and everything went smoothly. 5/5 again.
• He was happy with price – 5/5 for value.

Even if he did think location was a 4/5 star, its not a problem IF the Airbnb system did not use the 7th OVERALL RATING option, because I would still get a FINAL RATING of 5 STARS if it was properly calculated using ONLY the 6 SUB-CATEGORIES.

Why?

Because I would still have 29/30 stars which would fall between the 25 to 30 sub-category stars, giving me a TRUE FINAL RATING of 5 STARS.

But thanks to the pointless 7th OVERALL RATING option, all that literally throwing them into the trash, even though I did brilliantly in all of them.

This is not fair, Airbnb are stealing your hard earned reviews from us.

---------------------------------------------------------------------

PROOF:

Don’t believe me? Go to your https://www.airbnb.com/progress/ratings page and select the “Filter by overall rating” to 4 or below (anything except 5 stars)

Now, take any of your ratings, let’s say you filtered 4 stars and calculate up the total sub-category stars you earned. Then compare that to these scoring brackets:

• 1 STAR final = 0 to 20% (0 to 6/30 sub-category stars)
• 2 STAR final = 21 to 40% (7 to 12/30 sub-category stars)
• 3 STAR final = 41 to 60% (13 to 18/30 sub-category stars)
• 4 STAR final = 61 to 80% (19 to 24/30 sub-category stars)
• 5 STAR final = 81 to 100% (25 to 30/30 sub-category stars)

If you achieved from 25 to 30 sub-category stars, you should have gotten a 5 star rating and NOT a FALSE 4 star or below.

I did this with my TEN, 4 star ratings of mine and was in absolute disbelief that NINE (9), were in fact supposed to be 5 stars!

Yes you read that right, 90% of my so called 4 stars are incorrect. Some guests even gave me 30/30 sub-category stars, but then gave me a 4 star overall rating, with no reason why - This is insane!

Seriously Airbnb, are you seriously not seeing the VERY BIG issue here?

I then even went to my two 3 star reviews, which actually added up and that came up to 23/30 and 28/30 sub-category stars. Meaning that the first  is supposed to be a 4 star and NOT a 3 star, and the second a 5 star and not a 3 star.

In fact the first was falsified and by doing by the sub-categories, we can see evidence to this, more of that further down this post.

I have put up examples of all these reviews to show you as proof here. https://drive.google.com/drive/folders/1UI2ezQx9LqJ5MEJKxtXUobNAh3OSwWnd?usp=sharing

THE SOLUTION:

• Its simple really: REMOVE THE OVERALL RATING – Its BROKEN AND WE DONT NEED IT.

Ill repeat this as you have a knack of conveniently “ignoring” certain parts of what we write to you:

REMOVE THE OVERALL RATING - Your sub-category ratings should AUTOMACTICALLY be handling all of this, NOT THE GUEST.

He/she, already left this sub-category reviews based on the 6, INDIVIDUAL areas of service delivered to them by the host.

The rest is up to your system to calculate and give the Host the final rating, NOT THE GUEST.

--------------------------------------------------------------------------

WHAT ELSE WILL THIS SOLVE:

Well Airbnb, I don’t know if you have noticed, but your Facebook page is an absolute DISASTER with guests complaining about fake and false reviews, left right and centre.

Now please enlighten us as to why you think this is and why you still don’t seem to see an massive problem here?

• Is it because 90% of all hosts are bad? I think not
• Is it because rating a host based on something that wasn’t promised in their listing (let’s say aircon as an example) is fair? I think not.
• Is it because a guest should be allowed to review you, even if the never stayed at the apartment – I think not
• Is it because your rating system is broken and stealing our hard earned – YES! AS PROVEN.

So actually LISTENING for once and removing the very BROKEN OVERALL RATING, will not only make the reviews fair, but also big a big help in filtering FAKE reviews.

How you ask?

Example 1:

This overall final rating option opens the flood gates to guest, being able to just angry, impulsive 1 star a host, because they weren’t happy with a small thing that ticked them off, let’s say a dusty desk, or a drop in internet for a few hours, even though they gave them a much higher sub-category ratings for the rest. This is an immediate indication of a false review. An overall rating is too much power for some (actually most) people and they abuse the power.

Example 2:

Now if a written review says something like: “The apartment had no aircon”, but they gave you a good a 5/5 for value, isn’t this a massive contradiction?

I am offering them my place without aircon at a very affordable rate, (which they KNOWINGLY accepted, before booking). If they wanted aircon they should have chosen a place with aircon, most likely PAYING MORE for that listing, compared to mine, but they are cheap and they don’t want to do.

I am aware it doesn’t have aircon which is why my rate is lower. Aircon is a not only a massive upfront cost, but also a big running cost for electricity. I simply cannot provide aircon and the low price point together. I would be running my AIRBNB BUSINESS AT A LOSS. If the guest is happy with the value, then you CANNOT complain about a luxury item, that should be costing them, far more per night. Again a FALSE REVIEW.

CONCLUSION

So Airbnb what are you going to do about this? I have tried to communicate this with your support staff multiple, painstakingly times and just get throw the same copy, paste terms and condition link pages in my face, that have ABSOLUTLYE NOTHING to do with this issue.

Yet you feel you have ‘solved” this HUGE issue.

You create a huge “talk” and forum about it, when I’ve posted this to you months ago and you didn’t even have decent to reply.

Its really not rocket science, it just you plain arrogance that is the issue here where “you are always right”.

The solution itself can be done in less than a week by your coders.

[Ping-And-Daryl0]

@Ryan0 Brilliant analysis and insightful ideas. But it seems like the wheels of progress turn very slowly at AirBNB. Words are cheap, let's see some action!

[Steven65]

Better to add the overall rating to the subcategories 

• Accuracy
• Check-in
• Cleanliness
• Communication
• Location
• Value
• Overall

Where 5 stars meets expectations and accuracy. Then average them all for a final rating.

[John220]

I just had such a one-off review after the newsletter came out and all I got from the Airbnb rep was the application of the former policy, which offers no protection from such reviews and reflects a very poor understanding of the whole review process. The Airbnb rep also said that the policy is to ensure that people prepare honest reviews. But the review that I got was anything but honest. Nor was it true or fair. Also, the guest made comments that were inconsistent with the listing and the amenties I provide. Surely such inconsistencies should be reviewed, easily detected, and counteracted by Airbnb. 

[John220]

To indicate how ridiculous guests can be about the numeric rating consider what friends of mine have said about their ratings. They said, sight unseen, they'd give a listing at best 3 stars no matter what.

Another friend, who has a place on Airbnb, says she asks her guests to give her a full 5 star reviews.

These are what Airbnb considers to be unobjectably 'honest' reviews .

[Linda1224]

My impression is that guests either give 5 stars or don't review and hosts do the same with guests. It makes a nonsense of the system really . I suggest that Airbnb should only threaten to delist when hosts are regularly given 1 or 2 stars and that 4 stars  should be recognised as a very good score.

[Daniela678]

I never did understand the location rating. Before I book, I look at the general location and decide if it is where I want to be. Why rate it? Look at the map and decide, it part of planning your trip. 

The one thing I want to know that drives me up the wall:

HOW COULD I HAVE EARNED THE 5TH STAR? 

some guests have a fabulous time, we connect, they are bubbling with what a great place I have all my attention to detail etc.. then leave 4 stars!!!! 

I am very detail oriented and I NEED TO KNOW. Why they give the rating they do. What could I have done to earn the 5th. What did I miss or they not see? 

Pushing the guest to detail their rating .. WE ALL NEED TO KNOW. 

Some people just don’t give 5 stars.. WHY? I must know. Please Airbnb guide the guest to detail their less that 5 star rating and let go of location rating. 

[Michael2413]

4 stars on "value" and "location" keep me up at night.

Airbnb guests have no idea that this is a homeshare and NOT a hotel.  Everyone who comes just thinks it's a cheap hotel...and we get ruined if we were at work and weren't able to clean up their hair in the shower.  Or we didn't give them early check in ...2 minutes after someone else checks out.

[Tim-and-Holly0]

@Airbnb:
As a seasoned host who has hosted over 2,000 guests in our homes, location continues to be the most frustrating rating. We have one home that is across the street from an African-American family. This is a family of 8 (6 kids) and the man is actually a preacher and runs a ministry helping African refugees displaced in the United States. Yet we continue to receive low location reviews due to "black people in the area," "black men/teenagers standing across the street," (in their yard or sidewalk in front of their home), and drug deals across the street" (remember this is a preacher). 

We had a guest stay in another home and praised in the public review that the home was "only 10 minutes from downtown" yet gave it low stars and said privately that the reason was that the home was ten minutes from downtown. Had she read our listing or even looked on a map with minimal research she would have known that PRIOR to booking, thus would have no reason to lower our location rating.

"Location" is so subjective it simply needs to be removed entirely. If guests who stay do not even know the criteria of "location" then how can we expect potential guests to figure it out?

[Sarah977]

@Tim-and-Holly0  I would report these guests who make discriminatory comments in their reviews and ask Airbnb to remove those reviews. That type of commentary is totally against Airbnb's guidelines for what is acceptable in a review and those guests should also be banned from Airbnb, as their behavior is contrary to the anti-discrimination policy Airbnb is so proud of.

[Peta7]

Reporting the guest will be doable in a perfect world, unfortunately in the real world we have the infamous CS department without insight or language skills always siding with guests.

We had a recent incident where the guest made racist statements in writing and offered us cash for extending his stay in the same thread and this all resulted in us receiving our last warning for discriminating against the guest who Partied, got sozzled, attempted to HiJack our Residence and made sexual advances at an elderly host while being under the weather.

[Manager0]

I do not think it is helpful for airbnb to send to a host a warning on how their superior host title can be jeopardised after one bad review. The review in question was from a guest who did not arrive on their day and arrived the morning after, with the door code, pushed past the cleaner and used the apartment knowing they were not supposed to be there. It was well past check out time.  And airbnb has the nerve to warn me. Seriously this is not decent and goes against what we are here for. If you want to warn someone you may want to look in the mirror.

[Allison50]

@Airbnb2

1.  Remove location rating completely - it’s already shown on the map and is too subjective.

2. Add a scale to the star rating (1 = poor; 5 = great/as expected) to eliminate any misunderstanding and explain scale the same to both host and guest

3. Reverse the order of would you recommend Yes No so that it is No Yes, or better yet make it vertical with Yes on top and No below.  When clicking through ratings, 5=excellent is on the right hand side of the screen so you click right, right, right for consecutive 5-star ratings, then suddenly the positive rating of yes is on the left side of the screen.  It’s easy for people moving quickly to click 5, 5, 5, No.