Making reviews more fair for hosts

[Airbnb]

 Many of you have asked us how Airbnb can protect hosts from one-off bad reviews. When this question came up at the most recent Host Q&A, we told you we were working on ways to make the review process more fair for hosts. Specifically, we made 2 promises:

1. We committed to launching a tool to detect outlier reviews—or one-off bad reviews. A common example is when there’s a discrepancy between the overall rating and the category ratings provided by a guest (when a guest gives a host 5 stars for cleanliness, accuracy, and the other categories but a 2-star rating overall, for instance).
2. Based on your feedback, we also committed to exploring ways to help guests better understand that the location category rating is meant to be objective. The location category rating doesn’t impact your overall rating (or Superhost status), but we know it’s important to you, and we want to make sure the whole system is as fair as possible.

Today, we’re excited to announce two improvements to the review process that directly address these issues. Since these changes have been introduced, we’ve already noticed a tangible uptick in more accurate, fair reviews for hosts, and we hope they solve some of your pain points. Here’s what’s new:

One-off review alerts

We’ve added a step to the review process for guests when they give a host an inconsistent overall rating. For instance, the guest may have given 4-star or higher ratings for all the categories (cleanliness, accuracy, etc.), but then give an overall rating of less than 3 stars.

The new pop up screen asks guests: “Is this right?” And goes on to explain that they rated their overall stay lower than they rated it in specific categories. It gives guests an option to either change the rating or ignore the alert.

This new alert has led to higher overall review ratings for hosts. Since we launched, we’ve seen a 2.8% drop in 3-star reviews and a 3.9% drop in 2-star reviews. While these percentages may seem small, they’re driving real improvements in the accuracy of our review system, and hosts are benefitting.

Location, location, location

We’ve heard from you that the location rating can be particularly frustrating because some of you have experienced guests dinging you in this category, unexpectedly, after great stays. This category is tricky. It gives valuable information to prospective travelers, which we don’t want to lose. At the same time, we hear your concern that you’re being graded for something you can’t control: guests’ opinion of your location. This opinion is inherently subjective—one person’s “rustic rural retreat” may be another’s “too far from public transportation.” So we made it more clear in the review process that guests are rating the accuracy of your location description, rather than the location itself.

Now, when a guest goes to rate you in the location category, if they give you less than 3 stars, they see an explanation: “Was the listing’s location not described accurately?” So far, this has led to a 0.8% increase in the average rating for location.

While we were working on this, we also made similar improvements to the value category. If a guest gives you less than 3 stars there, they’ll see this message: “What would have made this listing a better value?” This has led to a 0.25% increase in the average rating for value.

These changes were designed to begin to address your concerns around unfair reviews, and to help make sure that guests understand what ratings mean. We still have a journey ahead of us to keep making the review system better, and you’ll continue to see updates from us on this throughout the year. Thank you for hosting!

[Loren35]

Might I suggest the following approach to all my fellow hosts as Airbnb seems slow if not reluctant to take action similar to this:

Welcome to Our Airbnb Home

This is especially designed for the many new to Airbnb guests that we host. For many of you, the review system for host and even for you as a guest is new and sometimes confusing or challenging. As a guest you are reviewed in 3 areas with a possible 5 points for each area. The total number of possible points is 15. The areas are 1) Communications 2) Respecting the House Rules 3) Cleanliness. Let’s say you get 5 points for communicating well; then 5 points for respecting the rules and 3 points for cleanliness. That is 5+5+3=13 total points divided by 3= 4.3 for your overall score. Hosts see these review scores and can decline guests who have low scores. No one wants or needs a “problem” guest(s). Also, a host can indicate to Airbnb that they would NOT, repeat NOT recommend a guest(s) to other hosts. So remember you are not just a customer, you are a guest in our home. Please act accordingly.

Cleanliness is the most often low scoring area based on our experience. A pillow, blanket or sheet left on the floor; bed that is unmade and has the mattress cover pulled down; spots/stains on the duvet or sheets. We have guests who make the bed so it looks ready for new guests or remove the sheets, pillow cases to make washing them easier. That is someone who is being thoughtful and took a few extra minutes to think about us. Notes left by guests are also signs of thoughtful actions and gratitude. These kinds of acts earn 5 stars.

Communication is another key area. If you did not book this reservation or you are adding an extra person, you NEED to let us know in advance. Having extra guests beyond the 1 or 2 normally expected, has an extra charge in our home. ( Extra electricity, water and gas usage.)

Note well: ALL reviews are permanent!!! They cannot be changed as Airbnb will not do so, even if you would like to or the host would like to change them.

Hosts are rated in 6 areas with a possible 30 points. Two of those areas are pretty subjective: 1) Value and 2) Location. Now here are things that leave many hosts shaking our heads because we miss the logic. Let’s say you booked with us and you had the general location followed by the EXACT location after booking. You then give us a 3 or 4 star location score. Every other area was 5 stars. The Overall score the host sees is 4 stars. Let’s do the math. 5+5+5+5+5+3=28 points. Divided by 6 = 4.6. That score is closer to 5 points than 4 points. We know of guests who have given us 30 points and overall scores of 4 points!!! The logic of that escapes anyone who uses math objectively.

One other area that leaves us puzzled are the guests who give a less than 5 star rating in ANY area with no explanation. We have had guests say the place was not 5 star clean. When asked what was dirty, the answer came back that there was “clutter” somewhere. OR maybe the “Check-in” area or “Communication” area were less than 5 stars. The whole point of a review is to CONTRIBUTE to the host or guest what was MISSING so that it can be improved. Most hosts want 5 star reviews. That is one reason we ask that any issue or concern is promptly communicated to us. We ask that you observe and accept this request.

Hosts getting low reviews lose money is a pretty solid fact. We are in business to make money by satisfying our guests. We do our best to under promise and over deliver.

So we hope this clarifies the rating system for you. If you have any questions, please ask. We will do our very best to answer them. And to be clear, we may deserve less than 5 stars on some area(s). We are human, too.

[Sarah977]

@Loren35  I appreciate most of what you've written, but the scores don't work like that. Each score is not given points to be added up to reach the Overall score. The overall score is a separate rating, not an average of the others. A guest may rate all sub-categories 5*s, but leave an Overall 4* because there was something that made their stay less than great which isn't covered under the other categories, like perhaps they found the bed quite uncomfortable.

The review system is faulty and needs to be overhauled, for sure, but it seems that you are confused about how it presently works.

[Loren35]

Sarah, I'm a Super Host that has been with Airbnb for over 2 years.  What you are pointing to is a bit confusing in that it would appear that guests ARE rated based on how many of 15 possible points divided by 3.  Plus guests have NO "over all" score given by the hosts.  That shows up as a double standard in my world.  Also, when one communicates something it is important to check in with the other party to see how it lands.  Why?  Well, we humans have filters.  So I may say "X" to you  and you hear "Y" due to your personal filters.  I doubt that I am the only host who does not get this system based on what you said.  That in itself is a pointer to what is a breakdown in the review and communications in general with Airbnb.  

Here is another issue.  Airbnb has been in business for some time now.  Maybe 8 or so years.  Where is the leadership, partnership and commitment to deal with what I hear is an important issue for many hosts?  Let's take the area of Communications for rating guests.  We are supposed to "assume" that a guest has communicated all their concerns, requests, etc. to us.  Only, when we see our rating, that may not be the case.  So we "ass/u/me" to give them a Communications review that has an obvious flaw.  I have personally followed up with several guests who gave less than 5 star reviews regarding "What was missing?"  Response-None given!!!  How can one make improvements based on ratings that lack FACTS?

You make a good point about an issue like a bed; I can think of noise from neighbors or dogs as another.  Still the list of possible other issues is FINITE or limited.  And when one DOES give such a lower star rating, the host is clueless as to why in many cases.

People that are serious about their commitments make PROMISES.  "I will do X by Y time."  
That is a clear missing in this whole conversation with Airbnb.  If I decline a guest that has several Airbnb visits and has a 4 star or less rating, I suffer a loss of my Super Host rating.  How do any of us know if we have a guest(s) that are prone to damaging items and being less than responsible and accountable for paying for them?  And letting these same guests post retaliation reviews is completely unacceptable.

Thank you, Sarah for adding to the conversation.  If anything that I might contribute forwards the creation of a fairer, more objective review system, it was worth my time and effort.  I will add one thing:  "You don't listen!  You never have listened!  You NEVER will listen!" was what a coach told me and others.  Once you sit with that and accept the possibility that there is TRUTH to the statement, the possibility of real listening is available.  Btw, a test of one's listening is to ** record** a conversation you have with another with a give and take.  Meaning no one does all or the majority of the talking for 2 minutes.  Then see if you can "word for word" state exactly what they said.  The next level is whether you can do that AND state exactly what they said compared to the recording.

[Denis-And-Valerie0]

Airbnb, it's really simple. "One-off" reviews, whether given by guests or hosts, should trigger your individual involvement. Call the reviewer and get more details. Then call the reviewee. If there is no actual proof given of the bad review, then it should be deleted. There are lots of "revenge reviews" out there from people who cause their own bad experience, and want someone else to pay. No proof--no case. Don't punish a good host/guest for doing a good job! 

[Nina444]

This thread overwhelming demonstrates the system is not working. Hosts are subject to petty reviews and are loath to leave poor reviews for fear of revenge ratings which affect their hosting status. If I represented Airbnb I would feel I was letting my host community down. Guests need to be aware of exactly what they are rating, i.e, does the rating match the description, not rating a woodland cottage against The Ritz. Also guest expectation should be in line with what they pay. Hosts should be able to leave a fair review without being held to handsome and guests who break host rules should be held to account. Make this fair for all, please.

[Iris6]

from 5 stars to 4 stars? How come that all the past five star reviews are now four star reviews? I noticed this today and I am stumped. Even the guests who gave me a complete and full on 5 stars all over, today it seemed it was all down to four stars, what's going on?

[Susan453]

Location and value - I think if they ding you at all, even with a 4 star, you should ask the question.  I had a guest and the only 4 star ding was location??  I got the impression it was too far from THEIR event.  that is NOT my fault!!! It should be "Was the location correct as described in the listing??"  Its not about how far from THEIR event... so frustrating.  And value?  I had a guest from out of the country ding during HIGH season in AZ, value".. I was the same price as anyone else, much lower than hotels...  why did you book my place  if it was NOT a value?  Maybe something like "Was this a value when you booked?" I dont think that location and value should even be a rating. Hosts only have 3 catagories to rate the guest...

Guest book based on the "right" Location and price (value ) that is right for "them".  The host should not be penalized for their "mistakes" in picking location, or not liking to pay a high price when the choose to travel in high season...  its expensive in Tempe during March  - Baseball Spring training  - stadium are sold out!! 

Let the guest rate value and location, but NOT count them in our HOST rating... just use the data without "dinging" the host.. 

[David-And-Frances0]

Determine a cut off date and remove older reviews from the averaging process.

Do not include as 4 stars when guests give 5 stars on everything and 4 stars on overall impression. (Guests merely copy their notions of commercial review standards which are irrelevant to the spirit of airbnb).

Listen very seriously to the excellent suggestions and concerns. Airbnb is growing a culture and revenge reviews are completely of karmic concern. Creating a climate where guests can tyrranize!? Not a pretty picture, Airbnb.

Reviews for non stays? I had heard the opposite: that a review will be removed if the guest does not stay the night. It happened to us once. We were told it would be removed. Shall check. 

Location: should be up to the guest to determine that ahead of time? Please, Airbnb, do not foster guests who do not do their due diligence.

Thank you.

[Lee150]

I wrote my first negative review of a guest. They stayed 5 minutes. She never told me when she was coming ( after I sent TWO EMAILS )  and I thought maybe she was a no show. Anyway its 9 Pm and I am sitting in my living room in a towel and the doorbell rings. Its her ..with her kid ...No hello just "is this where I am supposed to be" ..I showed her her accomadations and she looked down her nose at me and she " I thought there was a tv room NO PICTURES OF TV ROOM  ) I held my tongue ..because I could already tell she was going to be a pain in the ass said "you will probably be happier somewhere else" and called AirBnB

I have to say whoever I got was lovely and was FLOORED because they actually said I hope you continue to share your home "
AND the next day I received an email that EVEN though I cancelled her reservation I could review her

And I DID !!!

[Anthi-and-Yiannis0]

@Airbnb I believe that is very nice improvement. I believe that you work really hard to avoid the wrong review.

One more topic that needs to be looked at, is the translation of the review system. Here in Greece for instance, we are rated about the accuracy. That means if the description of the listing is accurate. In greek language, accuracy is translated to ακρίβεια by your review system steps, which is also the same word for "being expensive" and the greek travelers always press that everything is very expensive, since we are on an economical crisis!! The correct word should be "Σωστή περιγραφή" which means "accurate description".

So please check the statistics of greek travelers and you will realize what needs to be done.

[Marlon35]

Location, location, location.

Really?

The person(s) behind Airbnb star rating system about location by guests needs to recall it totally.

The persons(s) have a profound misunderstanding about location.

It is faulty from inception as the damaging records show.

This is better left to be publicly explained in writing by each guest, it has to continue to be their choice, in their individual reviews but without stars. 

The person(s) behind this rating system  may even think of a system asking guest to star rate the racial composition of the neighborhood and it is bad.

How beatiful is the monalisa?

How well located is NYC, San Francisco, Rome, Paris, Egypt, Machu Picchu, Barcelona, Mexico, Mount Vesuvius, London, Dubai, The Everest and I am sure the person(s) have definitely more locations to add here for sound understanding.

Likewise anything that star rates referring to access as some can only afford walking when there are bikes, horses, by air etc

Please undo it, leave it to the experts: our guests written explanations about hosts locations.

Airbnb Guest's star rating system, thank you for your very good intentions but anything that allows anyone to misuse intentionally or not cannot be put in place.

; )

[Samantha372]

Hi, Thank you for initiating this important discussion. I have 7 months experienc. So far I have not faced that serious and hard guest. But as a guest I also feel all these concerns. I have only one 4 start for location from one guest. That also was due to sudden 1 hour powwer cut in Sri Lanka which was beyond our control. If I do not get that all my reviews for all aspects are 5 start from all other guests so far visited. So as a private note even I requested please explain why you rank as 4. But I did not get any response for that. As many of our guests elaborated here some times we feel what will happen to review. Another side of the story is with this system most of the good guests are empowered. In this regard this is very tricky. The system enable both guest and the host alert and accountable. This will happen when both guest and host is honestly following the rules (game of the business).  I feel that host is more vulnerable at this situation. So we need to balance it and resolve.I am sure that Airbnb will be able to innovate an adjustment to resolve this kind of situations by taking care and consider the both views and opinions of both sides.

[Keanda0]

I had a terrible guest who didn’t want to stay in my apartment because she realized how far away she was staying from her friends. She wanted to get her money back so she threw anything she could at me. She said it was unclean and said her body itched from mosquitoes, saying I should have mosquito nets on windows. I didn’t advertise my apartment as having them. I was actually releíved when she said she was going. It had been an herrendous 24 hours! She threatened me that if she didn’t get her full refund, she would give me a bad review which she gave me anyway. I have had excellent reviews for cleanliness etc before and after from different agencies. I am annoyed that her review still stands when she was clearly a bad guest

[Aj48]

haha . I have just had a negative review about items I do not even provide.  Like how can you mark your whole stay to a negative expeirence, when my listing does not even mention coffee maker. Complete shambles Air bnb review system. 

[Marg11]

We have a framed 'joke' on the TV cabinet we stole from a backpackers hostel in Amsterdam listing many things we do not supply. We used to have it at the top of our introduction letter until we added more instructions filling both sides of the page. Hopefully, the reality of $50 a night for two is obvious.

"Sorry, there’s no valet service, BMW for your own use, chauffeuring, 6 Star rating, pool, spa, massages, sauna, acreage, donkey-horse or camel riding, security guards, gourmet meals, daily cleaning, butler, maid, entertainment, dishwasher or dryer."