Can we collective agree on a star rating guide?

[Cody235]

There are posters available on Etsy, Amazon, etc. to help with the rare terrible review that is unjustified. Can we collective agree on a star rating guide and post it as an offical airbnb policy/document?

The other day the review was basically along the lines of "everything was great and we enjoyed the stay!" = 4 stars. How is that not 5 stars?

I have printed/laminated the below in effort to have a more standardized rating system. I feel weird about using my own, but it is definitely more honest than some I've seen for purchase online and at some airbnb's I've visited.  I don't think it is fair to suggest that a 4.6 rating is considered negative by airbnb when I don't see any such policy on airbnb. Anyways, I see this as an opportunity to standardize a critical process and minimize unmet expectations and miscommunications.

rating guide
this is not a Hotel or Restaurant
our goal is to offer our Home as an Enjoyable and Economical stop for visitors
★★★★★	my experience matched Listing description and Host messages
★★★★	there were minor mismatched expectations resulting from Listing or Host communication
★★★	there were major mismatched expectations resulting from Listing or Host communication
★★	I am contacting Airbnb support because…
★	Description and Communication was completely false and unjustified

[Helen3]

Airbnb is also not a boat, a plane or an amusement park - it seems rather arbitary and odd to include these comparators about what an STR is not.

They have already published their explanation on their ratings.

@Cody235 

[Cody235]

@Helen3  Really!? Where are the have They have already published their explanation on their ratings? I have looked and have not found it. 

[Herbert39]

Cody,

Good suggestion. Having a guide to the rating system that is presented to guests prior to their actual evaluation could be a great service to everyone, guests included. 

Nearly every rating system where the customer is asked to rate a service tells you what the rating scale means. I see these evaluations and ratings all the time. Leaving a scale to be totally subjective to the guest is a disservice to the hosts and other potential guests. 

Although I have had 5-star ratings in all my bookings, I can see how the system leaves gaps that should be improved. If the "C" level executives don't have an appreciation for statistics as it pertains to the behavioral sciences (for example how biases influence our perceptions) we are in a situation where much improvement needs to be escalated and discussed.

Here is one example. Imagine that you have two apartments and have had only two guests. One apartment had a 30-day booking with a 5-star rating. The other had a 3-day booking with a 4 star rating. Clearly a simple averaging of the 5-star and the 4-star would bias the data towards the negative... a 4.5 rating. In effect, the proper average should be 5*30 + 4*3 or a raw score of 162 or an average daily score of 4.9 - in other words a weighted average is closer to the truth. And it could be more nuanced. If there are 5 elements of the rating and only one of the five is a four-star then the rating is disproportionately skewed to the negative. The rating scale is woeful and deficient in many ways, it seems. But this is not rocket science. The answer to this problem is perfectly doable and simple. 

The additional problem is that an issue that is problematic to one guest, may not be a problem to another. For example, the use of smart thermostats. Some guests are simply befuddled by them. So, the one guest with the frustrating experience (and who said nothing during the explanation of the thermostat) might give a lower rating despite having a wonderful experience otherwise. 

The behavioral sciences are an essential background for this business, and it is a skill set that appears to be missing based on the policies that I see. 

Here is another: 
Airbnb pushes for the instant booking, but the host cannot set the preconditions on the number of positive reviews he requires for instant booking. A potentially problematic guest bent on having a party for lots of people, may set up one booking, prior to booking with his intended victim, for one day and get a positive review, then use that positive rating to validate his booking. A host, after-the-fact, may get the false impression that he's got a conscientious guest and won't decline the booking. 

Guests without reviews are a potential problem given some of the demographics that are relevant to guest potential behavior. Sociopaths, by some studies, comprise 4% of the American population. So, lying, cheating or leveraging concessions by threating a bad review are to be expected. 

Airbnb "C" level executives need behavioral scientists in their midst to help them guide future policies and ratings are an important aspect of this business. 

[Gillian166]

@Herbert39 many behavioral scientists (or now called "social scientists") are increasingly becoming Woke and therefore at odds with reality. I would hesitate to ask ABB to employ such people; they won't be helpful. 

[Herbert39 · Accepted solution]

Gillian, 

When you consider that there are perhaps tens of thousands of professionals in the behavioral sciences, is it reasonable to consider them as part of a specific cultural mindset? The study of biases and natural inclinations may serve Airbnb as a guide to develop better measurement tools. 
 On the topic of neuroscientists and psychologists and other academics being beholden to the "Woke" mindset, what evidence is there that can give you a different perspective?  If you are interested, the Heterodox Academy initiated by psychologist Jonathan Haidt may persuade you to evaluate your position with the nuances that such a delicate and controversial topic requires. 

As for Airbnb executives and the rating system that evaluates guests and hosts, they could benefit from a nuanced discussion of the weaknesses and opportunities of validating such an essential tool to the platform and to our industry. It is worthwhile to push the topic to a prominent place the next time we get to meet the founders of the platform or any top-level manager. 

Let's push for a genuine effort to improve the dismal rating system we now have to adapt to. 

[Gillian166]

@Herbert39 I am aware, and a fan of, Jonathan Haidt. He has been pointing out the changing face of western society for quite a while now.  The Weinsteins (Brett and Heather) have much to say on this topic and they lament the downfall of modern education systems to woke dogma. Just because it's not fully happened yet doesn't mean the decay hasn't begun, it has, and many are pointing it out but it's being denied as truth. Even law is filled with it now, and I recently watched a discussion about how the Medical faculties are now putting DEI concerns first. It's easy to dismiss this idea now when it's not endemic. 

[Cody235]

@Herbert39  thank you for the well thought-out response, it felt welcoming! That is a lot for me to think about and I’d say I generally follow and agree. 

[Inna22]

@Cody235 

I love your guide and your wording. I do agree though that your fist comment should not be "I am not...". You are not, true, however addressing gusts with a negative is just not a good idea particularly since you are trying to get a raving review. You can get your message across in a different way. Something like "Since Airbnb rating system is different from hotel, here is a guide other guests found helpful" or "universal guide to Airbnb rating" or at the end- please keep in mind that we are different from... or whatever other positive language you can think of

There are in fact plenty of discussions here on the topic. Perhaps they are not coming up because the site has been revamped. People posted guides like these. Having said that, I do not see anything wrong with talking about it again. There has been sell activity here all around so that might be a reason for lack of responses. 

And no, airbnb is not going to post this anywhere. You can spread the word but you will never make it an official guide. If you do, you are a determined genius and you should be doing something bigger and better with yourself.

[Cody235]

@Inna22 Thank you for the feedback/support. I think I will continue to use a personal rating guide and let it evolve over time (e.g. add your suggestion)! I appreciate your concluding note, however, I am probably not that genius! I do hope the conversation stays alive in the community, as I genuinely do believe there is potential for great improvements through this topic. Even if most people say it won't happen. It wont happen... until it does! That is the best attitude I see. Let continue to advocate and refine the process/rating-system.

[Gillian166]

@Cody235  i've seen a few such tables, even printed out one for my guest manual (which i recently removed as it felt pushy) but I really like what you've done here. I think I could format this nicely, with the ABB logo and make it look official, and it's a non-aggressive way of letting guests know that a 4*  review is not actually "good".  Funny how if I drop below 4.8 I lose superhost, but a guest at 4.5 is perfectly fine??? (and a guest at 4.5 is likely a guest at 4.1)

[Huma0]

@Gillian166 

I am not sure guests are even displaying at 4.5* anymore, but automatically bumped up to 5.0*.

I had a recent guest who had only one review and 5.0* in each category. I scored this guest 4* in every category (she was a nice girl but there were numerous small problems), so that should mean that she would now have a 4.5* rating. Nope, it still shows as 5* in every category, so future hosts will not see that one out of two hosts marked her down.

[Gillian166]

@Huma0  yes, this is 100% what happens. That's why I now score guests 4* if they didn't bother to communicate, even if they were fine guests. No communication means they didn't read the house rules.  

and then there's hosts who do this: 

He must have marked her down but why write this? 
this woman actually did a 3rd party booking with us 😠 very abrupt communication and of course, totally lied to me. Instead of a couple I had 2 tradies stay over, and they smoked.... but on the plus side they went out for dinner and left at 6am, and you gotta laugh.... they peed in the loo but didn't flush, and didn't use any toilet paper 😅  I'm thankful they used the loos in the pub!!    I will do 5* for cleanliness but mark her down for rules and comms, and see where it lands after that. I'm not doing the review until day 14, I see she isn't shy about whinging in her reviews. (we actually did a full video walkthrough before "she" arrived)

[Huma0]

@Gillian166 

Now I am confused. Why is your guest showing some 4.5* ratings and mine is not? Does that mean that a guest who gets 4.5* shows as 5.0*, whereas a guest who scores between 4.1 and 4.4* gets bumped up to 4.5*?

And what does that mean about a guest who averages 4.0*?

As for why the previous host would do that, well, I guess some hosts find it really difficult to say anything negative in the written review and easier to leave lower star ratings and and, now, those individual comments that aren't public on the profile.

I am wary if I ever see ratings lower than 5* because I know that A.) a lot of hosts are too generous in their reviews and B.) think that anything below 5* is bad, because that's how it is for hosts and C.) Airbnb bumps up the guests' star ratings. So, I will always do some investigation in that case to try and figure out what the problem really was.

Also, you mention that the booking guest was not shy about whinging in her reviews, and that is a red flag for me. Sometimes I forget, but generally I try to check the reviews the guest has left for previous hosts before I accept a booking.

I think you are generous to leave 5* for cleanliness, but if the smoking and other stuff didn't result in extra cleaning, maybe that's fair? I would however, be very annoyed about a third party booking and would mention that in the review as a warning to future hosts.

[Gillian166]

@Huma0  yes, I believe that a guest who shows up at 4.5 is actually well under that. even if they were 4.500001 I reckon ABB would bump them up to 5*. 

We were pleasantly surprised at how clean the place was, they only used 2 coffee cups, they didn't use anything else and even re-made the beds. Pretty considerate for what you might think of two male tradies....! I only got lucky and saw one of them smoking out the front via the security cam, and one pillow case was a bit smelly. Otherwise it looked like no one had been in there. 

[Huma0]

@Gillian166 

Ah, in that case 5* for cleanliness seems fair, and maybe mark down for house rules for smoking/third party booking etc.