The General Confusion About Star Ratings

[Huma0]

I've seen a lot of posts on this forum where hosts are upset about receiving a four star overall review when the guest has given them five stars in every category. Those reviews/ratings look something like this:

There is no explanation given in the feedback or review as to what they were unhappy with and they have marked each category with five stars, so what on earth was the problem?  The problem is that they have NOT given five stars in every category. When a guest reviews, it is totally optional to fill in the star categories. They can rate some, all or none of them. Many guests choose to rate none and just give an overall rating (as in the example above). When this happens, all the categories will show up as five stars under that review in your Progress tab, but they do not count as five star ratings, i.e. your percentages in each category will neither go up nor down.

How do I know this? Because I keep track of my overall and individual star ratings (you can see these percentages under each listing in your Progress tab), so I can see each time a guest reviews which categories, if any, they have rated. Here's an example.

This is not fail safe as how much the percentages go up or down with each rating depends on how many reviews you have for that listing. If you only have ten, each rating can be worth 10%, whereas if you have 100, it would only be worth 1%. If you have over 100 reviews, you might not always see a change in the stats with each guest that rates. Still, you can get an idea overall if guests are rating the individual categories or not.

Of course, I shouldn't need to keep a log of these percentages. The stars under each review should only show what the guest has actually given, not show five stars for a rating that doesn't even exist. It is obviously causing confusion amongst hosts and Airbnb needs to change this, but I thought I would shed some light on it, so that next time you see that four star rating with five stars in each category, it is not so confusing.

[Huma0]

And what happens when a guest only rates some categories, but not all? The guest below gave me five stars overall, but only four for accuracy and location. 

It looks like she gave me five stars for check in, cleanliness, communication and value, but actually she didn't. She only rated the first two categories and not the others. They then show up as five stars each when they are not. My stats only changed for the first two categories and the rest stayed the same. Note the order here. Location usually appears lower down, but because she has only rated Accuracy and Location, they now both appear at the top above the categories she has not rated.

This is another issue, albeit a different one, that needs to be addressed. I am finding that many guests will only rate the individual categories they are not 100% happy with, but not the ones they are 100% happy with. The problem is, if a large number of guests do this, your individual category ratings (and the stars that appear on your listing) will go down over time.

I don't know if this happens because guests assume if they leave five stars overall, the categories they don't rate will automatically be counted as five stars also and so they think they only need to fill in the ones they felt were, for example, four stars. They don't realise by doing this, they are collectively bringing your star ratings down.

I fear there is something in the review process that is encouraging this misconception and prompting guests to do this, i.e. asking them "What could be improved?". It is my belief that if a guest chooses to rate the individual catetgories, then they should be required to fill in ALL or NONE of them. That would immediately solve this problem in a super simple way.

[Shaun69]

@Huma0 Thank you Huma that was an interesting post which has clarified a few issues.

Regards Shaun.

[Lilly28]

@Huma0

Thank you!!! Huma, thank you - I did not know this. It is very important point! Thanks again!

[Huma0]

UPDATE to this topic.

@Emilia42  brought to my attention just yesterday that the star ratings for individual categories are no longer optional. So, if that is the case, that means the information above is probably no longer relevant. If all guests must rate those categories, then the ratings the host is seeing must be the ones actually given.

I think @Emilia42 is a reliable source of info, but it would be great to hear from others that have recently reviewed as guests if they also found this to be the case.

[Lilly28]

Huma,

It was the first time I have seen this - but I still cannot understand it, in my case, i.e. it is showing that I was given 5 by the last 5 guests for overall - and value, but 91 guests that gave me 5 for check-in and accuracy and 7 last guests for cleanliness. So does that mean that all the others did not give me any star-rating for overall - but only the last 5? Or should I check myself each time a new review is posted - how this percentage has changed? I will be glad if you could answer me, thank you for your attention!

[Huma0]

@Lilly28

I'm afraid you have to check it after each review is posted and after you have waited for the stats to update, which can take a few minutes or a few hours. You will know when they have updated because the overall number of reviews has gone up. If you leave two reviews at the same time before waiting for this, you won't necessarily be able to tell.

I actually keep a log of it in a Word document, which I update each time. It is a complete pain, but it's the only way I have figured out to accurately see what individual ratings the guests has left (or not).

So, when it says you wre given 5 by the last 5, that's not the last 5 guests. That's the last 5 who left a rating in that category. In the image I posted above, it says, "Your last 5 Communication ratings were each 5 stars", but that means the last 5 guest who rated that particular category. I know for a fact that not all of the last five guests who stayed in that room rated Communication.

I hope that makes sense! I know it is all very confusing when it doesn't need to be. Airbnb should not mark non-rated categories with five stars under a review, only mark the stars that guests have actually left. That would make it all really straightforward.

[Huma0]

Oh and @Lilly28 when it says "the last 5 guests left 5 stars", that means the last five who rated it, so probably the one who rated before them left less than 5. That doesn't mean only five guests rated you on that category in total or that only five left 5 stars.

I would ignore those little Airbnb comments and focus just on the stats. The comments are just showing you how many 5 star ratings you got in a row for a category. They don't indicate how many you got altogher, but the percentages in that category do.

For example, because the guest above gave me 4 stars for Accuracy and Location, there is no little message above the stats, because she interuppted my little five star run in that category! The next time a guest rates one of those categories as 5 stars, it will say "Your last rating was 5 stars". Once there are a few more, it will say "Great work! Your last 8 ratings were each 5 stars", and so on. I suppose it's designed to encourage us or something?!

[Ange2]

"There is no explanation given in the feedback or review as to what they were unhappy with and they have marked each category with five stars, so what on earth was the problem?"

@Huma0    It is completely frustrating , but 4 stars does not mean the guests are unhappy, it means Airbnb is unhappy and that unhappiness is passed onto the host.  The problem is that Airbnb tells the guests that the stars mean one thing and the host experiences it meaning something completely different with penalties : i.e. less than 5 stars is a fail.

Not sure how Airbnb explain  'Overall',  but my guess is it's worded in a nebulous fashion and open to myriad interpretations such as overall vacation: less than 5 stars can mean it rained, they got lost on the metro, they had an argument,  they didn't like the color of the sheets even though extremely comfortable. They expected more for less!  Or in a response I got "ny is nothing like the "gossip girls"" ding!, ding!.  Or an all too frequent rejoinder—everything was fabulous, amazing, wonderful, but nothing is perfect!

As for "What could be improved?"  When Airbnb is pressuring the guest to drill down to details  via the lengthy review survey it's not hard to find one thing that was never noticed until Airbnb got out the microscope and focused it on minutiae.

"What could be improved and how much more are you wiling to pay for it"  would be a far better question.

My guess is Airbnb is collecting data for creating alogrithms and analytics (a gold mine) thus they will never streamline the  survey length review or make the rating sytem simple because doing so means less data to amass, play around with, manufacture perception and expand their business.

I have given up on Airbnb doing anything for hosts past offering the site as a fee based  booking platform.

I do my best for my guests and make improvements when feasible and that is as far as I am willing to go these days. I am not a mindreader, I can't subsidize guest's vacations, and there is no accounting for people even though technology has made a fortune trying to convince us otherwise when it's really just all about selling.

The review and star system is not designed per se to help hosts it is designed to help Airbnb.

[Anna14038]

Absolutely and 100% agree. Wise words. Airbnb is there for itself. Case closed.

[Huma0]

@Ange2

I am afraid you are right. I also can't see much reason for Airbnb making the whole review/rating process so complicated for guests, other than to collect data. However, they are slightly shooting themselves in the foot as I think less and less guests are now inclined to fill in that review.

The whole telling guests one thing, e.g. "4 stars = good" and hosts another, e.g. "if you keep getting 4 stars we might delist you" is a nonsense, but not really the topic of this thread. I could go on about that forever because it is so utterly stupid and causes so much stress for hosts.

I have thought about printing out a document for guests that explains the whole review process and leaving it in their rooms, but it seems a bit heavy handed. For now, my reviews and ratings are mostly great and I am managing to keep Superhost status, but that doesn't mean I think the rating system is fair at all.

[Lilly28]

@Huma0

Do you know in this stat and percentege given - are all guests who left a review counted or only the ones from the last year? Thank you, if you could answer that question, Huma. And I am just interested to know your opinion on my suggestions as Superhosts to be devided by categories, thus you are not going to be afraid to loose the Superhost level - only you will pass from Gold to Silver Superhost, or from Diamond to Gold, or from Silver to just Good host. And I think that price should be more or less connected with the level attained. Now with this system of either yes or no - pass or fail - so often we have to make discounts just to stay and keep this YES.

[Huma0]

@Lilly28I'm not 100% sure as I haven't seen this confirmed by Airbnb anywhere, but I believe it is for all reviews, not just from the last year.

My overall stats are currently showing 198 reviews. My current Superhost assessment stats, which count for one year, including the current quarter, show 152 stays, but only a 69% review rate, which means around 105 guests reviewed me in the past nine and a half months. That is just over half of the total reviews, which makes sense as I've been hosting for roughly two years.

[Huma0]

@Lilly28As to your suggestion RE categories of Superhost, I think it is an idea worth exploring. However, I am not sure how fully guests would comprehend this. Until fairly recenty, I found that the vast majority of my guests had no idea what Superhost even was, let alone that there was a filter they could use in the search.

Since Airbnb started highlighting that people were Superhosts on their listings last Spring, I found that now many guests do seem aware that there is such a thing, at least they often mention to me that they know I am a Superhost. Well, that's a step in the right direction but, although it would seem logical from a host's persepective, I'm not sure how many guests would get their heads around all those different categories and understand the difference between them. Already most guests take in only the most basic of listing info and it's hard enough sometimes to get them to acknowledge the most important house rules.

I do agree though that there needs to be some major changes to the Superhost criteria because at the moment, one very low (perhaps retalitory) review can totally blow a host's chances of retaining the status even when their other ratings are all excellent. I am lucky right now because I have around 200 reviews, most of them 5 star, so I'm less vulnerable, but earlier on, when I had less reviews, one or two four star ones could ruin any chance of remaining a Superhost.

[Penny309]

I'm so stressed about my 4 star and being threatened to be removed from the website I injured my foot. only second guest.

also it cost me as much in cleaning as they paid.. now I'm totally stressed .. about the process... third guest in now.