The General Confusion About Star Ratings

[Huma0]

I've seen a lot of posts on this forum where hosts are upset about receiving a four star overall review when the guest has given them five stars in every category. Those reviews/ratings look something like this:

There is no explanation given in the feedback or review as to what they were unhappy with and they have marked each category with five stars, so what on earth was the problem?  The problem is that they have NOT given five stars in every category. When a guest reviews, it is totally optional to fill in the star categories. They can rate some, all or none of them. Many guests choose to rate none and just give an overall rating (as in the example above). When this happens, all the categories will show up as five stars under that review in your Progress tab, but they do not count as five star ratings, i.e. your percentages in each category will neither go up nor down.

How do I know this? Because I keep track of my overall and individual star ratings (you can see these percentages under each listing in your Progress tab), so I can see each time a guest reviews which categories, if any, they have rated. Here's an example.

This is not fail safe as how much the percentages go up or down with each rating depends on how many reviews you have for that listing. If you only have ten, each rating can be worth 10%, whereas if you have 100, it would only be worth 1%. If you have over 100 reviews, you might not always see a change in the stats with each guest that rates. Still, you can get an idea overall if guests are rating the individual categories or not.

Of course, I shouldn't need to keep a log of these percentages. The stars under each review should only show what the guest has actually given, not show five stars for a rating that doesn't even exist. It is obviously causing confusion amongst hosts and Airbnb needs to change this, but I thought I would shed some light on it, so that next time you see that four star rating with five stars in each category, it is not so confusing.

[Lilly28]

@Huma0

Your rooms are from the few I had a look at - I like them so much, and I like your bathroom, and like even love you! I think you did not understand all my points, and besides - I have to rethink how to address the matter. What I meant was that -  LOCATION and PRICE - should not be included into the valuation - as the guest has chosen them alone! In My system - as you point out - all your three rooms would have the same location - given from the sytem, not by the guests/ as for price it should be moved according the performance - and all your doings - as your painted chest!

[Huma0]

@Lilly28

Thanks for your lovely words. 

Yes, the finer details all need working out to make this review system work, but I find the new development @Robin4 mentions has only made matters worse.

At the moment, Location ratings are not counted towards the overall score. However, guests can still downrate a host overall because they didn't love the location. My 'best' room mentioned above, has good star ratings for all categories (between 86 and 97%) except for Location, but an overall score of only 84%, as compared to my other rooms which have 90 and 92%. Guests in the better room aren't unhappy with the room, they always say they love it, so what's the problem? I will bet my bottom dollar that guests in that room (with their high expectations) are marking the listing down overall because of Location. It's the only category that consistently gets low reviews for that room.

So, I agree, Location needs to be taken out of the equation altogether so that guests aren't encouraged to think about that during the rating process. However, I don't think Airbnb are going to do it. Many hosts have been calling for this for a long time and the response is always, "It's important for guests to get an idea of the locaiton, so we're keeping that category, just not counting it towards the overall star rating."

Likewise, taking Value out of the ratings categories makes sense. Like you've said, the guest has chosen both the location and to book the listing at the advertised price. They made that choice. However, if Airbnb is not even going to remove the Location category, you can be sure they're not removing Value.

My problem with your suggestion is more to do with pricing listings according to the levels they achieve. The price should be based mostly on what is being offered, not on how guests rate it. I only gave Location as an example to show how nonsensical guest ratings can be. I describe the location in the same way on all listings and the listings are all around the same price, so why do only the guests in one of the rooms have a problem with it? Because they are rating based on some unrealistic expectation they came with. Should my income be affected by this? No!

My point is, it doesn't matter whether it's the location category or any other. When you base prices on guests' opinions rather than what is actually on offer and its real value, hosts will end up being held to ransom, having to bend over backwards, allow guests to break rules, never claiming for guest damages etc. just to avoid ratings that can directly affect their income. That would be a scary situation.

[Lilly28]

@Huma0@Robin4

In some way there should be a holt - from the ever going down prices and guests who are allowed to value the price. I have read many reviews recently - in different cities - and I found a big difference with the reviews even a year ago. Guests want Ritz for the smallest price possible and if things are going to run that way - we all should close our airbnb rentals. And the sad thing is - that only new commeres will stay into their listings...

[Huma0]

@Lilly28 

Yes it's true. The Value rating is problematic. In my opinion (and I know many hosts agree), the guest should not be able to downrate a listing on Value as they chose to book it at that price and it is the guest's responsibility to do the research on prices in the area and on location. The only exception is when the listing is not as described, i.e. the location was misrepresented or what was promised at the listing was not actually what was offered, but those problems can be addressed with the Accuracy rating.

[Fred13]

(Back from Mexico looking at boats)

   The central problem with the Airbnb is that it has become too complicated, now full of band aids and now is too critical since so many of its parts are co-dependent on each other for it to work, as intended.

   The new rating 'categories' like Much, A bit, About the same, A bit worse, Much worse exacerbate the problem by making the whole process even more emotional and that much more co-depended on individual interpretation. The whole Superhost scene now has become even more  nervewracking for hosts..

   And @Ange2, you made a great point - 'Overall' may also include many things, like the experience of the  local taxi service from the airport, that has nothing to do with where one is staying. 

   The less one worries about all the whistle & bells of Airbnb, and just focuses on doing a good hosting job with one's guests that they send (being the best booking agency in the world today for most), the happier one will be. 

[Huma0]

@Fred13 

I want to be in Mexico looking at boats!

Yes, you have hit the nail on the head. The new rating categories are too 'emotional'.

It makes no sense to base ratings on guest expectations, which often have nothing to do with the listing description. Let's face it, we all know that many, many guests do NOT read that, so their expectations may have nothing to do with your accuracy nor your offering.

Recently a couple from Asia booked, it turned out, because they wanted to take wedding photos in my living room and garden. What? My living room is pretty, but it's just a living room. My garden is a typical city courtyard style about 4 metres by 6 metres and they were staying in late October when hardly anything is in flower and the weather is usually grim. What were they expecting? I fear it was something along the lines of Downton Abbey. Were they going to be disappointed? Of course!

And, @Fred13, like you and @Ange2 have said, they may be basing their overall rating on their overall trip, not just the listing. It's not like Airbnb are reminding them to stick to the listing in their valuations. I know that I get better ratings when the sun shines.

Some people travelling to London the first time expect it to be all cobbled streets, vicars on bicycles and populated only by the likes of Hugh Grant, not the multi-cultural, diverse and urban metropolis that it is. I have hosted guests that thought we all stopped at 4pm to have a cream tea (erm, no, that's just for tourists and cream teas are from Devon anyway) or that it would take 15 minutes to get to mine from the airport (it can take 15 minutes sometimes just to exit certain tube stations). They come having done zero research (including not reading the listing), but full of 'expectations'.

At the very least, these categories should be changed to:

Much better than described

A bit better than described

About the same described

A bit worse than described

Much worse than described

Take the 'I' out of the equation. That's at the very least, but given that having a 4 star rating is not considered good enough by Airbnb, how on earth can they justify encouraging guests to give that rating when the listing is better than described?

[Fred13]

Bright post @Huma0 and yes like you said - "It makes no sense to base ratings on guest expectations..."

    Sure, they come my place and get 'emotional' and leave the most positive reviews based on the romance of the moment, but that doesn't apply with 99% of other places who are not offering some off-the-wall listing.

   Talking about getting emotional  - I had a Swiss couple stay in June,  they have sold everything and quit their jobs and moving to Belize why I was looking for some big yatch for them. Crazy.

[Huma0]

@Fred13

Wow, that is emotional. Your Airbnb has contributed to changing that Swiss couple's life in a dramatic way.

I am sure I told you this before, but years ago, I spent six months travelling in Mexico and Guatemala. I really wanted to go to Belize, but my domineering boyfriend didn't, because he believed it to be dangerous and dodgy (in hindsight I realise that he suffered from some sort of paranoia, partly from smoking too much weed!). So, instead, we simply drove through Belize without stopping and my heart ached along the way. Later he admitted that it did look kind of wonderful and we should have stopped.

A note to you young ladies out there (I think I was around 25 at the time): do not ever let your man or anyone else stop you from following your heart's wishes (providing they're not the crazy, criminal kind of wishes).

So, @Fred13 I will have to rectify this at some point and make that trip to Belize. See, you've got me all emotional! 

🙂

[Fred13]

🙂 

[Jo13]

"Some people travelling to London the first time expect it to be all cobbled streets, vicars on bicycles and populated only by the likes of Hugh Grant, not the multi-cultural, diverse and urban metropolis that it is. I have hosted guests that thought we all stopped at 4pm to have a cream tea (erm, no, that's just for tourists and cream teas are from Devon anyway)"

@Huma0

What?! You mean I'm not going to bump into Lord Grantham about his business in London? 😉

[Huma0]

By the way, this post has gone massively off topic! It was supposed to be about a discrepency in the way guest ratings are displayed on the system, an ongoing technical error of sorts, but I guess I chose the wrong title!

Calling it 'The General Confusion About Star Ratings' has opened up the discussion to cover the many issues with the review/rating system in general. I don't mind as clearly this system is one of (if not the) biggest stresses for hosts right now.

I do wonder if, when Airbnb makes changes to its review processes etc., the different teams working on it (and I assume it is different teams as one team couldn't be so blind to its own inconsistencies) actually speak to each other?

Example:

Team A: Hosts should achieve a minimum 4.7 average to stay on the site and 4.8 to be Superhost.

Team B: Let's tell guests to rate 4.0 if they think the listing exceeded their expectations.

Brian Chesky: We want all hosts to eventually become Superhosts.

Example:

Team A: Let's add a criteria for Superhost that at least 50% of guests must leave a review.

Team B: Let's make the review process really long and complicated. Probably far less guests will fill it in, but at least we can collect some detailed data.

Brian Chesky: We want all hosts to eventually become Superhosts.

Spot the differerence? Team A is doing something counter to Team B and the end result is the opposite of what Mr. Chesky claims to be his goal. Genius. Or perhaps Mr.Chesky wasn't quite telling the truth?

[Lilly28]

@Huma0

I was laughing out loud when I read your last post!

No, Huma - I do not thing that the discussion now went into a wrong direction from your first suggestion. You just opened a new perspective at looking into reviews and evaluation host,s work.

Now I understood what exactly happened to my discontinued 5 ratings on overall and some other categories. I had a host as a guest - and she gave me 4 overall!! - I have already told the CC about this strange case, and as Letti answered me - this woman knew exactly what she was doing when giving me 4 after I worked 15 hours - for her only 2 days stay - as her private guide, she used my garage for free, had special offer, and more - she is a NEW host - giving her flat at twice and a half the price she paid me - and not doing washing, not staying at the property, wanting her guests to do the cleaning and washing for her next guests! She gave me 4 for price and 4 for location - and now I know that she just skipped to value the other categories.

That is just one of the reasons I would like every airbnb to have something as an anchor - as hotels have, they cannot by their will or just the reviews of guests become 5-stars - as Brian Chesky want us all to be Superhosts!

Another case - guests from my country badly damaged the furniture I made myself - and it is on the first page on my listing. This is 18 years old sofa, all wood and 50 by 50 by 70 sm latex seats for each of the two - 180 kilos guests have been sitting on it for almost 2 weeks 2 months before these gests - and there was not a sign of damage - and suddenly these young men... and they agreed they have done it - telling me that the furniture is not worth...  

I can point out yet another reasons - all my 4-stars ratings for accuracy had nothing to do with the description of my property! One guest gave me 4 stars because I did not agree to payment by hand, he was married with her lover - wanted their stay to stay unknown, three times I was given 4 because the airbnb team put my flat in a place - imagine a flat inside Regent park...  and once I was punished for my super accuracy as I collected guests with a taxi from a strange place where they began sending me pictures - they did not know English, neither French - so could not read my address... Lastly I had 1 star for accuracy because a guest burned my electric kettle and I reported him to the Resolution center - he denied the damage and angry - gave me 1 as a revenge.

Well, these are just some cases - I am sure you are having much more similar ones.

[Huma0]

@Lilly28

Yes, I for sure have had some similar experiences. I think after a time, most hosts will have them unfortunately. I check in on all my guests to make sure they are happy and ask if there is anything else they need, but most of them will say  "Everyting is great. Everything is perfect." You only find out in the review if that was true or not.

Even then, many will leave a 4 star rating and no explanation as to why. As @Ange2 says, that doesn't necessarily mean anything was wrong. They are being told 4 stars = good, so maybe they were happy with everything. When there is no indication in the review or feedback or individual star ratings, I take it simply that they thought it was good and hence the 4 stars.

When guests do give reasons in the review/feedback, they are often quite silly, in my opinion. Some examples of my 4 star reviews:

- Guest claims she was never informed of something, which is actually clearly stated in the listing and house rules and which she was also reminded of in two separate messages after booking. She never mentioned this during her stay, just said my place was wonderful.

- Mentioned my place is not in the city centre. Listing does not claim it is in the centre, but that there is convenient transport to the centre (with a breakdown of journey times). No complaints during stay.

- Said the listing was very detailed and accurate but gave me 4 stars for Accuracy. Said it was "sparkling clean" but gave me 4 for cleanliness etc. etc.

- Complained that they did not see me after check in. I was home all day and evening but they were out from 7am until late at night, which is why I didn't see them. They didn't respond to my messages asking if everything was okay with their stay.

- Said total price was more than original price advertised (i.e. clearly didn't expect Airbnb to add fees).

-  Didn't like tthat I didn't respond immediately to a message they sent me while I was in hospital, which is the only time I didn't immediately respond to their many messages.

- There were some renovations going on in the house. I had told the guest this in advance, but I think it is totally FAIR that she marked me down for this because she was not aware of it when she booked (I didn't know that many months in advance).

-  Said the room was cold although they had said it was warm when I checked them in and never mentioned being cold during their stay. I only realised after they left that they had turned the heaters off themselves!

- Said the location was 'lively' but not noisy and very convenient. These folks came from the woods with bears on their porch, so London was going to be a bit of a culture shock!

- Guests who totally broke my house rules (smoking weed in the house, having people over without permission) and then blatantly lied in their review to cover themselves.

- Very unfit guest who found the five minute walk to the tube station too 'strenuous' and had difficulties with the stairs (mentioned on the listing). Found my four storey house (2.5 times the size of the average UK home) too small. Not sure why she expected to stay in a mansion in London for £40 a night, but hey ho, she was from Texas where everything is supersized.

I am not saying I am not open to criticism. If there is a problem with the WiFi, then there is a problem and the guest has a right to mark me down. If the heating wasn't working, I hadn't cleaned properly, failed to mention something important, then fine, but I really think the only guest listed above who was fair was the girl who stayed during renovations. She had a great time here. In fact, in the week she stayed, she barely left the house and had her friend over most days, she liked it that much. Still, there were renovations going on so it was not as originally advertised = FAIR.

The rest I think are either a bit silly (it's not my fault if guests ignore my instructions and turn their heaters off and then fail to mention they are cold when I ask if everything is okay) or just blantantly untrue (saying the WiFi and fridge weren't working when they were, because they were expecting a bad review from me due to their drug use and partying).

This is why the new 'expectations' language in the review system is so stupid. Of course my location in a residential area with no bars or restaurants within sight or earshot of the house is going to seem too 'lively' to people who live in the woods with bears on their porch! 

[Huma0]

Sorry, that last post was a bit ranty and I honestly try not to get wound up about the occasional fours star review anymore. If there is room for improvement, I try to improve. If the guest is complaining about something silly (being cold because they turned off the heating after I asked them NOT to fiddle with the heating controls) then I'm not going to stress about it, just try to reiterate more to the next guests to please NOT turn off their heaters!

The problem is this new rating language of 'better than expected' etc. It's really encouraging more of these nonsensical ratings.

While they are few and far between for me now so don't really affect my overall scores too much, I'm expecting that now to change because now apparently my listing has to be SOOOO much better than described to get a 5 star. Maybe I should take off the nice photos and put some really shockingly bad ones on instead and then talk down my place in the listings, highlighting all the 'cons' before the 'pros'. I'm sure that would boost my bookings no end...

[Petrina3]

so the wifi was not working - i gave them my phone number to ring if a problem - they didnt call. he is an orthopaedic surgeon - they paid $ 346 for for 2 nights in a studio by the beach with access to 3 pools  -1 heated) a gym, beachfront restaurant, tennis courts. a continental breakfast was provided and they gave 4 stars for value. WTF?