Rating System - suggested improvements

[John784]

Agreed: Airbnb's rating system kinda sucks. Bad.

As a HOST: Every time I sign onto my dashboard I'm greeted with the warning that "You’re at risk - Your account could be suspended if you don’t focus on improving to meet Airbnb hosting standards."

I know of no other realWorld system in which a score of 4.5-out-of-5.0 (i.e. "90% positive") is the cutoff point for threatening someone with suspension. Do you?

Clearly:

* airbnb's rating system is a mystery to all of us
* airbnb claims that they have "hosting standards", but nowhere are they defined in terms of the rating system on which they're supposedly based, so ...
* those who do the rating don't know to determine the value or implications of airbnb's 1-2-3-4-5 rating

Most of us in the realWorld reasonably assume that a 4-star rating is 'pretty darned good'. Only in airbnb's fantasyWorld is a 4-star rating the basis for suspension.

Okey dokey. Let's make the rating system simple:

1. Unacceptible
2. Needs improvement
3. What I expected
4. Nice
5. Excellent

The baseline (3) is "What I expected"

Below that (1, 2) is the basis for finger-wagging

Above that (4, 5) is the basis for reward

Transparency : Label-the-Table

The ratings have meaning

and implications

and they're unambiguous

to everyone

Now you've actually crowdsourced some Wisdom

rather than just collecting numbers

Case Studies, etc
the UX Craft
the Ideas Blog

[Susie5]

@John784, totally agree! Your reviews are great (no stars visible).  I have encountered a few guests for whom 4 is an excellent review and they have no idea that Air expects 5 as a matter of course. Kind of like my students who "expect" an A ("but I worked so hard!")....  There was an article in Forbes about this very thing, if you google "The One Issue With Airbnb Reviews That Causes Hosts To Burnout"  you'll find it.

Some hosts print out this article and leave it prominently in a guestbook etc. so that guests take the hint!!

Good luck - don't give up!!

[John784]

Bingo with the Forbes article about ratings, @Susie5.  One wonders why airbnb hasn't posted this one themselves in the course of their relentless, shallow self-marketing to us (the folks who make them rich).

Participation Trophies for everyone!!!

BTW

I just graduated from a 'Level 1" to a "Level 2".  Why = A Mystery to Me

Then again, do I care?

[Susie5]

@John784, the level just refers to the number of posts on this forum - no measure of status, just level of activity. Seems to cap out at 10.  Glad you found the article useful! I certainly did - "we're not alone" in deploring this rating system.

[John784]

More 'airbnb funny-numberz' ...

Even more useful:  If airbnb actually addressed the issue

No offense to them or you @Susie, but I'm not particularly concerned about "being alone" in my insights.  Professional usability practitioner for 3+ decades.  Being 'that annoying guy ... who's right' was a career.  It must seem selfish, but I'd really just rather have airbnb a) get their thumb out and b) make my life easier.

[Lawrene0]

I like your Unacceptable, Needs Improvement, What I Expected, Nice, and Excellent. It is clearer and more precise than the stars (that get confused with Michelin stars). Will you take this to Host Voice, @John784? It is in the dropdown under Discuss on this page. 

[John784]

Thanks for the pointer to Host Voice, @Lawrene0

The solution is now shared.

[Lawrene0]

 I've hit the thumbs up on it over there, @John784. Thanks for the update.

[Christine615]

I like the rating system suggested here too. Because honestly - I 've seen some hosts who are offering a spare room with a futon. So to some people that might not be worth a 5. But if it's what you offered, why would Airbnb let guests rate it lower and then let the rating count it against you?  If it was what you expected then a 3 - it if was better than you expected than a 5. A 4.5 cutoff is not fair to some hosts.

I have a premium space so mostly 5's but some people write rave reviews and give me 4's. It's not logical. I guess I'd be a 5 if I were a mansion guilded in gold? 

That question came up in the online forum. I'm not satisfied that the CEO answered the woman's question (which was upvoted by thousands of us) about apartments getting lower ratings when the photos and descriptions are accurate.

I think Airbnb is still going through growing pains.

[Simon237]

If I was in charge of the ratings system, this is what I would do

1. Get rid of location. It is meaningless. The location is disclosed before the guest arrives, and is not in the hosts control.
2. For the five other criteria, change it from a 0 - 5 star system, to a Yes / No response. I have had many reviews where the guests are very happy, but still only give a four star review for no apparent reason.
3. The hosts score is a percentage of Yes they receive against the total number of reviews.

This seems much fairer to me, and easier for the guests to do a review.

[Barbara123]

Agree - 'Location' is silly and inapropriate.     One of my guests docked me on 'location', and I'm thinking what more do you want?  Guests are given a map, a physical description of location, and can read other guest comments.  How does location become the hosts issue????      If they wanted to be near the beach and the description clearly states that it's an inland property - what the heck?

The other rating that is wholly inapropriate is 'Value'.   How the heck does a first time guest have any perception of what else is available, any comparative measure for price/value?  

[Marko17]

Obviously it is not so black & white in this matter, just to say Yes or No. This rating system also pushes hosts to be more competitive and be better hosts then others. Obviously the one who give more effort to his listings, communication etc deserves a better rating. 
Lots of people equals to lots of different perspectives (also in the rating system). We can not put them all in one basket. Most guest on AirBnB will rate 5 stars if the listing description/photos/communication reflects real situation. From the mass of those people sure you'll get some "special" guests who sees things differently. But we must swallow that and move on. If you are a good host, you will be rewarded with good reviews. If still not, then open your eyes and try to find out where you can improve.

[Edward19]

I just encountered my first rating dichotomy, where I received 5 stars across the board but got a 4-star Overall. When I asked Airbnb about this, they said that the Overall was also a category. By that logic, someone could give 5 stars in all categories and a 1 star Overall, and THAT'S the star rating that the public sees. This discrepency was verified by the Airbnb supervisor I was speaking with.

That's outrageous.

The Overall rating should be the average of all of the categories. Additionally, I do like your proposal for Unacceptable through Excellent.

[Frank73]

I was having the same issues so I wrote a note to guests explaining but find most guests don't read so now put in the main description IMPORTANT PLEASE READ: Guest Resources/House Manual... especially because I'm in the BUDGET "one star" Motel category, so I have 14 separate listings!   I get mixed reviews and one was paused, so in that section, I have added a Dear Guest note but I find it too long but if you say too little guests may still NOT understand, if anybody can make this simpler, I truly would appreciate it but I feel it has helped more to receive OVERALL 5 stars from guests...:

Dear Guest/s,
PLEASE READ THIS, it’s important for our guests to understand Airbnb’s rating system being it won’t matter what we do or offer because it’s your 5 star “OVERALL” score that will determine if we continue with Airbnb...

Airbnb is not like other platforms, that whatever scores guests give, that accommodation will “continue” to be on their platforms, that’s NOT the case with Airbnb, WE will be DROPPED if going lower than Airbnb’s “overall” 4.7 average score... I repeat, even leaving a HIGH 4 star “overall” will lower the average “overall” 5 score and we’ll be DROPPED, if not maintaining a 4.7 score. So GOOD “budget” accommodations rating HIGH “overall” scores of 4-4.7 are DROPPED!

Can you see, how this presents a challenge for “BUDGET” accommodations to remain on Airbnb and how it's up to you, how that selection will become very limited? “Budget” accommodations, like ours are NOT constructed or will never meet the higher quality standards of other better “chain” budget accommodations that you’re familiar with or located in better areas of Costa Rica. This causes us to be easily criticized and unfairly scored and because ALL Airbnb’s are different from each other, it’s not even fair to compare them with each other. It’s why I stress that the only fair thing to do is to READ the details of description and if it’s “AS DESCRIBED” including trivial issues encountered, which could happen in the BEST of Hotels but you still feel they shouldn’t be dropped, please ALWAYS leave an “OVERALL” 5 star. This doesn’t mean we want to ignore the issues; it’s just NOT meant to be factored in the “OVERALL” score. They’re meant to be left in the 6-feedback score survey where you can leave low stars, and where we welcome your sincere comments so we can do better, that survey won’t affect the “overall” score...

So please, keep in mind, if we took the time to write the details, we appreciate you take the time to read them as to not unfairly expect more from us than what we can give, it’s not our intention to disappoint you. Though we’re “budget” accommodations, it’s still our hope to give quality hospitality care and strive for your 100% satisfaction. So of course, if you are disappointed or dissatisfied with any aspect of your stay, please feel free to communicate this to me while you're “there” to give us an opportunity to address the issue as the best solution and ensure a 5-Star experience which is all our main goal at Mapi's! As an out of the country host, we do everything we can to have our staff create a positive experience that meets expectations based on "as described": “one-star category” BASIC simple accommodations.

We, our staff and even this small community in this remote area are affected by your OVERALL 5-star score and remain grateful to YOU, if we’re still here with Airbnb...

Thankyou,
Frank & Nilda Bienes (Owners)

[John784]

Bingo.  The Airbnb rating system and marketing is skewed and inappropriate to handle the real-life range of hosting environments - especially those of us at the 'budget' end of the scale. 

Don't you just love the luxury Sharper Image airline magazine they send us every month?  I haven't met a guest yet who's even glanced at it (and I ask).  It is Delightfully, Frothily Slick and Totally Irrelevant to this portion of the marketplace.

As is their Rating system. 

Tell me:  Have you ever encountered an environment where scoring +90% on your rating earns you a lecture and punishment?

Frank & Nilda:  You nailed it. 

Sad insight:  No, nobody ever reads the documentation.  I say that - honestly - as a guy who worked professionally as a Documentation Manager.  And also a Usability Design Professional.  Because ... the solution to customer (and host) satisfaction is in the effective design of your service platform.   If airbnb cared about issues like that, we might see something reflected in the design of the tools they provide us.

In any case, Airbnb doesn't really seem to be too interested in supporting the non-luxury arena.  Have any of you ever run the numbers on the budget end of your market segment?  Are we really that worthless?

Here's hoping that airbnb marketeers will address the very real systemic support needs of those few of us who provide them income at the 'lower-income' end of the hosting spectrum.

Or maybe someone else will...