Statistical Method for Calculating Super Host May Be Fatally Flawed

Status changed to: New

I’m new to AirBnb and have just had my 10th guest and was hoping to acquire Super Host status. My access to statistical information is limited to my own limited experience but, at AirBnB, it would be nice if you employed a statistician to review both the data AND the model used in this regard. On 1 April you claim to have changed the criteria for awarding super hosting status from requiring 80% 5 Star rating to now requiring 4.8 Overall Rating. Statistically, and in all probability, there is NO DIFFERENCE BETWEEN THE TWO METHODS – so nothing has changed. To illustrate on my small sample of 10 reviews (naturally AirBnB would need to review a wider statistical sample). I currently sit with an OVERALL RATING of 4.7. It also happens to be that I have 70% FIVE STAR rating. When you analyse my data, it so happens that the exact same 3 out of 10 reviews that marked me down and gave me a 4 score on one or more of the other criteria such as: accuracy (4.9), cleanliness (4.9), communication (5), check in (5), location (4.7), value (5) – are the exact same three who gave me a 4 for overall experience. It is highly unlikely that a person who has some gripe on the “detailed” review will award a 5 star overall experience. BUT, if you aggregate my scores, then you get a totally different picture. My average score on the detailed review is 4.91 (excluding the “overall rating”). My average score on the detailed review including the overall rating is 4.88. Thus, statistically speaking, the system places far too much emphasis on an “overall experience” metric when what should be included is ALL the metrics in the assessment. Including ALL METRICS gives a far more accurate and balanced indication of what exists, rather than the single inaccurate “overall experience” metric. It is fairer to the host, but it is also a more accurate reflection to the guests because the double weighted bias in the system will have been removed. It would be really nice is this was attended to.

cancel
Showing results for 
Search instead for 
Did you mean: