Search "Instant Booking" + "Host recommendation required" related bug

[Michael5689]

I sometimes do searches in a separate browser to check where my listing is showing up in the search results.  I'm not sure when this behavior changed but it looks to me like a bug has been introduced when you search with the "Instant Book" filter toggle button on.  I've been seeing some complaints from user about not being to find their home ever in the "All" category so I was doing some digging and this is what I found.

The bug is basically that an "unknown user" on the website (not logged in) will not see a home show up when the "Instant Book" search criteria are toggled on in the Filters and IF the host has Instant Book turned on AND the toggle button for "Host recommendation required" is turned ON.  I call this a "bug", because I tested this and it isn't affected by whether the "Government-issued ID required" or "Pre-booking message required" selections are toggled on.  This ONLY occurs if the "Host recommendation required" button is toggled on.  I know that as recently as March when I did these searches as an "unknown" and not logged in user that I could still see our property even with Instant Book turned on so it's a recent change in behavior.

I see no logical reason why this would be designed behavior.  If the intention was for unknown users to not the be able to see hosts that offer Instant book, then any of the instant book settings turning on would make it invisible to this search.  Also seems to really be against AirBNB's strategy to encourage as many hosts as possible to use Instant Book.

I saw this behavior on my desktop on  "Microsoft Edge Version 101.0.1210.47 (Official build) (64-bit)"  I don't think it's a browser related bug however as I also checked on my cell phone through Chrome where I had logged out of AirBNB and the same inability to find the home with "Instant Book" set in the criteria occurred if and ONLY IF I had the "Host recommendation required" toggle on.

@Pat271 I thought this one was an important enough bug that I'd report it through this forum anyway 🙂  AirBNB really needs to work on it's testing and not rush out a release just in time for the summer season with too many new features.

@Emilie I'm curious if there is a way to report bugs in a better defined intake form than this forum?  I've found a few others that don't rise to this level of importance and don't want to clutter the board with rather technical bug reports that might not add much value to most of the audience here.

[Kristina46]

Yup. This is major.

If you are not familiar with the geography/terrain of the Big Island, Holualoa and Kalaoa are "upcountry" areas that do have good views because of their locations, but they are literally miles and miles away from the beach.  And way up the slope so to speak at elevation of about 1500 feet, so you would need to take a 10-minute drive down a steep road just to get to the main highway serving town, and then wind your way down through some other roads before you finally reached a town beach.  Assuming you could find parking! 

I felt guilty hijacking your thread to talk about something that had nothing to do with IB! 

[Michael5689]

No worries.  I know the topography well there, but am more familiar with the beaches on Maui.  I did a quick search on Maui for similar red flags and it seemed like for the popular areas I checked the geotags for beaches were considerably more accurate.  Might just be a quality of the database they are leveraging issue compounded by an AI/ML that clearly is making some big mistakes.

[Kristina46]

I'm just wondering how places not near the beach can get categorized under Beaches, while I'm a quarter mile from mine and I miss the category, even though I have it in the title. As well as Views which is in the title and main cover photo.  But oh well! 

So just an IT question for you. When you make a major change this way and the machine learning AI is expected to eventually correct it, isn't that something you check out or beta test before making the whole thing live? How would they know about Beans Beach or Pohue Bay unless a live human told them and they corrected it? 

PS. My friend in Virginia just searched for me in Beaches and was delivered a result in Lahaina.  Not surprising, after all, it is only 20 minutes from here... BY AIR!

[Michael5689]

The problem is always how much training of the data and algorithm is enough so it's a matter of how much time and money they want to spend on it before having it go live.  There are always going to be bugs with something like this, but you hope you've done enough testing to limit them.

There are various level of post testing strategies and methodologies you can take, but basically it comes down to how much pre-data you've trained the models on, and how much post testing of the models you done with additional sets of data before going live.  The problem with the auto image recognition of what constitutes a "beach" probably lies somewhere in that balance.  They maybe could have added a post-check of not allowing anything to be called a "beach" that has a location that is more than X miles away from a coastline, but they either didn't do that or there's a problem with their software's determination of where a coastline in.  I've seen some replies on here that support is saying these determinations are partially based on the text in postings as well so that may be playing a role in fooling the algorithm sometimes.  These are not trivial or easy problems to solve.

The assigning of the wrong beach name to a location is probably something else entirely and could be both bad input data (GIS and location tags) or something wrong in the algorithm that is picking and assigning the names.  Really hard to say and I'm just guessing here.  It seems to be working in some locations though so I'm guessing bad input data.  They could probably add some software to do a post-check but it all depends on how far that Bean's Beach location extends in area in the input data from the actual real physical location on the map.  Basically is Bean's beach a 50 foot long strip of land on the coast that is 10 feet wide, or is it more like an X mile radius centered on the actual beach location.  I don't know enough of the details to comment on it any more knowledgably than that.

I'm really just guessing on all this stuff and have no inside knowledge other than friends that I know have been working in AirBNB for years with AI/ML job types.  I'd just judge that they were a bit aggressive on what features they chose to go live in this release.  As far as Lahaina, the public statements make clear they've tried to design something into the algorithm to show places outside what you'd expect in the search so I'd call that less a bug and more intended feature gone awry. 

[Helen744]

@Michael5689 I use IB all the time . so do a lot of others ,so this is relevant to me anyway . I dare not even look a the moment in case I accidentally push some button, I am frightened to update my pics Yikes. I hope they listen and as you say start to take some stuff on board but everyone seems to be missing in I hope action Yikes to you H

[Michael5689]

@Helen744 In this case it's a fairly easy thing to toggle.  I can literally make my search results appear or disappear in seconds just by toggling the "Host recommendation required" button on and off inside the Instant Book settings.  It's not even related to the volume of search results returned.  If I narrow down the map area I can impact the search results of AirBNB to get 3 instant book homes when I have that requirement turned into 4 homes when I have that requirement turned off.  The other two "Pre-booking message" and "government photo ID required" toggle buttons don't have any effect on the results.

[Gillian166]

I just tested this and was able to find my own listing, i have IB turned on + host recco. I've got all 3 requirements checked.

[Helen744]

@Gillian166 Maybe our roll out has been completed first? H

[Michael5689]

@Gillian166 Ok, good to know.  Thanks for checking.

[Emilia42]

@Michael5689 I'm confused but maybe because I read too quickly. If a user is searching for instant book listings (because they solely want to instant book) but they do not have any reviews (ie. they are not recommended by other hosts) then wouldn't the listings that are available to them ONLY be those that do not require a recommendation from other hosts? This is the only thing that makes logical sense to me.

[Michael5689]

@Emilia42 I agree it makes logical sense if that was the intent, but I don't think that's the intent based on how it behaves when the other IB requirements are turned on.  That same non-user who also doesn't have a government verified ID with AirBNB and is still able to see listings which require a government ID for instant book.  It seems like it's uneven across the world as @Gillian166 above tested it and even though she has gov ID + host recommendation + pre-booking message non-users can still see her listing for instant book options. 

[Emilia42]

@Michael5689 

Yes, but a government ID can be uploaded as part of the booking process. Airbnb may even ask the user to submit an ID when the new account is created. (That is what happened to me 5 years ago when I created an Airbnb account.) So those listings shouldn't necessarily be hidden from those wanted to instant book. But it is highly unliking for a guest who filters for an instant book property (because that is the type of booking process they desire) to obtain a review from another host at the moment they want to book. 

[Emilia42]

@Michael5689 

P.S. I also just searched for my listings incognito and only 1 of them showed up in the search results with the instant book filter on (my one listing that does not require host recommendations.) So I am getting the same results as you.

[Michael5689]

@Emilia42 Thanks for checking/verifying the behavior. @Gillian166 Could you confirm that when you tried this test you tried the test in a "non-logged" in mode to AirBNB in Australia?  Please note that you also need to ensure you have the "Instant Booking" filter on in your search criteria.

Mike

[Emilia42]

@Gillian166 And you have to select the "Instant Book" filter.