Sorry you've been frightened by a potentially bad person and glad that you had the smarts to send them away. My suspicion is that this situation with this person is very rare.
This has never happened to us in 2 years of hosting and you can see the number of my home on the first picture of our property listing. Our house is pretty unique looking from the outside, it's not hard to find if you look at the map and do a Google Street View search. If someone wants to find your home, it's not hard to do via property tax records and your name alone (if you own your home). Heck, if someone were especially enterprising, they could go peer into the windows until they found the property that matched the pictures. Shifty people are going to be shifty, there's only so much you can do.
This is exactly why we insist on a picture of a guest's face when they place a booking - any day a person could just walk up and say they are a guest... if we are expecting someone, without some knowledge of what they look like, we might be inclined to let them in. Knowing if we were expecting someone wouldn't be hard to figure out either, our calendar shows when we're booked or not. It's also why we have an alarm system, give every guest an alarm code of their own and announce it on our listing that we have one a guest will have to use. This deters the shifties from thinking we're an easy target.
