Thanks so much for your response, @Emiel1 .
In the end, I guess it doesn't matter why the guest felt uncomfortable or unsafe, the result is the same - choosing not to stay in my home. But his request for a refund stated "sketchy, unsafe area" as the reason, and this is a snap judgment made without even getting out of the car. The only thing I can imagine giving someone the impression of "sketchiness" or a lack of safety on my block in the middle of the day on a Saturday afternoon is noticing that the residents are all people of color. Folks on my block just sit on their porches and relax on a weekend afternoon. There's never even anyone hanging out on the corner. Yes, there are a few empty, overgrown lots around, and some buildings that are in the process of being renovated, but it's a pretty quiet block.
What I think happened is that someone had a reaction based on racial bias, and this upsets me. It's beside the point in figuring out whether or not I'm repsonsible for giving the refund, which is really what I'm asking, but it does make me feel sad and angry.
Thanks again for the advice.