@Sophie1105 I can totally relate- so many things offend my aesthetic sense 🙂 Are you by any chance a Libra?
I've found nice-looking pump dispensers quite inexpensively in Walmart, dollar stores, all kinds of places. I'm talking pottery, or stone or stainless steel. I don't know how you'd secure them, though, so no one would actually take them if they fancied. The only problem with pottery or stone is that if they drop it on the tile floor, they'd break. So I prefer stainless.
You could print out small labels for what's inside, laminate them, and glue them on the containers, or if you have an artistic hand, or a friend who does, paint the labelling on with a small brush and oil-based paint.
And as David pointed out, it's important to use antibacterial wipes on the outside of the containers between each guest, removing the lid and cleaning all the crud off.