Hi @Mark-And-Mary-Jane0, I would most definitely decline the request. You can tell him that while you do accept up to four people, your approval was based on one person. You are not comfortable with a group in town for a sporting event. Tell him you do not want the furniture to be moved around. You are very serious about quiet times and any kind of partying is not allowed. Since you are right there, you are in a position to enforce these points. You requested info/names on the others and it was not provided. Mentioning some of this stuff will make him go running. They will not want to feel as if they are being monitored.
I do not see why you would be penalized for declining a reservation change. But then who knows what the fellow will say or if he will complain. Offer a full refund to eliminate the hassle. Also be aware that if he is coming for a tournament, others are as well. Ask lots of questions of any other potential bookers.