I wouldn't cancel since you have nothing in your listing saying you don't accept babies. Chalk it up to a learning experience. Here's what you need to do:
1.) If you don't want babies, children under a certain age, etc. you need to say this either in your description or House Rules.
2.) You need more specific House Rules on your listing. Look at other listings and see what other hosts write and add those that apply to you.
3. ) You need to establish a security deposit in the event anything is ruined or damaged by a guest. Believe it, it happens.
Again, don't ruin your track record by cancelling. These guests might not be a problem at all!