I don’t I think it’s unusual for this to happen. I myself have backed out once or twice, but always having the courtesy to let the person know. Mostly when I say I’ll meet with somebody I’ll meet with them.

It is good to always meet the first time just to get acquainted. That way, you each can find out if it’s obvious that you’re not a good fit.