"Now imagine the pair of us at one of those parties, and reconsider who exactly is lucky to have escaped such a fate."

Sherlock is certain that only disaster could have resulted, and knowing Tony, that disaster would have befallen everyone else.