I’m running into the same thing. I’m relatively sure my DNS is up and running properly, and that I’ve created my self-signed certs properly, but mail just doesn’t wanna use ’em.
I followed the steps demo’d in the WWDC session exactly, as near as I can tell, but am not getting the same results.