Wow, that was hard to find. You know how some things are just hard to Google because the search terms are so common, and others are hard to find because everything on the entire Internet starts with the presumption of some knowledge you don't have? Well, this fell into both categories; even though I had a copy of Acrobat Pro and its help open as well! You can also tell that Forms in Acrobat is something I've never done before.
Here's the deal (and you probably won't like it): custom validation is not available with LiveCycle forms, only with AcroForms.
Did you know there were two completely different incompatible types of form in there? I didn't.
You don't "create" an AcroForm. You create a vanilla PDF document, and you add form fields to it. The easiest way is to enable the Forms toolbar (View | Toolbars | Forms); then you click on an icon to select, say, the text input tool, and then double click with that tool in the document where you want that field. Right-click the field and select properties; when they appear, select the Validate tab, and you will see a box (enabled by a radio button) where you can put the custom validation script (cut and paste from the page I first linked you to).
Good luck!