I like using large patterns in CyanogenMod because I find they're easier to input than PINs. However, we run into the issue with TWRP of how to decrypt these on the command line using twrp decrypt, since they are obviously not character-based. It is reasonable to assume that some grid is used, but not clear which direction (i.e. row-first or column-first), and what to do after 9.
As far as I can tell, the password derivation formula has changed at least twice over the years, first to make the grid start with zero instead of one, then to the new "consistent" (but very confusing) algorithm. Basically, we go filling rows first, and if we need more, instead of going to hex, we just continue in the rest of ASCII. I found this information by a combination of Google and reading TWRP source: https://github.com/TeamWin/Team-Win-Recovery-Project/blob/android-6.0/gui/patternpassword.cpp#L417.
Without further adieu, here are the grids:
| 1 | 2 | 3 |
| 4 | 5 | 6 |
| 7 | 8 | 9 |
| 1 | 2 | 3 | 4 |
| 5 | 6 | 7 | 8 |
| 9 | : | ; | < |
| = | > | ? | @ |
| 1 | 2 | 3 | 4 | 5 |
| 6 | 7 | 8 | 9 | : |
| ; | < | = | > | ? |
| @ | A | B | C | D |
| E | F | G | H | I |
Example: if your pattern is a "U" shape in the middle of the 5x5, enter twrp decrypt '7<ABC>9' in the terminal or ADB shell. Note that characters ;<>? are special to the shell and require backslash escaping or single quotes around the string.
I have tested the 5x5 grid with CyanogenMod 13.0, but it should work with any Android M ROM using standard encryption.