Turns out 90% of that work was in a category I am fond of calling "stupid bullshit". I mean, it's neither stupid nor bullshit really, but it's stuff that's not really the point: parsing, unparsing, writing more clauses for silly little substitution and whnf and map functions and so on. Inventing stupid syntax so that I can write injections into sum times, write down arbitrary patterns, case analyze over them in concrete syntax --- all that nonsense. So I finally got back to debugging the pattern compilation code itself about midnight and found a stupid bug that I had introduced while simplifying it down from arbitrary tuple patterns to just pairs since that's what the rest of the support code we already have handles.
Anyway it mostly seems to work now for "unambitious naive pattern compilation without a coherent notion of matrix column because I don't have the time to do it right" values of "work".