I didn't read this very carefully so maybe I missed it, but I'm surprised they didn't try using a classifier on top of a BERT-style encoder model?