One way: separately embed sender, recipients, subject, body - then use the embedding vectors as input to a logistic classifier
With that setup, I get 95% accuracy on email classification, training on 50-100 base examples. The model trains on CPU in under 1min, and it does inference in under 20ms (most of it is running the embeddings, so you can make it faster if you train your own embeddings model)
Here’s a gist with some sample code: <a href="https://gist.github.com/nicobrenner/056a5aaff5d0119c0032ecdad5029557" rel="nofollow">https://gist.github.com/nicobrenner/056a5aaff5d0119c0032ecda...
That code applies the embeddings + classifier setup on the Banking77 dataset. It gets 93-94% accuracy depending on the embeddings you use (SOTA for this is ~95%, with much bigger and slower models)
I wonder what numbers you'd get using another system one model - Contrastive Language Model <a href="https://contrastive-lm.notion.site/" rel="nofollow">https://contrastive-lm.notion.site/
That model scales very well with quantities of requests.
sharih · · focus · HN ↗
zihotki · · focus · HN ↗
nico · · focus · HN ↗
One way: separately embed sender, recipients, subject, body - then use the embedding vectors as input to a logistic classifier
With that setup, I get 95% accuracy on email classification, training on 50-100 base examples. The model trains on CPU in under 1min, and it does inference in under 20ms (most of it is running the embeddings, so you can make it faster if you train your own embeddings model)
Here’s a gist with some sample code: <a href="https://gist.github.com/nicobrenner/056a5aaff5d0119c0032ecdad5029557" rel="nofollow">https://gist.github.com/nicobrenner/056a5aaff5d0119c0032ecda...
That code applies the embeddings + classifier setup on the Banking77 dataset. It gets 93-94% accuracy depending on the embeddings you use (SOTA for this is ~95%, with much bigger and slower models)
zihotki · · focus · HN ↗
That model scales very well with quantities of requests.