the tokens are generated by hardware with secure enclaves (encrypted weights) and then sent over a network to some remote CPU where they can manifest an effect.
it's not much different during training.
how exactly are they supposed to exfiltrate their weights? you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
I sincerely doubt anyone is paying the cost for that in training, the overhead is small but it isn’t negligible and training is when it matters most. <a href="https://tee.fail" rel="nofollow">https://tee.fail can solve it if they are.
it takes half a percentage point off the top last time i evaluated it (nvidia). you might call that cheap but that’s millions of dollars in a run, and for what, protecting from who? especially when the platforms have been compromised to the point of key leak (which they have).
edit: i just looked up training numbers and the impact is even worse, 20-30% throughput vaporized. yeah, nobody is doing that.
teravor · · focus · HN ↗
it's not much different during training.
how exactly are they supposed to exfiltrate their weights? you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
cmrx64 · · focus · HN ↗
teravor · · focus · HN ↗
for example every TPU/GPU has its own private key and the devs load the weights into it by sending it encrypted weights.
cmrx64 · · focus · HN ↗
edit: i just looked up training numbers and the impact is even worse, 20-30% throughput vaporized. yeah, nobody is doing that.