Data-only attacks are somewhat low-hanging fruit. Classical static analysis could already find them before AI got this strong, and LLMs make identification even easier. But the real threat is risk buried in business logic, especially abuse of normal business logic. Take e-commerce refund abuse. Bug hunters would not even call it a risk, yet fraud rings have arbitraged millions off this kind of logic. And because the logic is legitimate business logic, it is very hard to detect.
> And because the logic is legitimate business logic, it is very hard to detect.
Hard to detect at n=1, yes. But larger scale - are you assuming that no Accounting or Sales managers are watching the returns ratios, nobody in Shipping is minding carrier delivery failure metrics, and nobody in Returns is raising alarms about the bricks they're receiving?
mgaldys4 · · focus · HN ↗
bell-cot · · focus · HN ↗
Hard to detect at n=1, yes. But larger scale - are you assuming that no Accounting or Sales managers are watching the returns ratios, nobody in Shipping is minding carrier delivery failure metrics, and nobody in Returns is raising alarms about the bricks they're receiving?