I find this - or perhaps the title - a bit surprising.
I've benchmarked GLM 5.3 and DSv4.1-F on my fully-annotated decomp of the Nintendo 3DS's kernel, which I have a good mental understanding of, tasking them to find vulns and other bugs (in Max mode w/ subagents). GLM 5.3 founds almost all the vulns in 30min for $22, while DS only found one vuln for $2 in 40min.
Perhaps DS works better where targets have low-hanging fruits than can be found fast?
A little off-topic: where does one use those models such as GLM or DS for this kind of reverse engineering tasks? I think I read many of them refuse to help with tasks like those on their official platforms.
It was very helpful in the aftermath of a client whose WordPress got stuffed up and analysing when it happened, how it happened, and what the entry point was. The American frontier models refused to help because, well, they just refuse to help with that kind of thing.
TuxSH · · focus · HN ↗
I've benchmarked GLM 5.3 and DSv4.1-F on my fully-annotated decomp of the Nintendo 3DS's kernel, which I have a good mental understanding of, tasking them to find vulns and other bugs (in Max mode w/ subagents). GLM 5.3 founds almost all the vulns in 30min for $22, while DS only found one vuln for $2 in 40min.
Perhaps DS works better where targets have low-hanging fruits than can be found fast?
severino · · focus · HN ↗
trollbridge · · focus · HN ↗
It was very helpful in the aftermath of a client whose WordPress got stuffed up and analysing when it happened, how it happened, and what the entry point was. The American frontier models refused to help because, well, they just refuse to help with that kind of thing.