I find this - or perhaps the title - a bit surprising.
I've benchmarked GLM 5.3 and DSv4.1-F on my fully-annotated decomp of the Nintendo 3DS's kernel, which I have a good mental understanding of, tasking them to find vulns and other bugs (in Max mode w/ subagents). GLM 5.3 founds almost all the vulns in 30min for $22, while DS only found one vuln for $2 in 40min.
Perhaps DS works better where targets have low-hanging fruits than can be found fast?
Also depends which glm 5.3 you use. I used open router and sometimes had good, then appalling performance. It turned out the provider being routed to - some were fp8 and some were fp4. The fp4 ones glitched on anything complex. It is such a difference that I only allow fp8 providers in my allow list.
TuxSH · · focus · HN ↗
I've benchmarked GLM 5.3 and DSv4.1-F on my fully-annotated decomp of the Nintendo 3DS's kernel, which I have a good mental understanding of, tasking them to find vulns and other bugs (in Max mode w/ subagents). GLM 5.3 founds almost all the vulns in 30min for $22, while DS only found one vuln for $2 in 40min.
Perhaps DS works better where targets have low-hanging fruits than can be found fast?
cregy · · focus · HN ↗
MisterMunchkin · · focus · HN ↗
Ey7NFZ3P0nzAe · · focus · HN ↗