I don’t think picking a handful of SPEC benchmarks that approximate today’s most common agentic workloads (compiling code, interpreting Python) and then calling them “agentic benchmarks” is misleading at all.
That you need a whole lot of “ordinary” compute to benefit from the scaling properties of agents is the reason Nvidia is making this chip in the first place.
The four benchmarks selected are cppcheck, llvm, cpython, and gcc [1]. These are all essentially compiler benchmarks... and all of the compiler benchmarks in SPEC cpu2026! This makes the benchmark selection somewhat suspicious to me, since it's not particularly representative of a diverse set of workloads.
I also don't buy that it's a particularly representative set of tasks you might do with agents. Also included in the SPEC benchmarks are multimedia codecs, lossless data compression codecs, sqlite (i.e., database), all of which are going to be things you should easily throw into the sets of tasks an agentic workload might do. Cherry-picking just the compiler benchmarks instead of all of SPECint... again, it just raises a couple of eyebrows.
[1] To be honest, I'm kinda surprised that both gcc and llvm are in SPEC cpu2026.
The code in compilers is the closest to your typical app you can get in a benchmark like SPEC, eveerything else is actually far more specialized. Compiler code is full of small basic blocks, lots of branches, indirect memory access; it's actually harder to get good performance for such code, both for CPUs and compilers (that was part of the death of Itanium too).
Death of Itanium was that it was a) VLIW and b) Intel was too arrogant. So it went to the same destination as later Larrabee and ATI/AMD attempts at VLIW GPUs.
That is, nowhere.
Also you are wrong and anyone sizing up an arch to put their loads onto must first try that load on it and not rely on "bah, compilers compile on it".
In a hypothetical world where AMD wasn't allowed to do AMD64, and Intel stayed committed to Itanium: Itanium would still have sucked, and both PowerPC and SPARC would have out-sold Itanium by an even wider margin than they did in this reality. Itanium could only have succeeded if AMD64 wasn't possible and literally all of the competing 64-bit architectures were killed off by their owners so they could jump on the Itanium bandwagon. Itanium managed to kill off PA-RISC and Alpha and (high-end) MIPS roadmaps, but it still had competitors that were not just viable but actually more successful.
twoodfin · · focus · HN ↗
That you need a whole lot of “ordinary” compute to benefit from the scaling properties of agents is the reason Nvidia is making this chip in the first place.
jcranmer · · focus · HN ↗
I also don't buy that it's a particularly representative set of tasks you might do with agents. Also included in the SPEC benchmarks are multimedia codecs, lossless data compression codecs, sqlite (i.e., database), all of which are going to be things you should easily throw into the sets of tasks an agentic workload might do. Cherry-picking just the compiler benchmarks instead of all of SPECint... again, it just raises a couple of eyebrows.
[1] To be honest, I'm kinda surprised that both gcc and llvm are in SPEC cpu2026.
fibonacci112358 · · focus · HN ↗
lstodd · · focus · HN ↗
That is, nowhere.
Also you are wrong and anyone sizing up an arch to put their loads onto must first try that load on it and not rely on "bah, compilers compile on it".
pjmlp · · focus · HN ↗
wtallis · · focus · HN ↗
pjmlp · · focus · HN ↗
Remember, the very first Windows XP 64 bit release was on Itanium.
my123 · · focus · HN ↗