Thread

Jeff – Jev-compatible 0.8B decision models, trained at home, ~30 ms

575 points · 225 comments · firelex

adrithmetiqa · 1790632786 · focus · HN ↗
Forgive my lack of understanding but how long before Jev type functionality is just built straight into all frontier models?
1. tbeseda · 1790633333 · focus · HN ↗
  If I had to guess, it's already built and is just waiting on Product's/Marketing's desk. How do you position this without looking like your roadmap is being determined by newcomers? Probably don't want to adopt the same verbiage+acronyms - but also can't be seen to be just sherlocking features.
  1. seizethecheese · 1790633838 · focus · HN ↗
    I think Apple has demonstrated that shipping second has essentially no negative impact if your product is seen as higher quality.
    bigyabai · 1790634141 · focus · HN ↗
    I think Nvidia has demonstrated that shipping first is a multi-trillion dollar opportunity if you don't shy away from a challenge.
    lucideer · 1790634641 · focus · HN ↗
    I think it depends on the model: Nvidia ship products with APIs, apple/jev/etc. ship end user products. The former is much more subject to lock-in, increasing the value of early market adoption because there's a 3P Nvidia ecosystem sprung up in response. Apple/jev/etc. do have APIs & corresponding 3P ecosystems but those are usually a smaller component of market capture than direct product end users, so the space ends up more competitive.
    8note · 1790650178 · focus · HN ↗
    jev ships an api, does it not?
    lucideer · 1790696404 · focus · HN ↗
    yup. I said that in my comment...
    throwaway27448 · 1790634811 · focus · HN ↗
    Chip manufacturing intrinsically comes with one hell of a moat. There's not much parallel in software.
    slashdev · 1790636087 · focus · HN ↗
    That’s kind of funny because Nvidia’s biggest moat is arguably CUDA, the software ecosystem around their chips
    mcmcmc · 1790637193 · focus · HN ↗
    CUDA is a lock-in moat, the infrastructure needed for chip manufacturing is a barrier-to-entry moat. Two different things.
    angry_octet · 1790638483 · focus · HN ↗
    There are many microarchitecture patents used in NVIDIA chips. I'm sure they have a team that rips apart AMD chips looking for infringement. The way CUDA works is tied to many GPU architecture decisions and it would be hard to decouple them efficiently. Obviously a huge effort was made to get PyTorch decoupled from CUDA.
    AtlasBarfed · 1790640196 · focus · HN ↗
    If cuda is an API, and llms make apis effortless, then how big of a moat is cuda?
    robflynn · 1790641446 · focus · HN ↗
    I ran across this a few days ago: <a href="https://zluda.org/" rel="nofollow">https://zluda.org/
    QuantumNomad_ · 1790657744 · focus · HN ↗
    All of the buttons and links on that page redirect to spam pages. Most of the times I clicked, it brings me to some site that wants to sell me a VPN.
    robflynn · 1790696321 · focus · HN ↗
    Oh, yikes, I should've checked those links before posting it here. That's certainly not a good look for them.
    There was a github repo but I have not checked it.
    edit I see, thats an unaffiliated site that latched onto that, my bad, here's the GH that I should've linked: <a href="https://github.com/vosen/ZLUDA" rel="nofollow">https://github.com/vosen/ZLUDA
    [deleted] · 1790696393 · focus · HN ↗
    [deleted]
    bobmarleybiceps · 1790643141 · focus · HN ↗
    IMO, yes a lot of the "nvidia pays lots of people to make non-portable, tightly coupled backends to open source projects" is potentially going to be less of a moat?
    (Though it could turn into "nvidia pay lots of people to use LLMs to make non-portable, tightly coupled backends to _even more_ open source projects")
    trollbridge · 1790644234 · focus · HN ↗
    It turns out the moat is “writing drivers that work”; Nvidia drivers simply work, and Intel’s are poor quality. So if I want stuff that works I need to buy Nvidia gear.
    latentsea · 1790650601 · focus · HN ↗
    It's becoming less of one. Previously I would have shied away from buying an AMD card because of CUDA, but with local LLMs getting good enough to be usable and frontier models becoming as good as they have, I bit the bullet and got an R9700 for local inference. Dealing with working around CUDA used to be more of a manual process, but when you can point an agent at it and get stuff working, it's dramatically less painful and scary than it used to be. Plus, at least in ComfyUI and local LLMs I'm finding support for AMD has gotten really good. Lately I've been witnessing a lot of people using agents to write custom kernels for RDNA4 and improving performance dramatically.
    flyinglizard · 1790643751 · focus · HN ↗
    I’m totally guessing, but I can’t imagine CUDA has any significance at the frontier lab scale. The operational and capex costs are so massive that the convenience of the platform becomes a minuscule consideration.
    It’s just that Nvidia’s stuff works, and available at scale, and includes the full stack with networking, cooling and such.
    angry_octet · 1790638231 · focus · HN ↗
    Historically there was a big patent moat in (Graphics) GPU design. This continues with CUDA, but obviously Intel and AMD could find ways to support eg PyTorch. What we don't know is how much effort that cost them, or why they decided they couldn't make a CUDA API compatible competitor.
    trollbridge · 1790643091 · focus · HN ↗
    Intel could have beat the pants off Nvidia a long time ago with Arc if they’d bothered to ship usable drivers. But they refuse to, and simply can’t figure it out, so Arc cards remain cheap because they’re so #%£€ing hard to get working well, and everyone is nervous they’ll lay off the driver team again.
    api · 1790643292 · focus · HN ↗
    AMD has always had software problems too. Hardware companies often devalue software and suck at it.
    If you can make them work Arc cards are a massive bargain. On raw compute the silicon is not bad.
    [deleted] · 1790635762 · focus · HN ↗
    [deleted]
    AndrewKemendo · 1790637997 · focus · HN ↗
    NVIDIA literally hired all of the 3DFX team and patents after they already proved the graphics accelleration hardware market was massive with the Voodoo card line
    In fact NVIDIA wasn't even a close competitor to 3DFX in the graphics card game at that point
    bigyabai · 1790640428 · focus · HN ↗
    It panned out great. 3DFX filed for bankruptcy less than 18 months later, and Nvidia could pivot from designing raster chips to considering CUDA's architecture.
    It's not like 3DFX was the first GPU vendor. Nvidia saw the opportunity to be the first true GPGPU vendor, and they beat their competitors.
    trollbridge · 1790643337 · focus · HN ↗
    IBM shipped the first PC GPU (the Image Adapter/A, 1989) following on the first PC 2D accelerator (the 8514/A, 1987). There was zero benefit to being first. 3dfx had the first mass market, cheap GPU in 1996.
    ATI (AMD) in 1989 copied the unpatentable parts of the 8514/A, improved it, and went on to dominate 2D accelerators.
    nVidia’s first 3D card was a complete flop. They did not achieve success for years.
    Nvidia is in the right place at the right time.
    bigyabai · 1790646189 · focus · HN ↗
    We're talking about GPGPU products, not raster GPUs. I cleared that up pretty well in my last comment.
    trollbridge · 1790685848 · focus · HN ↗
    The IA/A was general purpose.
    Keyframe · 1790647321 · focus · HN ↗
    not to be _that_ guy, but S3 was the dominant force in 3D. ATI was nowhere to be seen with their Mach/Rage crap. It was an S3 era, and then seemingly out of nowhere 3dfx swept in with Voodoo (not seemingly though - 3dfx came out from imploding SGI). Only a bit later Nvidia, after it recovered from NV1 fiasco, brute forced relentlessly from Riva 128 to TNT to TNT2 to Geforce 256 which broke 3dfx (along with their lack of business sense) and had Nvidia bought them. ATI did their 2D schtick only during that timeline while S3 stumbled with Virge and ATI didn't come as a threat until Radeon; After they bought ArtX (which did Gamecube graphics) which turned into Radeon. S3 died off in that race with Savage3D and the only other major players were Matrox and 3dlabs. Matrox kind of found temp refuge in video segment, and 3dlabs infamously pushed for OpenGL 2 and survived for a bit on 3d workstations. The most surprising (IMO) was the downfall of E&S which kickstarted most of the things mentioned. E&S and Real3D are both a great story in themselves how first movers can become absolutely forgotten and obscure real quick (with Intel740 ended up in ATI).
    trollbridge · 1790685824 · focus · HN ↗
    S3’s stuff was no more “3D” than the Image Adapter/A was.
    You should look at the IA/A - it had its own C like compiler, CPU, etc which did things reminiscent of a modern GPU or SIMD.
    PunchyHamster · 1790641796 · focus · HN ↗
    It's more due to ineptitude of competition. If AMD shipped second, but better product it would be another thing but it is still a bit of a mess of an ecosystem on AMD side
    bigyabai · 1790646366 · focus · HN ↗
    It's not AMD's responsibility to dethrone Nvidia any more than it is Apple's. AMD sells CDNA, but the momentum is with CUDA and Khronos can't get anyone to sit at the same table anymore.
    hgoel · 1790651433 · focus · HN ↗
    It isn't their responsibility, but it doesn't make sense to argue that being first got NVIDIA a multitrillion dollar market if the others aren't even trying to compete. There is no "first" if it's really just "only one even trying".
    The momentum is with CUDA because CUDA is the most broadly usable one. Especially with AI-driven optimization loops and similar APIs, competitors can more easily pick up momentum, if they'd actually try.
  2. michaelrwolfe1 · 1790646038 · focus · HN ↗
    There is zero stigma to shipping second. If anything, the labs’ customers are probably begging for them to add these features natively so they don’t have to deal with the hassle of adding another provider to their stack.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.