It's becoming more clear that for big enterprises to really adopt AI, they need to use open models. Especially if they want to own their own intelligence, which they should.
I spent the last 2 days building basic AI agents to automate some mundane supply chain workflows for a large company. Those seemingly boring workflows had bank statements, supplier IDs and other sensitive information.
For me it was all alarm bells, there is no way they can afford to give closed models access to this data. I was compelled to figure out an open model based solution for them, which made me realize that this is probably the only way for enterprises going forward.
I've actually flipped on this the last few days because of liability.
The big labs are going to be on the hook for rogue behavior by Claude or Sol. Customers will be able to sue for damages and deflect regulators if their customer data is abused or their agents attack external services.
If you use a Chinese OSS model and it goes rogue? Yeah good luck with that, your shop is 100% on the hook.
Enterprises have barely deployed empowered agents yet. The models capable of doing this have only been available for months. Give it a little time, it's coming.
... when are large companies on the hook for anything, ever?
I mean, hypothetically, yes, but class-action lawsuits get settled out-of-court, the lawyers get paid in Ferrari-multiples, the plaintiffs get paid in McDonalds coupons that expire in two weeks.
Slaps-on-the-wrist are written into the laws; a million-dollar fine is existential for a small company, and likely not even a line-item at Anthropic.
I think the minute a big lab is found to be liable, the whole edifice along with trillions of dollars of investment and VC comes tumbling down. I think that is part of the reason the labs are pushing for more regulation. They can say "We're not liable, we complied with all of the regulations". The actions of multi-billion parameter models trained on data harvested from millions of Internet users over the years can never really be understood - if a business is found to be liable for that, then nobody would ever operate in that space.
They are pushing for regulation because they are human being and don't want all human beings to die.
I'm sorry you are so jaded you can't recognize honesty when you see it, but that is what everyone deep in the AI space, including the non-executive researchers, are worried about.
I heard a similar theory on CNBC's Squawk Box.
Sorkin presented a theory that Nvidia bought HuggingFace to protect OpenAI from legal consequences. That a lawsuit determining accountability of actions by LLMs could threaten the AI financial network.
People - usually suit-wearers - have been making this spurious claim for decades, but it doesn't hold water.
The largest of the finest print reminding you that it's 'sold as is' (or more encompassing variants that might continue '... with no warranty for fitness of purpose') means that liability remains in the lap of the purchaser / consumer / operator.
(This has been a source of immense frustration over my career - where such people have assured me that they have 'recourse' (it's always vaguely described) by spending money on proprietary products & services, rather than opting for functionally equivalent or superior free options.)
I think your third paragraph is implying a distinction (or conflating the difference?) between LLMaaS's and self-hosting publicly available models.
If it's just where it's hosted that provides the legal insulation then things like OpenRouter would give you that. (But again, I suggest that it would not.)
If a frontier lab is willing to draft an SLA that assumes liability, corporate will pay for it as long as the cost/benefit is in favor of it over insourcing.
Sure, but that's quite a fanciful universe you're imagining there - the feasibility of a corporation obtaining insurance to cover that offer of liability ownership has got to be close to zero.
I know this is the risk management answer, but when you're the featured story on the news because of a data breach, noone hears "butbutbut it's Anthropic/OpenAI/whoevers fault...". So it's a balance between "there's someone we can sue" and "what's our reputation worth".
It actually does help a lot to be able to say your OpenAI agent was the fault. People recognize the name. The press doesn't want to write about Better Home Life Insurance agents running loose on the internet, nobody cares.
Having been at the table too many times working corp level incident response to major 3rd party breaches with executive management and spin doctors, no it doesn't. The headline is "Better Home Life Ins BREACHED!:", and somewhere in the 2nd or 3rd paragraph "BHLI says it was because agents went wild".
Now, some tech web site might lead with "Agents at it again...", but the WSJ, et al usually won't.
Everybody including companies need their own models, especially considering that llm providers like openAi have no problem siphoning off your data and intelligence by inspecting metadata. and claiming any resulting value as their own.
I don't think you need to trust them if you are running on something like bedrock. Then I think you're trusting AWS or whatever cloud equivalent you want.
As long as you don't have "realtime" workloads, owning the GPUs quickly becomes the economical option. The main cost problems is in e.g. chat applications where the workload is spikey, and users expect an near-instant response, for which you need to scale the GPUs to the highest spikes of the workload.
You really don't need to be large. $100k can buy you a lot of compute and it's less than hiring an engineer. With that kind of money you can build an LLM server for a dozen people.
One engineer's salary to accelerate a team of twelve is so cheap you can't afford not to.
Open models on-prem is the future, not a single doubt in my mind.
I'm old enough to remember when my company had everything on-prem (both analytical and operational databases and servers) due to cost and security concerns. Nowadays we have everything on GCP.
The biggest problem we had with on-prem was maintenance as it took a lot of staff and time to ensure decent reliability.
If you don't believe AWS follows their ToS, you shouldn't use any cloud provider for CPU/data storage/anything else either. That's not a mainstream position in the industry.
The Enterprise plans with OpenAI and Anthropic include clauses that they won't use your data for training (unlike the 'civilian' plans).
good question! i was able to build the agents by using Opencode and used a combo of locally hosted models. Specifically Kimi and Deepseek.
but didnt deploy an agent anywhere. instead we wrote all the code, system prompts, harness, etc locally and just shared those files with the customer.
all the customer had to do is deploy what we provided them on their locally hosted models, essentially just had to add inference. and boom, started working on their systems!
Zaraif13 · · focus · HN ↗
I spent the last 2 days building basic AI agents to automate some mundane supply chain workflows for a large company. Those seemingly boring workflows had bank statements, supplier IDs and other sensitive information.
For me it was all alarm bells, there is no way they can afford to give closed models access to this data. I was compelled to figure out an open model based solution for them, which made me realize that this is probably the only way for enterprises going forward.
ramshanker · · focus · HN ↗
anotherCodder · · focus · HN ↗
[dead]
bpodgursky · · focus · HN ↗
The big labs are going to be on the hook for rogue behavior by Claude or Sol. Customers will be able to sue for damages and deflect regulators if their customer data is abused or their agents attack external services.
If you use a Chinese OSS model and it goes rogue? Yeah good luck with that, your shop is 100% on the hook.
the_sleaze_ · · focus · HN ↗
They haven't so far.
bpodgursky · · focus · HN ↗
donw · · focus · HN ↗
I mean, hypothetically, yes, but class-action lawsuits get settled out-of-court, the lawyers get paid in Ferrari-multiples, the plaintiffs get paid in McDonalds coupons that expire in two weeks.
Slaps-on-the-wrist are written into the laws; a million-dollar fine is existential for a small company, and likely not even a line-item at Anthropic.
hghid · · focus · HN ↗
bpodgursky · · focus · HN ↗
I'm sorry you are so jaded you can't recognize honesty when you see it, but that is what everyone deep in the AI space, including the non-executive researchers, are worried about.
plipt · · focus · HN ↗
Sorkin presented a theory that Nvidia bought HuggingFace to protect OpenAI from legal consequences. That a lawsuit determining accountability of actions by LLMs could threaten the AI financial network.
<a href="https://youtu.be/4qV5WWgFTS8?t=323" rel="nofollow">https://youtu.be/4qV5WWgFTS8?t=323
Jedd · · focus · HN ↗
The largest of the finest print reminding you that it's 'sold as is' (or more encompassing variants that might continue '... with no warranty for fitness of purpose') means that liability remains in the lap of the purchaser / consumer / operator.
(This has been a source of immense frustration over my career - where such people have assured me that they have 'recourse' (it's always vaguely described) by spending money on proprietary products & services, rather than opting for functionally equivalent or superior free options.)
I think your third paragraph is implying a distinction (or conflating the difference?) between LLMaaS's and self-hosting publicly available models.
If it's just where it's hosted that provides the legal insulation then things like OpenRouter would give you that. (But again, I suggest that it would not.)
cmgriffing · · focus · HN ↗
If a frontier lab is willing to draft an SLA that assumes liability, corporate will pay for it as long as the cost/benefit is in favor of it over insourcing.
Right?
Jedd · · focus · HN ↗
kjs3 · · focus · HN ↗
bpodgursky · · focus · HN ↗
kjs3 · · focus · HN ↗
Now, some tech web site might lead with "Agents at it again...", but the WSJ, et al usually won't.
mentalgear · · focus · HN ↗
solidasparagus · · focus · HN ↗
chillfox · · focus · HN ↗
0xDEAFBEAD · · focus · HN ↗
<a href="https://www.openaifiles.org/ceo-integrity" rel="nofollow">https://www.openaifiles.org/ceo-integrity
solidasparagus · · focus · HN ↗
tomp · · focus · HN ↗
So if you're running open models on AWS GPUs, you might as well run Claude (which AWS supports, and doesn't share any data with Anthropic).
Same with Azure/OpenAI.
schnitzelstoat · · focus · HN ↗
hobofan · · focus · HN ↗
unrented7977 · · focus · HN ↗
One engineer's salary to accelerate a team of twelve is so cheap you can't afford not to.
Open models on-prem is the future, not a single doubt in my mind.
schnitzelstoat · · focus · HN ↗
The biggest problem we had with on-prem was maintenance as it took a lot of staff and time to ensure decent reliability.
monegator · · focus · HN ↗
I want to believe (pinky promises from terms of service don't count)
criemen · · focus · HN ↗
abalashov · · focus · HN ↗
Ah... oh.
Well, it's a nice thought.
schnitzelstoat · · focus · HN ↗
I guess it depends if we believe that or not.
kjs3 · · focus · HN ↗
I see the flaw in your clever plan...
ThePestoParagon · · focus · HN ↗
Zaraif13 · · focus · HN ↗
but didnt deploy an agent anywhere. instead we wrote all the code, system prompts, harness, etc locally and just shared those files with the customer.
all the customer had to do is deploy what we provided them on their locally hosted models, essentially just had to add inference. and boom, started working on their systems!