This is an excellent use case for completely local, small model inference, yet for inexplicable reasons Mozilla wants to normalize uploading your entire private browsing history to a cloud.
These (Mistral's and Mozilla's) marketing pages aren't candid enough to clearly explain the difference between local and cloud inference, and that they're asking you to consent to enabling the latter. I'd call that the bare minimum of ethics. If you're in a position of authority over your less technical users, you are ethically obligated to give them the full picture of what are you doing and why they should consent.
(Aside to any Mozilla people who might be reading HN, your page here[0] has an oversight, it advertises this model as "Mistral Small 4" but the hyperlink is to OpenAI's model card for gpt-oss-120b).
I am not trying to defend Mozilla doing this, and I don't support sending data to cloud based services like this in a way that users won't understand.
But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser.
You don't want a browser that takes 10GB of extra RAM (on top of the memory hog that is having just 3 or 4 complex tabs open on its own already) and pegs your CPU at 99% usage for minutes at a time. And not in an era when mass market consumer laptops are still commonly 8GB or 16GB of total system RAM. Many of those with integrated-into-CPU onboard graphics (eg: not a gaming laptop with a discrete GPU on the PCI-E bus).
It'll be a catastrophe for laptop battery life, among other resource use problems. And an LLM that fits in under 8 to 10GB of RAM for CPU-only inference is not going to be nearly as capable as an off-device inference system.
I wish they had just done this with a very clear up front opt in (not enabled by default) thing that explains what Mistral is, that it's not some big American cloud company but a relatively small startup in France, and that your prompts/LLM interactions will go to their servers. And some documentation on how it will be handled/stored in a supposedly trustworthy manner.
> that it's not some big American cloud company but a relatively small startup in France
How it's any better? Small companies can be bought by big companies. French government is as capable of trampling over their citizen's privacy the moment they feel they need it as US government is. And putting a squeeze on a small startup is way easier than on a major cloud company (not that either is particularly hard). Also, OpenAI used to be an idealistic non-profit one day too, then it started to smell trillions and all that went of of the window.
> is not going to be nearly as capable as an off-device inference system.
I rarely need PhD-level research into my browsing history. I'm not going to solve millennium problems on my bookmarks. The tasks that I will realistically need are well within capacity of most very basic local models. Maybe they'd be a bit slower, who cares.
>How it's any better? Small companies can be bought by big companies. French government is as capable of trampling over their citizen's privacy the moment they feel they need it as US government is.
It's better for Europeans because the company in question is European and so will not export their data to New Jersey. And because any success it has will presumably better benefit France and Europe.
> It's better for Europeans because the company in question is European and so will not export their data to New Jersey
There aren't many government institutions in New Jersey that would take a lot of interest in anything Frenchmen are doing. There are a lot of institutions in France that would be interested in anything Frenchmen are doing, if it contradicts what the French government wants to be happening. The main threat to citizen's privacy always comes from the government closest to them, the government overseas has its own citizens to worry about and pays much less attention to foreign citizens on foreign land. There could be exceptions, true, but as a rule, if you look into New Jersey, you'd sooner find mass surveillance of American citizens than mass surveillance of the French.
Same goes for commercial interests. If I want to run targeted ads in New Jersey, I want to have profiles of New Jersey people, not French people. So buying data from France in New Jersey would not be a routine occurrence, but buying local data would be.
peri-cl · · focus · HN ↗
These (Mistral's and Mozilla's) marketing pages aren't candid enough to clearly explain the difference between local and cloud inference, and that they're asking you to consent to enabling the latter. I'd call that the bare minimum of ethics. If you're in a position of authority over your less technical users, you are ethically obligated to give them the full picture of what are you doing and why they should consent.
(Aside to any Mozilla people who might be reading HN, your page here[0] has an oversight, it advertises this model as "Mistral Small 4" but the hyperlink is to OpenAI's model card for gpt-oss-120b).
[0] <a href="https://support.mozilla.org/en-US/kb/smart-window-models" rel="nofollow">https://support.mozilla.org/en-US/kb/smart-window-models
walrus01 · · focus · HN ↗
But I also think that the state of the art in small LLM and user device capabilities aren't there yet to put a "good enough to be actually useful" local-only LLM as a prepackaged thing in a mass market distributed browser.
You don't want a browser that takes 10GB of extra RAM (on top of the memory hog that is having just 3 or 4 complex tabs open on its own already) and pegs your CPU at 99% usage for minutes at a time. And not in an era when mass market consumer laptops are still commonly 8GB or 16GB of total system RAM. Many of those with integrated-into-CPU onboard graphics (eg: not a gaming laptop with a discrete GPU on the PCI-E bus).
It'll be a catastrophe for laptop battery life, among other resource use problems. And an LLM that fits in under 8 to 10GB of RAM for CPU-only inference is not going to be nearly as capable as an off-device inference system.
I wish they had just done this with a very clear up front opt in (not enabled by default) thing that explains what Mistral is, that it's not some big American cloud company but a relatively small startup in France, and that your prompts/LLM interactions will go to their servers. And some documentation on how it will be handled/stored in a supposedly trustworthy manner.
smsm42 · · focus · HN ↗
How it's any better? Small companies can be bought by big companies. French government is as capable of trampling over their citizen's privacy the moment they feel they need it as US government is. And putting a squeeze on a small startup is way easier than on a major cloud company (not that either is particularly hard). Also, OpenAI used to be an idealistic non-profit one day too, then it started to smell trillions and all that went of of the window.
> is not going to be nearly as capable as an off-device inference system.
I rarely need PhD-level research into my browsing history. I'm not going to solve millennium problems on my bookmarks. The tasks that I will realistically need are well within capacity of most very basic local models. Maybe they'd be a bit slower, who cares.
bluebarbet · · focus · HN ↗
It's better for Europeans because the company in question is European and so will not export their data to New Jersey. And because any success it has will presumably better benefit France and Europe.
smsm42 · · focus · HN ↗
There aren't many government institutions in New Jersey that would take a lot of interest in anything Frenchmen are doing. There are a lot of institutions in France that would be interested in anything Frenchmen are doing, if it contradicts what the French government wants to be happening. The main threat to citizen's privacy always comes from the government closest to them, the government overseas has its own citizens to worry about and pays much less attention to foreign citizens on foreign land. There could be exceptions, true, but as a rule, if you look into New Jersey, you'd sooner find mass surveillance of American citizens than mass surveillance of the French.
Same goes for commercial interests. If I want to run targeted ads in New Jersey, I want to have profiles of New Jersey people, not French people. So buying data from France in New Jersey would not be a routine occurrence, but buying local data would be.