The problem imo is the slow deterioration of institutional knowledge that offloading the mental task of wisdom gathering to AI is causing.
One interesting comparison is to the history of manufacturing. West/America decided one day that manufacturing would be cheaper to outsource and better (short term) profit was to be made by outsourcing it all to China. The institutional expertise started to deteriorate, to the point that America simply didn't even have the capacity, or expertise anymore to produce stuff (such as grill brush [1])
I feel like you could take all the handwavy comment that are made today to dismiss this caution, and find equal dismissal back then when companies were actively outsourcing the manufacturing.
"I'm coding 10x faster"
"look at the output velocity per employee"
"we are producing much more (in China)"
"look at profit / number of (manufacturing) employers"
Seems ok if you're American / Chinese but I'm struggling to understand how the rest can be OK with allowing institutional knowledge to deteriorate while having an active dependency to the former two. We already see this with the tech dependency towards USA and manufacturing competition from China.
> West/America decided one day that manufacturing would be cheaper to outsource and better (short term) profit was to be made by outsourcing it all to China. The institutional expertise started to deteriorate, to the point that America simply didn't even have the capacity, or expertise anymore to produce stuff
Americans produce the highest technology equipment in the world. Our machining base is structurally sound, but its all making weapons, so you don't hear about it.
It's true that China benefitted immensely from outsourcing, but they took the jobs Americans didn't want. It's the same with immigration today - folks cross the Rio Grande to do chores that Americans won't, or others fly in and work for nothing in academia while they wait for their PhD.
> The problem imo is the slow deterioration of institutional knowledge that offloading the mental task of wisdom gathering to AI is causing.
Have you considered the institutional knowledge could actually be actively preserved and distributed with AI? The kind of tacit knowledge that is situated and not readily preserved in a book might be absorbed by thinking machines and proliferated to the next person who needs it. The caveats would be trade secrets, skill differentiators, that people might not be willing to discuss, and manufacturing secrets of national importance. Maybe you can think of others.
>Have you considered the institutional knowledge could actually be actively preserved and distributed with AI?
Well that's what this whole debate comes down to. And, to my mind at least, it's a rare case of an actually interesting question about AI, because like the article says, it can plausibly deteriorate exactly that kind of knowledge. But as you note, it can also maintain it.
I think there's a sense in which it might do both at the same time. AI's version of on call tacit knowledge might be something like lazy-loading just-in-time tacit knowledge, but at the cost of who knows what cognitive paths we might have by keeping that knowledge resting in-house. We would gain real efficiency but we wouldn't know we wouldn't know.
>The kind of tacit knowledge that is situated and not readily preserved in a book might be absorbed by thinking machines and proliferated to the next person who needs it. The caveats would be trade secrets, skill differentiators, that people might not be willing to discuss, and manufacturing secrets of national importance. Maybe you can think of others.
It's funny that you credit AI with this (and I don't disagree), because tacit knowledge was exactly the thing Hubert Dreyfus spent a career insisting computers would never have, and his wisdom was taught to generations of undergraduates across the country and world who treated it as received wisdom and still is regarded as such in certain academic corners.
I edited that sentence to try and clarify a bit, but Dreyfus was a continental philosopher out of Berkley active basically from the 1950s to the 2010s. He engaged in decades of friendly debates with other philosophers about the limits of computers.
When he described those limits, he always frustrated computer scientists and analytic philosophers because he spoke in a kind of informal philosophical vocabulary and didn't really formalize his ideas. So he would say computers didn't have things like "tacit knowledge" or "insight" and he railed against "symbol manipulation". Famously he declared chess would never surpass human expert play. He wrote a book called "What Computers Can't Do" and another called "What Computer's Still Can't Do".
I personally think his argument was laughably wrong if well intentioned. But some people think it was respectable. I think in the present day, he's often rehabilitated with a kind of apologetic reinterpretation, such that things like transformers, weights, vectors, etc were what he really meant all along.
I think he was not wrong that some higher layer of sophistication would prove to be necessary, but he was wrong, I think definitively, to think that "symbol manipulation" of computers was a kind of category error. Even today's best models are still running on logic gates over 1's and 0's, and it was his failure of imagination to doubt that those could be the conceptual bedrock for AI, tacit knowledge and all.
He passed away in 2017, which is too bad because I would have loved to have seen his interpretation of things like GPT-6 Astra.
> think that "symbol manipulation" of computers was a kind of category error
This is where we do drift off into the semantic bog. If there is tacit knowledge in an LLM, then where is it? It must be in the weights, and it must have somehow come from the training data. Therefore the weights represent "compressed" knowledge. Is that then not "tacit" since it's explicitly encoded?
You'll get no disagreement from me here. But be ready because some people live to debate this, and Dreyfus is the poster child for a whole philosophical tradition.
I think what it comes down to, is trying to turn the specialness of human intelligence into undefinable magic, essentially playing god of the gaps with the concept of human insight. So by design, it has to be something that can't be amenable to any formal representation like weights.
> Famously he declared chess would never surpass human expert play. He wrote a book called "What Computers Can't Do" and another called "What Computer's Still Can't Do".
He never made such a claim. In the introduction to the 1972 print of this book he discusses the forecasts from Turing to his time of computers' abilities to play chess and the then state of the art. He criticizes the early optimism in 1950s mentioning that in 1957 H. Simon though in 10 years computers would excel in chess.
Dreyfus goes on to discuss the history of forecasts and progress in computer chess in a nuanced and highly informative analysis.
Whatever your opinion of his work I don't think it's fair to say his arguments are "laughably wrong" at any turn. I haven't read his books in detail but from what I know he made great contributions to the dialogue about technology and I don't know any instance of his making crass predictions or anything that he wrote that could be labelled "laughable".
Dreyfus certainly claims he was mischaracterized, but what he "really" said always turns out to be a conveniently moving target.
He thought that the complexity of chess rendered it solvable in principle but "uncomputable" in practice. You're right that he was speaking to the times he was familiar with, and what he meant by "impossible in practice" was something like letting a 1Mhz computer explore all the possible chess moves from now until the heat death of the universe. Relying on that to insist that computers defeating humans in his lifetime was consistent with what he envisioned stretches past charitability and into sophistry.
And I don't think you can do that without also extending the same charity toward proponents of computer intelligence he was critiquing, and if you do that his thesis, which was disproportionately preoccupied with retelling the failures of the 1950s over and over again using them to represent the whole of computing while the world moved on, because the same charitable repairs extended the other way make AI into something less easily caricatured, and still based on the same logic gates and 0s and 1s he was criticizing.
If that's not enough, Dreyfus explicitly said that what was lacking in chess programs was (1) any practical ability to do the brute forcing needed, (2) any kind of nim-style logical shortcuts around brute forcing or (3) any kind of expert level heuristics because he categorically believed those simply weren't programmable. And he believed that those exhausted the options. [1]
There's no version of this that can be correct because even if you think he's right that chess engines got better by progressing to some different conceptual paradigm, that paradigm is still embodied in same logic gates and 1's and 0's that he thought only pertained to prior paradigms he was criticizing. He was wrong to assume such things as "heuristics" were out of that scope.
NalNezumi · · focus · HN ↗
One interesting comparison is to the history of manufacturing. West/America decided one day that manufacturing would be cheaper to outsource and better (short term) profit was to be made by outsourcing it all to China. The institutional expertise started to deteriorate, to the point that America simply didn't even have the capacity, or expertise anymore to produce stuff (such as grill brush [1])
I feel like you could take all the handwavy comment that are made today to dismiss this caution, and find equal dismissal back then when companies were actively outsourcing the manufacturing.
"I'm coding 10x faster" "look at the output velocity per employee"
"we are producing much more (in China)" "look at profit / number of (manufacturing) employers"
Seems ok if you're American / Chinese but I'm struggling to understand how the rest can be OK with allowing institutional knowledge to deteriorate while having an active dependency to the former two. We already see this with the tech dependency towards USA and manufacturing competition from China.
[1] <a href="https://youtu.be/3ZTGwcHQfLY" rel="nofollow">https://youtu.be/3ZTGwcHQfLY
engineer_22 · · focus · HN ↗
Americans produce the highest technology equipment in the world. Our machining base is structurally sound, but its all making weapons, so you don't hear about it.
It's true that China benefitted immensely from outsourcing, but they took the jobs Americans didn't want. It's the same with immigration today - folks cross the Rio Grande to do chores that Americans won't, or others fly in and work for nothing in academia while they wait for their PhD.
> The problem imo is the slow deterioration of institutional knowledge that offloading the mental task of wisdom gathering to AI is causing.
Have you considered the institutional knowledge could actually be actively preserved and distributed with AI? The kind of tacit knowledge that is situated and not readily preserved in a book might be absorbed by thinking machines and proliferated to the next person who needs it. The caveats would be trade secrets, skill differentiators, that people might not be willing to discuss, and manufacturing secrets of national importance. Maybe you can think of others.
glenstein · · focus · HN ↗
Well that's what this whole debate comes down to. And, to my mind at least, it's a rare case of an actually interesting question about AI, because like the article says, it can plausibly deteriorate exactly that kind of knowledge. But as you note, it can also maintain it.
I think there's a sense in which it might do both at the same time. AI's version of on call tacit knowledge might be something like lazy-loading just-in-time tacit knowledge, but at the cost of who knows what cognitive paths we might have by keeping that knowledge resting in-house. We would gain real efficiency but we wouldn't know we wouldn't know.
>The kind of tacit knowledge that is situated and not readily preserved in a book might be absorbed by thinking machines and proliferated to the next person who needs it. The caveats would be trade secrets, skill differentiators, that people might not be willing to discuss, and manufacturing secrets of national importance. Maybe you can think of others.
It's funny that you credit AI with this (and I don't disagree), because tacit knowledge was exactly the thing Hubert Dreyfus spent a career insisting computers would never have, and his wisdom was taught to generations of undergraduates across the country and world who treated it as received wisdom and still is regarded as such in certain academic corners.
pjc50 · · focus · HN ↗
Could you elaborate on this? It sounds like something any anthropologist would laugh at; there's always an oral culture.
glenstein · · focus · HN ↗
When he described those limits, he always frustrated computer scientists and analytic philosophers because he spoke in a kind of informal philosophical vocabulary and didn't really formalize his ideas. So he would say computers didn't have things like "tacit knowledge" or "insight" and he railed against "symbol manipulation". Famously he declared chess would never surpass human expert play. He wrote a book called "What Computers Can't Do" and another called "What Computer's Still Can't Do".
I personally think his argument was laughably wrong if well intentioned. But some people think it was respectable. I think in the present day, he's often rehabilitated with a kind of apologetic reinterpretation, such that things like transformers, weights, vectors, etc were what he really meant all along.
I think he was not wrong that some higher layer of sophistication would prove to be necessary, but he was wrong, I think definitively, to think that "symbol manipulation" of computers was a kind of category error. Even today's best models are still running on logic gates over 1's and 0's, and it was his failure of imagination to doubt that those could be the conceptual bedrock for AI, tacit knowledge and all.
He passed away in 2017, which is too bad because I would have loved to have seen his interpretation of things like GPT-6 Astra.
pjc50 · · focus · HN ↗
> think that "symbol manipulation" of computers was a kind of category error
This is where we do drift off into the semantic bog. If there is tacit knowledge in an LLM, then where is it? It must be in the weights, and it must have somehow come from the training data. Therefore the weights represent "compressed" knowledge. Is that then not "tacit" since it's explicitly encoded?
glenstein · · focus · HN ↗
I think what it comes down to, is trying to turn the specialness of human intelligence into undefinable magic, essentially playing god of the gaps with the concept of human insight. So by design, it has to be something that can't be amenable to any formal representation like weights.
huurtehoog · · focus · HN ↗
He never made such a claim. In the introduction to the 1972 print of this book he discusses the forecasts from Turing to his time of computers' abilities to play chess and the then state of the art. He criticizes the early optimism in 1950s mentioning that in 1957 H. Simon though in 10 years computers would excel in chess.
Dreyfus goes on to discuss the history of forecasts and progress in computer chess in a nuanced and highly informative analysis.
Whatever your opinion of his work I don't think it's fair to say his arguments are "laughably wrong" at any turn. I haven't read his books in detail but from what I know he made great contributions to the dialogue about technology and I don't know any instance of his making crass predictions or anything that he wrote that could be labelled "laughable".
glenstein · · focus · HN ↗
He thought that the complexity of chess rendered it solvable in principle but "uncomputable" in practice. You're right that he was speaking to the times he was familiar with, and what he meant by "impossible in practice" was something like letting a 1Mhz computer explore all the possible chess moves from now until the heat death of the universe. Relying on that to insist that computers defeating humans in his lifetime was consistent with what he envisioned stretches past charitability and into sophistry.
And I don't think you can do that without also extending the same charity toward proponents of computer intelligence he was critiquing, and if you do that his thesis, which was disproportionately preoccupied with retelling the failures of the 1950s over and over again using them to represent the whole of computing while the world moved on, because the same charitable repairs extended the other way make AI into something less easily caricatured, and still based on the same logic gates and 0s and 1s he was criticizing.
If that's not enough, Dreyfus explicitly said that what was lacking in chess programs was (1) any practical ability to do the brute forcing needed, (2) any kind of nim-style logical shortcuts around brute forcing or (3) any kind of expert level heuristics because he categorically believed those simply weren't programmable. And he believed that those exhausted the options. [1]
There's no version of this that can be correct because even if you think he's right that chess engines got better by progressing to some different conceptual paradigm, that paradigm is still embodied in same logic gates and 1's and 0's that he thought only pertained to prior paradigms he was criticizing. He was wrong to assume such things as "heuristics" were out of that scope.
1. <a href="https://repository.essex.ac.uk/42372/1/Martin%20and%20Williams%202025%20What%20ChatGP%20accepted%20version.pdf" rel="nofollow">https://repository.essex.ac.uk/42372/1/Martin%20and%20Willia...
huurtehoog · · focus · HN ↗
I do object to calling his writing "laughable".
Thank you for the article, it seems quite interesting on skimming and I will save it for later.