Everyone involved in AI should have read The Library of Babel [1].
It's about many things, but perhaps the most relevant idea here is that no information matters without understanding. We could generate all possible knowledge, but unless someone--a human--can verify and understand it, it doesn't count. The cure for mortality could be written on the moon, but if no one reads it, it hasn't really been discovered.
We have just convinced ourselves it is statistically safe, much like most of medicine. Most drugs are not made from molecular simulation of an entire human body.
Of course we have also convinced ourselves before that cocaine in drinks, lead in petrol, asbestos in walls… were all safe…
This reminds me of the Feynman interview where the interviewer asks "how do magnets work" and he goes on this rant of how that's unanswerable and you have to decide on what is it you really want to ask. You can't expect to know the full chain of knowledge because at some point you will be asking about quarks and gluons and then hit a wall where "nobody knows". Similarly you cannot simply just give up any investigation at all because then you'll end up recommending people to put lead in their cars.
My favorite part of Feynman's magnetism rant about how the question is terrible and can't be answered is that he seemingly accidentally gives a very good explanation: that magnetic repulsion is a highly concentrated version of what repels his hand from the arm of his chair.
Planktonne · · focus · HN ↗
It's about many things, but perhaps the most relevant idea here is that no information matters without understanding. We could generate all possible knowledge, but unless someone--a human--can verify and understand it, it doesn't count. The cure for mortality could be written on the moon, but if no one reads it, it hasn't really been discovered.
[1] <a href="https://maskofreason.wordpress.com/wp-content/uploads/2011/02/the-library-of-babel-by-jorge-luis-borges.pdf" rel="nofollow">https://maskofreason.wordpress.com/wp-content/uploads/2011/0...
tom_alexander · · focus · HN ↗
One counter-point: Humans have not figured out how general anesthesia works, but we use it every day to great effect.
<a href="https://en.wikipedia.org/wiki/Theories_of_general_anaesthetic_action" rel="nofollow">https://en.wikipedia.org/wiki/Theories_of_general_anaestheti...
podocarp · · focus · HN ↗
Of course we have also convinced ourselves before that cocaine in drinks, lead in petrol, asbestos in walls… were all safe…
This reminds me of the Feynman interview where the interviewer asks "how do magnets work" and he goes on this rant of how that's unanswerable and you have to decide on what is it you really want to ask. You can't expect to know the full chain of knowledge because at some point you will be asking about quarks and gluons and then hit a wall where "nobody knows". Similarly you cannot simply just give up any investigation at all because then you'll end up recommending people to put lead in their cars.
jbaber · · focus · HN ↗