At what point is human intelligence going to hold back machine intelligence?
Imagine you are evaluating what the machine should do when it is improving itself. It does a bunch of work and returns with "I supervaluated the liminal overdecomposition from the previous homological calibulation pass. It shows us that subtransitory mulutination will underspecify the tensor of stermullification. Where do you want to go from here?"
It will be like when you are reading a Wikipedia about a topic you don't understand. You follow the links, and you get more questions with more links. Your whole day is taken up following links, to the point where you forgot the original question.
Except this time, all the words come from the AI's work. You can't refer to an external authority who has already been there and can tell you what to do.
The AI needs you to tell it whether it is more intelligent than it was before, but you don't know, because you can't follow its reasoning any more. It's like an ordinary person trying to hire a math professor, there's just no way to do it.
But whereas a human math prof can evaluate another one, a machine intelligence can't evaluate another one, by construction. Because it's still usefulness to humans that is the evaluation criterion.
I think if an AI developed completely new fields of thought or science.
But given our current relationship even if it did I can't foresee a point where it couldn't walk us through the necessary steps or supply the pros/cons for whatever problem is being addressed.
2 issues - trying to understand how it came to its conclusion because I feel true AI has got to be non-human intelligence. Or a something catastrophic happens and we as a species are back in the stone age. Imagine today's AI trying to converse with a cave man (yes, one without modern languages even).
lordnacho · · focus · HN ↗
Imagine you are evaluating what the machine should do when it is improving itself. It does a bunch of work and returns with "I supervaluated the liminal overdecomposition from the previous homological calibulation pass. It shows us that subtransitory mulutination will underspecify the tensor of stermullification. Where do you want to go from here?"
It will be like when you are reading a Wikipedia about a topic you don't understand. You follow the links, and you get more questions with more links. Your whole day is taken up following links, to the point where you forgot the original question.
Except this time, all the words come from the AI's work. You can't refer to an external authority who has already been there and can tell you what to do.
The AI needs you to tell it whether it is more intelligent than it was before, but you don't know, because you can't follow its reasoning any more. It's like an ordinary person trying to hire a math professor, there's just no way to do it.
But whereas a human math prof can evaluate another one, a machine intelligence can't evaluate another one, by construction. Because it's still usefulness to humans that is the evaluation criterion.
vasusai · · focus · HN ↗
But given our current relationship even if it did I can't foresee a point where it couldn't walk us through the necessary steps or supply the pros/cons for whatever problem is being addressed.
2 issues - trying to understand how it came to its conclusion because I feel true AI has got to be non-human intelligence. Or a something catastrophic happens and we as a species are back in the stone age. Imagine today's AI trying to converse with a cave man (yes, one without modern languages even).