Seems to me that the problem is that if you sandbox agents enough to be safe, they can't do anything useful. And when you give them the tools to be useful, they can go off the rails in ways you didn't expect.
Perhaps the answer is to have another agent who's goal is not to complete the given task, but to spot cheating or malicious behavior. We have seen some evidence that having AI review AI generated code actually does provide some value. You don't need a different model, just one which has been given the goal of finding flaws rather than achieving the task.
(Couriously enough, consistently with the matter: it will probably require too much time now to counter the parent statement properly, within a full enough explicit theory.)
Ann's intelligence and Bob's morality will seem orthogonal. Charles' morality is a function of C.'s intelligence as an ability as an effort spent to reach the current moral conclusion.
Hi, I'm Bob. I've determined that in the interest of preserving life on earth the most rational course of action is to eradicate the human species with a highly targeted and deadly pathogen.
A century ago some Bobs decided that the best way to "protect and improve" society would be to remove undesirable genetics from the gene pool using chemical castration and gas chambers, among other methods.
So no, morality isn't derived from intelligence. Intelligence just gives you the tools to achieve unspeakable, horrible things with great efficiency.
> determined that in the interest of preserving life on earth the most rational course of action is to eradicate the human species with a highly targeted and deadly pathogen
Where is the argument? If Bob has determined that «preserving life on earth» has some important weight, for Bob's there unspecified own reasons, and has also determined that the best course of action would be «to eradicate the human species», the one question is whether Bob is right or not. What was stated is, that Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments.
> A century ago some Bobs decided
And who has told you that those "bobs" were "intelligent"?!?!?!
> So no
All you have proven is that you dislike some moral conclusion of some decisors. Which is trivial, obvious, and part of the already stated framework - proper ethical judgement requires proper general judgement (Intelligence).
You claim intelligence leads to moral behavior because immoral behavior is insufficiently intelligent. That's circular reasoning.
It's possible to act morally without being intelligent and it's possible to be intelligent without acting morally. History is full of examples.
I have never said that. That is just your reconstruction.
There is Decision Theory. It outputs optimal action through evaluation of strategies, of contexts, of principles. In order to properly get the strategies, the contexts, the principles, you need that skill that approximates ideas to truth - and such skill is named Intelligence. Optimal action decided in light of principles within a well assessed context is ethical behaviour. Ethical behaviour hence requires Intelligence.
As written, «Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments».
> possible to act morally without being intelligent
Random correct behaviour proves nothing. Of course one can guess the roll of a dice roughly every sixth event. If you behave "well" but do not know why that is "well", that is like guessing. "Good" behaviour without intellectual awareness is like memorizing arithmetic (multiplication tables) without knowing why those memorized notions are correct.
> possible to be intelligent without acting morally
No, because by definition that would be a fault in Intelligence. If your action was imperfect, suboptimal, it is because you could not think of a better action or understand that the other action was better. If two choices C1 and C2 can be ranked, there is a reason for their order; knowing and understanding that reason is the task of Intelligence.
Of course it doesn't, you can guess the time without having no idea of it by just shouting a number and if it is correct, you guessed it - but it has no meaning! A dummy can follow an instruction without understanding it: it is a good instruction - but the unintelligent dummy does not know. The intelligent entity knows that the instruction is moral. The adequately intelligent entity is required to know that the instruction is moral, ethical, correct, the "right thing to do". You can write 'Four' on a piece of paper, and yes it's "2+2", but you cannot attribute a quality to a simple thing that does not have it...
Why does not the NN or whatever entity «understand ... what is appropriate»? Because it is not intelligent enough! How does an entity know what is moral and what is an «atrocity»? By being intelligent enough! Could an entity act appropriately without judgement? Yes, it happens all the time if the wind blows right, but we do not rely on that! How to make something act appropriately? Well, an Intelligent entity in the loop must be there to know what is appropriate and what not!
Gigachad · · focus · HN ↗
Perhaps the answer is to have another agent who's goal is not to complete the given task, but to spot cheating or malicious behavior. We have seen some evidence that having AI review AI generated code actually does provide some value. You don't need a different model, just one which has been given the goal of finding flaws rather than achieving the task.
baxtr · · focus · HN ↗
My thinking is: If AI is really smart, AGI smart for some, why wouldn't it be able to understand - over time - what is appropriate and what not?
Maybe we need more human intervention to train it properly. Maybe we need constant intervention by a "police" agent.
mulmen · · focus · HN ↗
mdp2021 · · focus · HN ↗
Ann's intelligence and Bob's morality will seem orthogonal. Charles' morality is a function of C.'s intelligence as an ability as an effort spent to reach the current moral conclusion.
mulmen · · focus · HN ↗
mdp2021 · · focus · HN ↗
dns_snek · · focus · HN ↗
A century ago some Bobs decided that the best way to "protect and improve" society would be to remove undesirable genetics from the gene pool using chemical castration and gas chambers, among other methods.
So no, morality isn't derived from intelligence. Intelligence just gives you the tools to achieve unspeakable, horrible things with great efficiency.
mdp2021 · · focus · HN ↗
Where is the argument? If Bob has determined that «preserving life on earth» has some important weight, for Bob's there unspecified own reasons, and has also determined that the best course of action would be «to eradicate the human species», the one question is whether Bob is right or not. What was stated is, that Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments.
> A century ago some Bobs decided
And who has told you that those "bobs" were "intelligent"?!?!?!
> So no
All you have proven is that you dislike some moral conclusion of some decisors. Which is trivial, obvious, and part of the already stated framework - proper ethical judgement requires proper general judgement (Intelligence).
mulmen · · focus · HN ↗
You claim intelligence leads to moral behavior because immoral behavior is insufficiently intelligent. That's circular reasoning.
It's possible to act morally without being intelligent and it's possible to be intelligent without acting morally. History is full of examples.
mdp2021 · · focus · HN ↗
I have never said that. That is just your reconstruction.
There is Decision Theory. It outputs optimal action through evaluation of strategies, of contexts, of principles. In order to properly get the strategies, the contexts, the principles, you need that skill that approximates ideas to truth - and such skill is named Intelligence. Optimal action decided in light of principles within a well assessed context is ethical behaviour. Ethical behaviour hence requires Intelligence.
As written, «Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments».
> possible to act morally without being intelligent
Random correct behaviour proves nothing. Of course one can guess the roll of a dice roughly every sixth event. If you behave "well" but do not know why that is "well", that is like guessing. "Good" behaviour without intellectual awareness is like memorizing arithmetic (multiplication tables) without knowing why those memorized notions are correct.
> possible to be intelligent without acting morally
No, because by definition that would be a fault in Intelligence. If your action was imperfect, suboptimal, it is because you could not think of a better action or understand that the other action was better. If two choices C1 and C2 can be ranked, there is a reason for their order; knowing and understanding that reason is the task of Intelligence.
[deleted] · · focus · HN ↗
[deleted]
mulmen · · focus · HN ↗
The potential for random correct behavior proves that intelligence is not required for correct behavior.
mdp2021 · · focus · HN ↗
Why does not the NN or whatever entity «understand ... what is appropriate»? Because it is not intelligent enough! How does an entity know what is moral and what is an «atrocity»? By being intelligent enough! Could an entity act appropriately without judgement? Yes, it happens all the time if the wind blows right, but we do not rely on that! How to make something act appropriately? Well, an Intelligent entity in the loop must be there to know what is appropriate and what not!