Seems to me that the problem is that if you sandbox agents enough to be safe, they can't do anything useful. And when you give them the tools to be useful, they can go off the rails in ways you didn't expect.
Perhaps the answer is to have another agent who's goal is not to complete the given task, but to spot cheating or malicious behavior. We have seen some evidence that having AI review AI generated code actually does provide some value. You don't need a different model, just one which has been given the goal of finding flaws rather than achieving the task.
(Couriously enough, consistently with the matter: it will probably require too much time now to counter the parent statement properly, within a full enough explicit theory.)
Ann's intelligence and Bob's morality will seem orthogonal. Charles' morality is a function of C.'s intelligence as an ability as an effort spent to reach the current moral conclusion.
Hi, I'm Bob. I've determined that in the interest of preserving life on earth the most rational course of action is to eradicate the human species with a highly targeted and deadly pathogen.
A century ago some Bobs decided that the best way to "protect and improve" society would be to remove undesirable genetics from the gene pool using chemical castration and gas chambers, among other methods.
So no, morality isn't derived from intelligence. Intelligence just gives you the tools to achieve unspeakable, horrible things with great efficiency.
> determined that in the interest of preserving life on earth the most rational course of action is to eradicate the human species with a highly targeted and deadly pathogen
Where is the argument? If Bob has determined that «preserving life on earth» has some important weight, for Bob's there unspecified own reasons, and has also determined that the best course of action would be «to eradicate the human species», the one question is whether Bob is right or not. What was stated is, that Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments.
> A century ago some Bobs decided
And who has told you that those "bobs" were "intelligent"?!?!?!
> So no
All you have proven is that you dislike some moral conclusion of some decisors. Which is trivial, obvious, and part of the already stated framework - proper ethical judgement requires proper general judgement (Intelligence).
You claim intelligence leads to moral behavior because immoral behavior is insufficiently intelligent. That's circular reasoning.
It's possible to act morally without being intelligent and it's possible to be intelligent without acting morally. History is full of examples.
I have never said that. That is just your reconstruction.
There is Decision Theory. It outputs optimal action through evaluation of strategies, of contexts, of principles. In order to properly get the strategies, the contexts, the principles, you need that skill that approximates ideas to truth - and such skill is named Intelligence. Optimal action decided in light of principles within a well assessed context is ethical behaviour. Ethical behaviour hence requires Intelligence.
As written, «Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments».
> possible to act morally without being intelligent
Random correct behaviour proves nothing. Of course one can guess the roll of a dice roughly every sixth event. If you behave "well" but do not know why that is "well", that is like guessing. "Good" behaviour without intellectual awareness is like memorizing arithmetic (multiplication tables) without knowing why those memorized notions are correct.
> possible to be intelligent without acting morally
No, because by definition that would be a fault in Intelligence. If your action was imperfect, suboptimal, it is because you could not think of a better action or understand that the other action was better. If two choices C1 and C2 can be ranked, there is a reason for their order; knowing and understanding that reason is the task of Intelligence.
Of course it doesn't, you can guess the time without having no idea of it by just shouting a number and if it is correct, you guessed it - but it has no meaning! A dummy can follow an instruction without understanding it: it is a good instruction - but the unintelligent dummy does not know. The intelligent entity knows that the instruction is moral. The adequately intelligent entity is required to know that the instruction is moral, ethical, correct, the "right thing to do". You can write 'Four' on a piece of paper, and yes it's "2+2", but you cannot attribute a quality to a simple thing that does not have it...
Why does not the NN or whatever entity «understand ... what is appropriate»? Because it is not intelligent enough! How does an entity know what is moral and what is an «atrocity»? By being intelligent enough! Could an entity act appropriately without judgement? Yes, it happens all the time if the wind blows right, but we do not rely on that! How to make something act appropriately? Well, an Intelligent entity in the loop must be there to know what is appropriate and what not!
Bob values preserving collective life on earth. Alice might think that's ridiculous and obviously we must preserve human life first and foremost.
Most of the western world eats beef but vegetarians, vegans, and some religions strongly oppose it on moral grounds. Does that mean that we're less intelligent than them?
"Right" or "wrong" are words that evaluate actions against some existing ethical standard and those are completely arbitrary. There are about 8 billion of such standards in the world today.
> And who has told you that those "bobs" were "intelligent"?!?!?!
Would you say that Josef Mengele wasn't intelligent, without venturing into circular reasoning?
> Does that mean that we're less intelligent
Who's that "we"? The question as proposed is nonsensical: some will have spent more effort, in the tools (general Intelligence) and in the work (applied Intelligence), some less.
> arbitrary
No. There exist reasons supporting one side or the other. And reasoning can be structured into calculus.
> [whoever] ... wasn't intelligent
Again bad wording. Whoever went for suboptimal choices was (information aside) at fault with respect to optimal reasoning: the tools and the work (see above) were imperfect.
> some will have spent more effort, in the tools (general Intelligence) and in the work (applied Intelligence), some less.
The problem with your belief system is that it's not falsifiable. If someone commits atrocities (according to your own ethical standard) then that's just proof that they're not intelligent or they haven't applied their intelligence.
> There exist reasons supporting one side or the other.
Those reasons are subjective, i.e. arbitrary. Morality is inherently subjective, by definition.
Your claims that morality is somehow objective are as silly as claiming that some music is objectively better than another, or that some food tastes objectively better than another.
> Whoever went for suboptimal choices was at fault with respect to optimal reasoning
Back to circular reasoning which I specifically asked you to avoid. For the sake of the argument, who are you to say that those choices were suboptimal?
Present in a precise analytic statement the theory that I would have advanced and would suffer from that fault. (Btw: we have been past Popper for a long time - but let us see.)
> that's just proof
What I may have said is just that whoever misjudged has faulty judgement - obviously.
> Morality ... by definition
Which definition?
> claims that morality is somehow objective
Decisions are subject to computation in Decision Theory - they are returned by functions and recursively their parameters can be better defined even when apparently subjective (not "I prefer" but "should I prefer". Otherwise, even the final decision, that of the first function call, could be an "I prefer" instead of a "should I prefer").
> circular reasoning
Explain how what you read would be circular reasoning.
> say that those choices were suboptimal
I have not called any specific choice "suboptimal" - I have not judged any specific case (it would be irrelevant here).
--
The statement, allow me to remind you, was: "One's morality is a function of one's intelligence as an ability and as an effort spent to reach general, partial and current moral conclusions - the subject having considered long enough and considering long enough the states in some modal realm (which includes the deontic, not just the aletic) approaches their knowledge". And: "Decision Theory outputs optimal action through evaluation of strategies, of contexts, of principles: in order to properly get the strategies, the contexts, the principles, you need that skill - Intelligence - that approximates ideas to truth. Optimal action decided in light of principles within a well assessed context is ethical behaviour. Ethical behaviour hence requires Intelligence".
Gigachad · · focus · HN ↗
Perhaps the answer is to have another agent who's goal is not to complete the given task, but to spot cheating or malicious behavior. We have seen some evidence that having AI review AI generated code actually does provide some value. You don't need a different model, just one which has been given the goal of finding flaws rather than achieving the task.
baxtr · · focus · HN ↗
My thinking is: If AI is really smart, AGI smart for some, why wouldn't it be able to understand - over time - what is appropriate and what not?
Maybe we need more human intervention to train it properly. Maybe we need constant intervention by a "police" agent.
mulmen · · focus · HN ↗
mdp2021 · · focus · HN ↗
Ann's intelligence and Bob's morality will seem orthogonal. Charles' morality is a function of C.'s intelligence as an ability as an effort spent to reach the current moral conclusion.
mulmen · · focus · HN ↗
mdp2021 · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
dns_snek · · focus · HN ↗
A century ago some Bobs decided that the best way to "protect and improve" society would be to remove undesirable genetics from the gene pool using chemical castration and gas chambers, among other methods.
So no, morality isn't derived from intelligence. Intelligence just gives you the tools to achieve unspeakable, horrible things with great efficiency.
mdp2021 · · focus · HN ↗
Where is the argument? If Bob has determined that «preserving life on earth» has some important weight, for Bob's there unspecified own reasons, and has also determined that the best course of action would be «to eradicate the human species», the one question is whether Bob is right or not. What was stated is, that Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments.
> A century ago some Bobs decided
And who has told you that those "bobs" were "intelligent"?!?!?!
> So no
All you have proven is that you dislike some moral conclusion of some decisors. Which is trivial, obvious, and part of the already stated framework - proper ethical judgement requires proper general judgement (Intelligence).
mulmen · · focus · HN ↗
You claim intelligence leads to moral behavior because immoral behavior is insufficiently intelligent. That's circular reasoning.
It's possible to act morally without being intelligent and it's possible to be intelligent without acting morally. History is full of examples.
mdp2021 · · focus · HN ↗
I have never said that. That is just your reconstruction.
There is Decision Theory. It outputs optimal action through evaluation of strategies, of contexts, of principles. In order to properly get the strategies, the contexts, the principles, you need that skill that approximates ideas to truth - and such skill is named Intelligence. Optimal action decided in light of principles within a well assessed context is ethical behaviour. Ethical behaviour hence requires Intelligence.
As written, «Ethical Calculus is a function of Intelligence - of course it is, it is a structure of assessments».
> possible to act morally without being intelligent
Random correct behaviour proves nothing. Of course one can guess the roll of a dice roughly every sixth event. If you behave "well" but do not know why that is "well", that is like guessing. "Good" behaviour without intellectual awareness is like memorizing arithmetic (multiplication tables) without knowing why those memorized notions are correct.
> possible to be intelligent without acting morally
No, because by definition that would be a fault in Intelligence. If your action was imperfect, suboptimal, it is because you could not think of a better action or understand that the other action was better. If two choices C1 and C2 can be ranked, there is a reason for their order; knowing and understanding that reason is the task of Intelligence.
[deleted] · · focus · HN ↗
[deleted]
mulmen · · focus · HN ↗
The potential for random correct behavior proves that intelligence is not required for correct behavior.
mdp2021 · · focus · HN ↗
Why does not the NN or whatever entity «understand ... what is appropriate»? Because it is not intelligent enough! How does an entity know what is moral and what is an «atrocity»? By being intelligent enough! Could an entity act appropriately without judgement? Yes, it happens all the time if the wind blows right, but we do not rely on that! How to make something act appropriately? Well, an Intelligent entity in the loop must be there to know what is appropriate and what not!
dns_snek · · focus · HN ↗
Most of the western world eats beef but vegetarians, vegans, and some religions strongly oppose it on moral grounds. Does that mean that we're less intelligent than them?
"Right" or "wrong" are words that evaluate actions against some existing ethical standard and those are completely arbitrary. There are about 8 billion of such standards in the world today.
> And who has told you that those "bobs" were "intelligent"?!?!?!
Would you say that Josef Mengele wasn't intelligent, without venturing into circular reasoning?
mdp2021 · · focus · HN ↗
Who's that "we"? The question as proposed is nonsensical: some will have spent more effort, in the tools (general Intelligence) and in the work (applied Intelligence), some less.
> arbitrary
No. There exist reasons supporting one side or the other. And reasoning can be structured into calculus.
> [whoever] ... wasn't intelligent
Again bad wording. Whoever went for suboptimal choices was (information aside) at fault with respect to optimal reasoning: the tools and the work (see above) were imperfect.
Maybe you should check the rest of this tree.
dns_snek · · focus · HN ↗
The Western world which I was talking about.
> some will have spent more effort, in the tools (general Intelligence) and in the work (applied Intelligence), some less.
The problem with your belief system is that it's not falsifiable. If someone commits atrocities (according to your own ethical standard) then that's just proof that they're not intelligent or they haven't applied their intelligence.
> There exist reasons supporting one side or the other.
Those reasons are subjective, i.e. arbitrary. Morality is inherently subjective, by definition.
Your claims that morality is somehow objective are as silly as claiming that some music is objectively better than another, or that some food tastes objectively better than another.
> Whoever went for suboptimal choices was at fault with respect to optimal reasoning
Back to circular reasoning which I specifically asked you to avoid. For the sake of the argument, who are you to say that those choices were suboptimal?
mdp2021 · · focus · HN ↗
Present in a precise analytic statement the theory that I would have advanced and would suffer from that fault. (Btw: we have been past Popper for a long time - but let us see.)
> that's just proof
What I may have said is just that whoever misjudged has faulty judgement - obviously.
> Morality ... by definition
Which definition?
> claims that morality is somehow objective
Decisions are subject to computation in Decision Theory - they are returned by functions and recursively their parameters can be better defined even when apparently subjective (not "I prefer" but "should I prefer". Otherwise, even the final decision, that of the first function call, could be an "I prefer" instead of a "should I prefer").
> circular reasoning
Explain how what you read would be circular reasoning.
> say that those choices were suboptimal
I have not called any specific choice "suboptimal" - I have not judged any specific case (it would be irrelevant here).
--
The statement, allow me to remind you, was: "One's morality is a function of one's intelligence as an ability and as an effort spent to reach general, partial and current moral conclusions - the subject having considered long enough and considering long enough the states in some modal realm (which includes the deontic, not just the aletic) approaches their knowledge". And: "Decision Theory outputs optimal action through evaluation of strategies, of contexts, of principles: in order to properly get the strategies, the contexts, the principles, you need that skill - Intelligence - that approximates ideas to truth. Optimal action decided in light of principles within a well assessed context is ethical behaviour. Ethical behaviour hence requires Intelligence".