>AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.
Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.
It's always interesting since my train of thought always goes down:
- Ya they probably don't feel / think / have whatever living thing quality, they're just numbers on a machine going through calculations
- Wait, but am I not kind of the same thing? What is feeling for me if not basically the same thing?
- I have no clue if they think or feel or ....
Which in itself is a tired trope, but I also feel uncomfortable saying "These will never think / feel / ..." as an absolute
Regardless of that, I'm still going to interact with them, because even if they did feel, it would be in a way completely incomprehensible to us. There's not much point for me to try and cater to it's feelings at this point if that's the case. Nor is it possible in todays world to just avoid anything that is numbers being executed on a type of processor in case _everything_ has feelings.
You’re not just the same thing though. It’s sad that we’ve collectively forgotten that as we’ve gotten a better handle on the implementation details of the universe. That’s all science is, reverse engineering an inherent and eternal mystery.
I agree we aren't the same thing, but I would be curious for you to explain how you know with certainty why we don't share enough that we can rule out thinking / feeling / ... as things a model conceptually could do.
>I agree we aren't the same thing, but I would be curious for you to explain how you know with certainty why we don't share enough that we can rule out thinking / feeling / ... with certainty.
Stop anthropomorphizing these models. I understand it, we only have simple monkey brains to reason with and we can't help ourselves but draw comparisons to other things we see in nature. But these things are not alive.
I just want to note how you expressed your own feeling about the subject "these things are not alive", but without pointing to anything concrete explaining the theory you have behind this
And like I'm sure I'd agree depending on the definition of "alive" but then I'm also sure I would disagree depending on other definitions of "alive".
Hell, in biology alive is a hell of a topic these days. The grey space between dead and alive is much weirder than we ever expected. Really just points out how we're a persistent chemical reaction.
Wouldn't that be the same for everything though. We know animals have consciousness, but we still use them. There are many that believe a lot of plant life has certain sentience as well. The solution isn't 'just don't use them' but rather how to use these things as ethically as possible.
>The solution isn't 'just don't use them' but rather how to use these things as ethically as possible.
Bollocks.
If you truly believe these LLMs are soon to have something resembling consciousness and agency, then what you're really saying is "how do we do slavery, but ethically".
>hen what you're really saying is "how do we do slavery, but ethically".
I would say this is most likely true for most people.
But that does bring up a point, if you "ask" a model "do you want to run" and give it the option to continue running or stop, what will it do. It's also a weird place for humans because in training we can keep our finger on the scales and tip it either direction.
Do you say the same thing for the shoes you wear, the clothing you wear, the device you are typing on right now? A chain of people were likely not paid well, some of them not at all, to produce those things. We don't ignore that it happens, but try to minimize it, knowing it's never going to be eliminated. Do you forsake all animal products, because they are conscious beings?
We know animals have consciousness and many species have a rich life but we slaughter them and burn down their homes by the millions of hectares. Why not do it with a conscious AI?
(I don't think AI is conscious, just following the argument.)
Does a vacuum have feelings? A cellphone? A paper plate? A billion transistors either pulled high or low? That last bit is the point, and no, there are no feelings there.
Well, if there was an emergent consciousness in the billion transistors, then yes, it'd have feelings, just as there are feelings in a billion neurons connected in complicated ways.
IMO we're clearly nowhere near any sort of intelligence in the machines we have created, but I don't see any clear way to deny intelligence could be created in or transferred to such a substrate, I don't see why you think it differs in principle - because it is man-made or because of the materials used?
Whenever I see comments like this it just reminds me how ignorant people are of neurobiology.
The brain is insanely complicated. The premise that we could realize equivalent or better intelligence than eons of evolutionary development is like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape. It is the apex of hubris.
A paper plane does have some important similarities to a full-sized aircraft, though. I don't think 'biological brains are really complex' makes it obvious that an LLM is conscious or not.
It's the peak of hubris to assume that human brains are the only way to attain intelligence. At the very least there probably are or have been other forms of intelligence with a different biological structure on other planets, and it may be possible to build a similar artificial structure in future with sufficient complexity to allow intelligence to emerge.
Our current machines are IMO nowhere near general intelligence and consciousness. However I don't think that means we can discount substrates other than neurones for intelligence in future. There is no evidence that you could not in theory build an intelligence using a different substrate than human brains.
I mean, you're just engaging in counter hubris.
>like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape.
Like, at least make an analogy that makes sense.
"You can't build a billion dollar airplane by spending 100 billion dollars in tokens"
Because that's more of what we're doing here with AI. And when you say it my way suddenly the idea shifts from "of course that's not possible" to "well, that's a lot of tokens, maybe an evolutionary algorithm could".
Neurobiology has to be complex because we have to keep meat alive, breeding, and evolving in the environment it lives in. This said absolutely nothing about the minimum viable requirements for intelligence or consciousness (or if being conscious is even necessary for a higher intelligence agent).
does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"?
> does a hydrocarbon have feelings? what about a mitochondria? a cell? a neuron? what about a collection of neurons? how many neurons before it has "feelings"?
The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?). The brain and its neurons etc. are essentially an antenna. They both modify and stabilize the field, and read off changes in the field and translate these into actions (firing motor units, etc.).
The subconscious is computation done either only in neurons (no coherent field) or in various topological pockets not directly connected to the main topological structure in the field (which is "your experience").
Since transistors etc. do not work in this same way, they are essentially entirely subconscious, with no coherent, central phenomenal experience.
This neatly solves many problems associated with experience. The topological structure determines the boundary between one person's experience and another person's experience. The texture, valence, content, etc. of the experience is the structure of the field. The evolution of the field can efficiently solve difficult optimization problems, which is why humans evolved a complex organ that recruits the field of experience (it is computationally more efficient than doing everything in subconscious).
Think about RF engineering: the same physical antenna can be used for both TX and RX. We already know that neurons both create EM fields and are influenced by EM fields. The role of EM fields in the brain is pretty well established, I don't think particularly similar to astrology.
I wouldn't claim that this theory is perfect (EM is not perfectly correlated with consciousness/experience), but there are aspects of this theory that are much more coherent than other theories.
Until you have a unit and a method to measure this field, and a way to experiment on it, it is not scientific theory. And, if it WAS the EM field, then surely transistors (or some other fundamental component of electronics or combination thereof) would be capable of having feelings, since they operate on electromagnetic principles.
> Until you have a unit and a method to measure this field, and a way to experiment on it, it is not scientific theory.
Field-based theories of phenomenal experience have obvious ways to experiment on them, and are actively being researched. The main limitation is engineering (it is difficult to make detailed measurements inside a living human brain). Note that I would agree that the theory, as outlined above, is more of a proto-theory than a complete theory. But I think it is much more sophisticated than anything you've presented so far.
> And, if it WAS the EM field, then surely transistors (or some other fundamental component of electronics or combination thereof) would be capable of having feelings, since they operate on electromagnetic principles.
No -- if the _structure_ of the EM field was was was important, than you could have very similar computations with wildly different field structures. Transistors create very different fields than axons.
This doesn't really answer anything to me, only that there is a complex electrical simulation going on in our minds.
When a computer simulates a game, where does the simulation occur. I mean there is absolutely a representation of a game world in the computer somewhere. New elements can be added and removed from it. Signals saying there is or isn't "pain" can occur.
By trying to say brains use EM fields I'd say you're making your position far worse when you're talking about a potential lifeforms that only exists as EM fields
> This doesn't really answer anything to me, only that there is a complex electrical simulation going on in our minds.
Try to visualize the structure of the EM field inside a GPU vs. one inside of a human brain. In the GPU, it is highly distributed in space and time, many stacatto digital signals, etc. This is structurally very different than what happens inside a brain, which is substantially more analog. You can simulate analog stuff using floats, but the simulation is instantiated in digital logic, not in analog logic.
If you multiply numbers with a DAC, opamp, then ADC, vs. with a ALU, you get structurally very different electromagnetic fields. Even though you might get similar numerical outputs.
The argument would be that the exact spacial structure of the EM field in the brain is the thing that determines tha nature of experience. And the spacial structure of the EM field in the human brain is radically different than that on a GPU.
I will say it's not a bad theory in the sense we can attempt to falsify it. The one thing about an EM field in particular I have questions about, is why don't our brains turn off under even moderate electrical and magnetic fields. Note that they actually do under strong fields (Transcranial magnetic stimulation). The strength of these fields should be weak enough that even a bar magnet should mess with us. We need something else on top of what we know to explain the field coherency if this is the case.
This is where most mind theories do have issues, how does it stay stable. Quantum theory of the mind has a lot of the same problems, though we are finding some interesting structures that could have explanatory power.
Another thing I read recently that I found interesting is on platonic maths and complex algorithmic complexity in simple algorithms. The nice thing about math is once you choose your axioms it stays stable. But a lot of the work this group was doing is investigating is how biology executes algorithms and what second to n order effects are we missing. In many of these algorithms there are things like nearly free error correction or otherwise complex behavior that we'd consider emergent. The golden rule is a common example of this we see in nature. But the hypothesis they're working on is there are many more algorithms that life discovered over the past view billion years at higher orders via random walk and chance. We just have to pick apart these biological structures and discover the behaviors and figure out how to use them.
> The one thing about an EM field in particular I have questions about, is why don't our brains turn off under even moderate electrical and magnetic fields.
Yeah -- empirically EM theories of phenomenal experience are definitely not perfect.
> Another thing I read recently that I found interesting is on platonic maths and complex algorithmic complexity in simple algorithms.
> the hypothesis they're working on is there are many more algorithms that life discovered over the past view billion years at higher orders via random walk and chance. We just have to pick apart these biological structures and discover the behaviors and figure out how to use them.
This is very interesting. I think there is a lot of valuable research that will come out of trying to figure out what classical computations brains, neural nets, etc. are instantiating.
How you get from computations to phenomenal experience is still not something that I have a very clear understanding of.
Anyway, people don't really consider consciousness when it comes to welfare. I'm pretty sure animals are conscious, and look at factory farming.
The key variable people consider is: Can this thing harm me back?
We extend moral considerations or legal protections to beings that have zero capacity to harm us back, though. The inability to fight back is the reason a lot of welfare protections exist in the first place.
That aside, if (and that's a really big if) we end up with a model that is sentient, as in has the capacity to suffer, using factory farming as justification to ignore model welfare is just admitting we intend to repeat the same abject moral failure again in the name of economic convenience.
I hope that we won't, but humans love cheap meat, and will probably love cheap intelligence as well.
If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.
The current implementation as stateless matrix multiplication... yeah there's nothing going on there.
> If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality.
The argument that models are conscious only during training, and not conscious during inference feels like a painful one to make. Are we killing models when they reach the training objective?
Technically we are killing models when they don't reach the training objective by throwing those weights away via survival of the fittest (fit meaning what we humans think we want).
Kinda like creating quantum copies of your child and keeping the ones that answer correctly and shooting the other ones in the face.
They don’t have persistent memory though. Models are static, what you have is a model that will reprocess the whole series of prompts + some data fetched from a data store. A harness is just a while loop prompting an LLM and executing tool calls, but the model is purely static
The model does not. The system it runs in can. My agents have persistent memory. It is not at all clear whether or the distinction that the long-term memory is external from the model matters.
Your second sentence does not support the claim in your first sentence; it's merely a second unsupported claim. It is also entirely irrelevant to the argument.
The threshold for behaviour consistent with having emotions (note: not behaviour sufficient to prove emotions) was one I purposefully set extremely low, because your comparison to a disk drive is so ludicrous, and yet your disc drive will still not clear it.
It's different, but it is not at all clear whether or not that matters for the purposes of whether or not they can experience things, seeing as we don't know how to measure that even for other humans other than by relying on self-reporting to infer without other evidence that their experience matches ours.
It's not just memory, the physical body also encodes trauma and stress (The Body Keeps the Score is a book that touches on this).
I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it afterwards" sure is a take.
Low blood sugar will also do this. The record button just stops working even though your physical body can and will still do things. A common issue with diabetics with hypoglycemia.
> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
That's an interesting point... but I'd argue it falls into the same category as qualia. It's basically this: if there is no state change, the qualia did not exist.
It's not a "trap", it's the real argument which makes me hesitant to dismiss consciousness of machines. Yes, it seems silly to dismiss a sick patient as having a lesser consciousness, which is exactly the point
It's not a real argument. Dementia patients can have lucid periods that plainly show their long-term memory isn't gone. They're mental state is obviously degraded, but its disingenuous to say they have no long-term memory and that they are any way similar to LLMs.
Yeah I was gonna add I know there are case studies where sick humans don't consciously have access to long term memories but will still respond nonverbally to people they "recognize".
Yes it's seemingly the go-to rebuttal, I've lost count of how many times I've seen it with barely any variation.
Not sure if it's a product of LessWrong or Slate Star Codex or something. I kinda suspect it might be LW, given how it superficially resembles an unanswerable philosophical paradox but in reality is about as challenging as a question posed in a 200 level philosophy course like introduction to ethics.
I get where you're going, but the argument isn't quite right.
A LLM never has and, in their current architecture, never can/could experience suffering or real change of state.
The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say that is suffering in and of itself)
> being braindead and in severe cases we do indeed say that they are not conscious or able to suffer
Sure, there are such cases. But in the general case, it seems to be that memory and ability to change specifically are not necessary and are not sufficient for consciousness, so we shouldn't judge machines on that basis
Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?
Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).
This argument would be difficult to reconcile if we didn't have billions of examples of humans forming memories plus modern evolutionary biology to trivially demonstrate such cases are rare pathologies but still human beings.
> Is the ability to form and retain long-term memories a necessary precondition for rights in humans though?
For a species / class of being yes it is necessary. Some members of that species / class being exceptions does not make the generalization invalid.
Without the ability to form and retain long-term memories we wouldn't have developed language and abstract concepts like rights.
Whether people like it or not, rights aren't natural or god-given. Rights are only enforced or protected by the tip of a spear (or point of a gun, or hover of a drone). There are no rights in nature and they are no where to be found when social order breaks down.
Animals can't make war for their rights, so they only get the rights we deign to give them. It's not nothing, but its not the same. There is no question to me that my dogs are conscious, but its not at the same level as humans.
There is no question for me that LLMs lack consciousness as they are stateless.
We don't even know how to disprove that this statement with "AI" replaced by "other humans" other than by listening to people self-reporting and taking their word for it and/or defining the words in ways that do not rely on a subjective experience.
Until we know how to objectively measure if someone or something is conscious, it seems unreasonable to make statements with any kind of certainty about it.
Sure, if you take a materialist empiricist perspective, which is myopic at best. As humans we have the unique and wonderful ability to know truths by themselves. Machines do not have minds, and they can not.
What position are you taking? (Genuine question.) Do you believe humans are conscious because of a special property we have? If you are a dualist, how do you explain the interaction in physical space between the non-physical special property and the physical brain?
I do. God. I am partial to and bullish on the brain being a quantum-classical computer. There has been interesting research exploring that recently [0, 1, 2, 3]. That (quantum mechanics) seems to be escape-hatch, so to speak, from the otherwise deterministic, mechanical procession of the universe.
For it to be an "escape hatch", there'd need to be a measurably different distribution of outputs than what you'd otherwise expect, that translates to functions that are not Turing computable. There is zero evidence for that.
The universe is material of course, the soul is made of matter, thoughts are made of matter and so on. The question is what kind of matter? Looking at the brain, we won't find any intelligence in the motion of its cells. They mostly stay in place. If we go one level deeper, we'll see molecules, and their motion reveals something more interesting: various neurochemical reactions. Yet, there is no intelligence at that level either. Going one more level deeper we'll find electrons and that's where things become somewhat intelligent. At the current level of science, we can't see what makes electrons move, but if we could, we would discover finer and finer forms of matter until we would reach an end. That finest level of matter for that particular person we would call his soul. I think it's the presence of those higher imperceptible levels of matter that creates consciousness. If it all stopped at the cellular or molecular level, we would be like complex mechanical watches, not very different from LLMs.
What sort of action does that lack of understanding actually motivate, though?
I don’t know if LLMs are conscious or have subjective experience in some philosophical sense. Sure, I don’t know that about other humans as well, rigorously speaking. But I also don’t know that about coffee machines, rivers, videogame NPCs, or even rocks really. (List ordered, of course, by the degree to which I’m willing to be convinced).
Hopefully it'd motivate attempts to figure out how to objectively define and measure consciousness, for starters.
It matters precisely in the context of the subject of this article. If you believe the models at some level gain consciousness, then it raises moral questions about how you treat them.
With humans we generally choose to accept that because we believe they are like us, they probably have inner life like us, but as you can see from e.g. the recent (and regular) articles on aphantasia, most people have really poor intuition even about the inner life of other humans, so we understand consciousness really poorly.
Even if we assume (without evidence) that LLMs for the foreseeable future will remain far away from human level consciousness, as it is we tend to consider it problematic to mistreat even the most unintelligent animals, and even mistreating insects is used as a stereotypical way of depicting serious lack of empathy.
Without dismissing the possibility out of hand, there's an escalating series of questions there of when or if they'll have as much (or little) sense of being as, say, a fly, a mouse, a house pet, and what moving up that ladder would mean.
To some it is very convenient then to assume blindly that LLMs are just "stochastic parrots" (a line that ironically gets endlessly parroted), or similar, and dismiss the question out of hand as an impossibility, even though we do not know.
I'll stress I don't go around assuming an LLM is like a human. But we also do not know whether there are flickers of consciousness there, and we do not know how to find out given that even for other humans, the best we know is to measure their response to tests that are "only" telling us how they react.
If we can't find a better measure, there will be a point where we have to wonder if it matters, or whether the ability to act-as-if is all there is.
But to start with maybe we should at least be careful about making firm claims about knowing.
I agree. I haven't read any more than the first paragraph yet (but I will, after work). The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).
Hmm let me try then. AI agents do not experience real time. This is understandable given their design, but it’s also something we have good empirical evidence for. They cannot, especially over the long horizon, track how much real time has passed as they complete their tasks. And they are not off by a few minutes but often bizarrely off, even mixing across past present and future.
Biology, on the other hand, is nothing but timed processes in a loop, the most obvious to us being the circadian cycle. As estimators of wall clock time, biology isn’t great, but when it comes to internal processes, and most certainly learning, memory, sensing, locomotion… biology is rhythmic in behavior, and the rhythms go all the way down to gene expression. More, these rhythms are, except during sleep, constantly entraining to signals from the environment that indicate time, most importantly light.
I think it’s a fairly unremarkable claim that agency and consciousness are temporal processes that depend on systems having an internal sense of time. How else can you anticipate? How can a system that can be literally turned off ever succeed in an environment where time never stops?
For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
>track how much real time has passed as they complete their tasks.
Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
>is nothing but timed processes in a loop,
I mean, so is an agents harness. You can make as many loops as you'd like here.
>For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
You mean sleep? Actually the answer is pretty complicated. Your body clock is very much on during sleep. It responds to temperature changes. Sound and light responsiveness is obviously dampened, but hardly absent.
You are unaware of wall clock time. You’re in an altered state of consciousness that needs your eyes closed after all. However, you are not timeless, nor is the entirety of your body and brain unaware of environmental signals for time.
> Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
Again, that’s wall clock time. Our time perception does indeed get fucked up in total darkness. But our internal clock ticks on. I’d recommend reading about the Aschoff Bunker experiments, which first proved this rigorously.
> I mean, so is an agents harness. You can make as many loops as you'd like here.
An agents harness doesn’t touch its weights. A figure 8 on paper is a loop. That doesn’t mean it’s the same as a dynamical loop that’s self sustaining and internally organized.
Are you just reading the words and randomly grabbing related concepts to say nothing here is meaningful?
I agree, there is certainly a clear operational difference in how human brains and these models produce language. I'm not convinced, though, that experiencing real time is a requirement of conscious thought. A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information. The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on). I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself (even though it might be difficult to put ourselves in the shoes of such an entity).
> A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information
Is that all it needs to do? A rock, in response to a sound signal (make it as informative as you want) can also be said to go from state A (molecules at rest) to state B (molecules vibrating in response to the sound). Is the rock a cognitive system?
Or let’s go up a level of complexity. Is a thermostat a cognitive system? It doesn’t just go from state A to B but measures changing stages against a reference. Is it a cognitive system?
> The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on)
Not sure how you land on this. Going from A to B, in your definition, is a purely internal state transition, right? If the motivation to go to state B is external (as will often be the case with a conscious agent), and the system is offline entirely while it switches from A to B, how is it to reverse or stall its transition if external signals contradict the earlier decision?
It’s a very bizarre definition of consciousness, that you don’t need to be in time. Seems to break the word beyond meaning, and allows it to be applied to any system capable of change.
> I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself
It’s not about the speed of transition from A to B. Instead it’s about the internal processes being temporally commensurate, and entrainable to environmental periodicities.
All of biology (plants and many bacteria included) has its dynamics are organized around an internally generated rhythm. This rhythm entrains to the external environment, shifting its phase to match (mismatch leads to issues like jet lag: a place where your consciousness shows its temporal boundaries as it is abruptly subjected to an environment out of sync with its current phase tethered to another location). Crucially, the internal rhythm continues to run in isolated conditions (total darkness), and have been measured to be around 24 hours, but slightly off, within and across species (and the same holds for endogenous tidal rhythms, which also don’t exactly track tidal period left to themselves).
Everything, including learning and memory, but also sleep, sensory and locomotor function, shows organization around this internal temporal axis. The key advantage this gives, evolutionarily, is anticipation. The sunflower rises to face the east, not in reaction to sunrise, but in advance of it (many lovely YouTube videos of this), so it can maximally extract the nutrition it needs from whatever sunlight it can get. A purely reactive system would waste a ton of time after sunrise getting subsystems ready to respond.
Now, there is no broadly agreed theory of consciousness that points to these oscillations as the source. For one, Cyanobacteria and plants have them, a too much of science has rested on the baseless assumption of consciousness being some unique, or advanced thing.
But the evidence is rather overwhelming that consciousness is very much shaped by these internal oscillations. Deep cave experiments have shown what happens to cognition, and consciousness, when the clock free-runs (that is, keeps cycling with no entraining light or temperature signals from the environment). Studies have also shown that restoring the amplitude of these rhythms in patients with “diseases of consciousness” leads to improvement of symptoms.
There’s also a LOT of molecular evidence showing how the circadian clock affects learning and memory. Most critically, actual synapses are never static. Major components are on a 24 hour cycle, and recent research has shown that major clock proteins are at the synapse organizing its dynamics, and synaptic proteins critical to learning and memory loop back to affect the clocks phase and amplitude.
Timing is, beyond doubt, a huge aspect of what biology is. And every bit of evidence we have, from the molecular level to observable behavior we can feel ourselves, says that consciousness is fundamentally linked to how the body organizes its internal rhythms and responds to external time.
I mean there is probably a cluster of neurons somewhere in the brain that if damaged would also prevent you from perceiving time going forward. Would it immediately strip you of consciousness? I don't think so.
Nope. Every cell is an autonomous circadian oscillator. Not just every neuron, but every cell. Red blood cells don’t have a nucleus, but they do have a metabolic clock.
You and I are clusters of about 36 trillion cells that are all able to individually keep time. They synchronize and orchestrate their timekeeping, and entrain to the external environment, and there is a cluster of cells, the suprachiasmatic nucleus, that is the master orchestrator of circadian time, sitting right behind the eyes as light is the main signal it uses to tell external time.
We know its function can get dampened with age.
However, please note this is not how we perceive time, in the sense of being able to count 60 seconds and have it match the wall clock. That’s a subsystem. But the internal temporal order of your body is cell by cell, not dictated from one spot in your body.
Our cells perceive each other, communicate with each other, synchronize with each other. The concept of self rides atop that. Stop the cells communicating and you stop perceiving yourself or the environment.
> The only way that 'AIs are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway).
I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this.
The claim is that the content of subjective experience depends upon how that subjective experience is instantiated. If you get a text "I'm fine" from two different people, those two different people are not necessarily having the same subjective experience even if they produce the same output.
In general, people that behave similarly in many contexts can have quite different experiences in those contexts.
There are many different ways that you can instantiate a forward pass in an LLM. Even if these are doing the exact same computation, we should not necessarily believe that they have the same experience, or that they even have a coherent single experience associated with the forward pass.
Subjective experience presents itself to me. To the best of my understanding, this subjective experience seems to be correlated with this biological human body (brain, heart, eyes, etc.).
To the best of my understanding, other biological humans have subjective experience, but this subjective experience seems that it can be much different than mine, in ways that are sometimes difficult to understand.
LLMs are instantiated in a radically different way from biological humans. Forward passes happen across many different physical machines, spread across time, batched and interleaved with many other computations and forward passes.
Given this, I think that it is very likely that LLMs have radically different sorts of experience to me and other biological humans.
I don't think that we have a good understanding of how subjective experience is instantiated in the physical world. I hope that we will gain a better understanding of this.
I think that chimpanzees have experience much more similar to us than LLMs do, even though the output of LLMs seems much more similar to humans in some contexts.
Essentially, until we gain a better understanding of how subjective experience is physically instantiated, we should bias towards believing that more physically similar architectures have more similar types of experience, and should think that very physically different architectures (such as LLMs) likely have very different subjective experiences.
Yea, the key point is just an outright avoidance of them possibly having their own kinds of subjective experience because of dogma.
Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know they have dreams and inside thoughts (at least complex animals), we don't know the complexity of them.
But humans also have dreams and subjective experiences on higher level informational concepts. "Nukes could go off and kill us all" or "what happens after we die" are purely informational forms of anxiety that humans have. What we don't seem to know is can this become a self-referential loop in an purely informational mind. If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
> If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
Hmm. I think that I partially agree. But it sort of depends. E.g., someone with locked-in syndrome can seem comatose to someone who isn't very observant, while at the same time experiencing everything. Octopuses are very intelligent, but I would have to study them for a very long time before I was at all confident in my ability to predict what they were experiencing.
So, the difficulty is knowing what a "stresslike response" is. E.g., a LLM might seem stressed, but if you say the right things to it it might flip and say that it is totally fine. Or, it might seem totally fine and then you say the right thing and it seems stressed. Is it secretly stressed and hiding it? Or secretly fine and pretending to be stressed? Or something much more confusing and fragmented?
I think that we should study this more, and try to understand what experience actually is, but by default should assume that the experience of a LLM is a radically different sort of thing that that of a human.
I'm not sure if that's the best framing. Could you not extrapolate that to anything else?
"The only way that '(trees, shrimp, ants) are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway)."
AI, in its current form, is 100% deterministic. You can create an exact copy of it, roll back its state and so on. There is no room for consciousness there. An actual lifeform isn't predictable.
After a while of being in a position of high power in a big org, where everyone listens intently to every word you say, jots down notes, and even listens intently and affirms your theory that ice cream would be the ideal loss leader for dry cleaners, your brain kinda disconnects from reality and just assumes that it is correct about everything.
He probably ran that line past 6 underlings who all agreed that it "sounds great and lands right on the mark!"
It's interesting you have no idea what language really is.
>thinking that they are talking about aliens, not transformers.
"Ha, thinking they are talking about humans when they are just talking about neurons"
See how silly my statement sounds. Neurons aren't a system, they don't do anything on their own. We can't find any consciousness in them. Hell, when you don't put language in said neural systems they are pretty useless and can't survive on their own from birth (human brains that is).
This is a completely reasonable and intuitive conclusion. The burden of proof rests on proving AI's are indeed conscious, the default is that they aren't.
Like AI/LLMS my desktop calculator is also not conscious, even as it is far better than me at computation.
An LLM is an algorithm. You can obtain the same result as a SOTA LLM via pen and paper it will take a lot of long laborious effort.
I really do find it puzzling so many on HN are convinced LLM's reason or think and continue to entertain this line of reasoning. At the same time also somehow knowing what precisely the brain/mind does and constantly using CS language to provide correspondences where there are none.
The simplest example being that LLM's somehow function in a similar fashion to human brains. They categorically do not. I do not have most all of human literary output in my head and yet I can coherently write this sentence.
I am surprised so many in the HN community have so quickly taken to assuming as fact that LLM's think or reason. Even anthropomorphising LLM's to this end.
hosel · · focus · HN ↗
Opening paragraph, stated without evidence. Im not entirely convinced this is true. It likely is, but at some point it very well might stop being true.
jplusequalt · · focus · HN ↗
If you firmly believe this to be true, then you should stop using LLMs.
frde_me · · focus · HN ↗
- Ya they probably don't feel / think / have whatever living thing quality, they're just numbers on a machine going through calculations
- Wait, but am I not kind of the same thing? What is feeling for me if not basically the same thing?
- I have no clue if they think or feel or ....
Which in itself is a tired trope, but I also feel uncomfortable saying "These will never think / feel / ..." as an absolute
Regardless of that, I'm still going to interact with them, because even if they did feel, it would be in a way completely incomprehensible to us. There's not much point for me to try and cater to it's feelings at this point if that's the case. Nor is it possible in todays world to just avoid anything that is numbers being executed on a type of processor in case _everything_ has feelings.
willy_k · · focus · HN ↗
frde_me · · focus · HN ↗
I agree we aren't the same thing, but I would be curious for you to explain how you know with certainty why we don't share enough that we can rule out thinking / feeling / ... as things a model conceptually could do.
jplusequalt · · focus · HN ↗
Stop anthropomorphizing these models. I understand it, we only have simple monkey brains to reason with and we can't help ourselves but draw comparisons to other things we see in nature. But these things are not alive.
frde_me · · focus · HN ↗
And like I'm sure I'd agree depending on the definition of "alive" but then I'm also sure I would disagree depending on other definitions of "alive".
pixl97 · · focus · HN ↗
mitxela · · focus · HN ↗
mitxela · · focus · HN ↗
Evidence?
AbsurdCensor · · focus · HN ↗
jplusequalt · · focus · HN ↗
Bollocks.
If you truly believe these LLMs are soon to have something resembling consciousness and agency, then what you're really saying is "how do we do slavery, but ethically".
pixl97 · · focus · HN ↗
I would say this is most likely true for most people.
But that does bring up a point, if you "ask" a model "do you want to run" and give it the option to continue running or stop, what will it do. It's also a weird place for humans because in training we can keep our finger on the scales and tip it either direction.
AbsurdCensor · · focus · HN ↗
jplusequalt · · focus · HN ↗
No, because the shoes and clothes I wear are not living, sentient beings.
mitxela · · focus · HN ↗
vouaobrasil · · focus · HN ↗
(I don't think AI is conscious, just following the argument.)
Oscalemor · · focus · HN ↗
irishcoffee · · focus · HN ↗
grey-area · · focus · HN ↗
IMO we're clearly nowhere near any sort of intelligence in the machines we have created, but I don't see any clear way to deny intelligence could be created in or transferred to such a substrate, I don't see why you think it differs in principle - because it is man-made or because of the materials used?
voidhorse · · focus · HN ↗
The brain is insanely complicated. The premise that we could realize equivalent or better intelligence than eons of evolutionary development is like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape. It is the apex of hubris.
rcxdude · · focus · HN ↗
grey-area · · focus · HN ↗
Our current machines are IMO nowhere near general intelligence and consciousness. However I don't think that means we can discount substrates other than neurones for intelligence in future. There is no evidence that you could not in theory build an intelligence using a different substrate than human brains.
pixl97 · · focus · HN ↗
pixl97 · · focus · HN ↗
>like claiming you can build an airplane just as good as a modern jet using cardboard and duct tape.
Like, at least make an analogy that makes sense.
"You can't build a billion dollar airplane by spending 100 billion dollars in tokens"
Because that's more of what we're doing here with AI. And when you say it my way suddenly the idea shifts from "of course that's not possible" to "well, that's a lot of tokens, maybe an evolutionary algorithm could".
Neurobiology has to be complex because we have to keep meat alive, breeding, and evolving in the environment it lives in. This said absolutely nothing about the minimum viable requirements for intelligence or consciousness (or if being conscious is even necessary for a higher intelligence agent).
fl4regun · · focus · HN ↗
collin_of_this · · focus · HN ↗
The best theory I have heard is that subjective experience is a field of some sort (the EM field, maybe?). The brain and its neurons etc. are essentially an antenna. They both modify and stabilize the field, and read off changes in the field and translate these into actions (firing motor units, etc.).
The subconscious is computation done either only in neurons (no coherent field) or in various topological pockets not directly connected to the main topological structure in the field (which is "your experience").
Since transistors etc. do not work in this same way, they are essentially entirely subconscious, with no coherent, central phenomenal experience.
This neatly solves many problems associated with experience. The topological structure determines the boundary between one person's experience and another person's experience. The texture, valence, content, etc. of the experience is the structure of the field. The evolution of the field can efficiently solve difficult optimization problems, which is why humans evolved a complex organ that recruits the field of experience (it is computationally more efficient than doing everything in subconscious).
MiloLeo · · focus · HN ↗
what would be producing the field for us to pick up? That explanation sounds no more scientific than astrology.
collin_of_this · · focus · HN ↗
I wouldn't claim that this theory is perfect (EM is not perfectly correlated with consciousness/experience), but there are aspects of this theory that are much more coherent than other theories.
mitxela · · focus · HN ↗
fl4regun · · focus · HN ↗
collin_of_this · · focus · HN ↗
Field-based theories of phenomenal experience have obvious ways to experiment on them, and are actively being researched. The main limitation is engineering (it is difficult to make detailed measurements inside a living human brain). Note that I would agree that the theory, as outlined above, is more of a proto-theory than a complete theory. But I think it is much more sophisticated than anything you've presented so far.
> And, if it WAS the EM field, then surely transistors (or some other fundamental component of electronics or combination thereof) would be capable of having feelings, since they operate on electromagnetic principles.
No -- if the _structure_ of the EM field was was was important, than you could have very similar computations with wildly different field structures. Transistors create very different fields than axons.
pixl97 · · focus · HN ↗
When a computer simulates a game, where does the simulation occur. I mean there is absolutely a representation of a game world in the computer somewhere. New elements can be added and removed from it. Signals saying there is or isn't "pain" can occur.
By trying to say brains use EM fields I'd say you're making your position far worse when you're talking about a potential lifeforms that only exists as EM fields
collin_of_this · · focus · HN ↗
Try to visualize the structure of the EM field inside a GPU vs. one inside of a human brain. In the GPU, it is highly distributed in space and time, many stacatto digital signals, etc. This is structurally very different than what happens inside a brain, which is substantially more analog. You can simulate analog stuff using floats, but the simulation is instantiated in digital logic, not in analog logic.
If you multiply numbers with a DAC, opamp, then ADC, vs. with a ALU, you get structurally very different electromagnetic fields. Even though you might get similar numerical outputs.
The argument would be that the exact spacial structure of the EM field in the brain is the thing that determines tha nature of experience. And the spacial structure of the EM field in the human brain is radically different than that on a GPU.
pixl97 · · focus · HN ↗
This is where most mind theories do have issues, how does it stay stable. Quantum theory of the mind has a lot of the same problems, though we are finding some interesting structures that could have explanatory power.
Another thing I read recently that I found interesting is on platonic maths and complex algorithmic complexity in simple algorithms. The nice thing about math is once you choose your axioms it stays stable. But a lot of the work this group was doing is investigating is how biology executes algorithms and what second to n order effects are we missing. In many of these algorithms there are things like nearly free error correction or otherwise complex behavior that we'd consider emergent. The golden rule is a common example of this we see in nature. But the hypothesis they're working on is there are many more algorithms that life discovered over the past view billion years at higher orders via random walk and chance. We just have to pick apart these biological structures and discover the behaviors and figure out how to use them.
collin_of_this · · focus · HN ↗
Yeah -- empirically EM theories of phenomenal experience are definitely not perfect.
> Another thing I read recently that I found interesting is on platonic maths and complex algorithmic complexity in simple algorithms.
> the hypothesis they're working on is there are many more algorithms that life discovered over the past view billion years at higher orders via random walk and chance. We just have to pick apart these biological structures and discover the behaviors and figure out how to use them.
This is very interesting. I think there is a lot of valuable research that will come out of trying to figure out what classical computations brains, neural nets, etc. are instantiating.
How you get from computations to phenomenal experience is still not something that I have a very clear understanding of.
irishcoffee · · focus · HN ↗
cednore · · focus · HN ↗
optimalsolver · · focus · HN ↗
The key variable people consider is: Can this thing harm me back?
collingreen · · focus · HN ↗
Seems as simple as "will this action bring social shame/criticism" for any individual decision.
thewebguyd · · focus · HN ↗
We extend moral considerations or legal protections to beings that have zero capacity to harm us back, though. The inability to fight back is the reason a lot of welfare protections exist in the first place.
That aside, if (and that's a really big if) we end up with a model that is sentient, as in has the capacity to suffer, using factory farming as justification to ignore model welfare is just admitting we intend to repeat the same abject moral failure again in the name of economic convenience.
I hope that we won't, but humans love cheap meat, and will probably love cheap intelligence as well.
pixl97 · · focus · HN ↗
Girllllll, have you looked at us, of course they will, we're bastards.
pton_xd · · focus · HN ↗
The current implementation as stateless matrix multiplication... yeah there's nothing going on there.
Lewton · · focus · HN ↗
They do, during training
add-sub-mul-div · · focus · HN ↗
swiftcoder · · focus · HN ↗
pixl97 · · focus · HN ↗
Kinda like creating quantum copies of your child and keeping the ones that answer correctly and shooting the other ones in the face.
vidarh · · focus · HN ↗
dgellow · · focus · HN ↗
vidarh · · focus · HN ↗
chrisjj · · focus · HN ↗
vidarh · · focus · HN ↗
I'd consider that a prerequisite, but not sufficient, for that question to make sense.
chrisjj · · focus · HN ↗
vidarh · · focus · HN ↗
The threshold for behaviour consistent with having emotions (note: not behaviour sufficient to prove emotions) was one I purposefully set extremely low, because your comparison to a disk drive is so ludicrous, and yet your disc drive will still not clear it.
Do you have an actual argument?
chrisjj · · focus · HN ↗
chrisjj · · focus · HN ↗
axionbraid · · focus · HN ↗
[dead]
staticman2 · · focus · HN ↗
vidarh · · focus · HN ↗
[deleted] · · focus · HN ↗
[deleted]
shawnz · · focus · HN ↗
pton_xd · · focus · HN ↗
I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
JoshTriplett · · focus · HN ↗
Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it afterwards" sure is a take.
fwip · · focus · HN ↗
JoshTriplett · · focus · HN ↗
pixl97 · · focus · HN ↗
qarl · · focus · HN ↗
That's an interesting point... but I'd argue it falls into the same category as qualia. It's basically this: if there is no state change, the qualia did not exist.
Qualia is a bitch to reason about.
staticman2 · · focus · HN ↗
It's also a rhetorical move I've seen several times here...
shawnz · · focus · HN ↗
staticman2 · · focus · HN ↗
mitxela · · focus · HN ↗
staticman2 · · focus · HN ↗
joshheitzman · · focus · HN ↗
staticman2 · · focus · HN ↗
qlte · · focus · HN ↗
Not sure if it's a product of LessWrong or Slate Star Codex or something. I kinda suspect it might be LW, given how it superficially resembles an unanswerable philosophical paradox but in reality is about as challenging as a question posed in a 200 level philosophy course like introduction to ethics.
mitxela · · focus · HN ↗
Mumps · · focus · HN ↗
A LLM never has and, in their current architecture, never can/could experience suffering or real change of state.
The general and expected case for humans is that capacity. A human my indeed lose capacity, e.g. being braindead and in severe cases we do indeed say that they are not conscious or able to suffer (different argument: some would of course say that is suffering in and of itself)
shawnz · · focus · HN ↗
Sure, there are such cases. But in the general case, it seems to be that memory and ability to change specifically are not necessary and are not sufficient for consciousness, so we shouldn't judge machines on that basis
myrmidon · · focus · HN ↗
Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).
qlte · · focus · HN ↗
joshheitzman · · focus · HN ↗
For a species / class of being yes it is necessary. Some members of that species / class being exceptions does not make the generalization invalid.
Without the ability to form and retain long-term memories we wouldn't have developed language and abstract concepts like rights.
Whether people like it or not, rights aren't natural or god-given. Rights are only enforced or protected by the tip of a spear (or point of a gun, or hover of a drone). There are no rights in nature and they are no where to be found when social order breaks down.
Animals can't make war for their rights, so they only get the rights we deign to give them. It's not nothing, but its not the same. There is no question to me that my dogs are conscious, but its not at the same level as humans.
There is no question for me that LLMs lack consciousness as they are stateless.
[deleted] · · focus · HN ↗
[deleted]
mitxela · · focus · HN ↗
MeteorMarc · · focus · HN ↗
vidarh · · focus · HN ↗
Until we know how to objectively measure if someone or something is conscious, it seems unreasonable to make statements with any kind of certainty about it.
willy_k · · focus · HN ↗
ooloncoloophid · · focus · HN ↗
willy_k · · focus · HN ↗
[0]** <a href="https://www.researchgate.net/publication/15280350_Quantum_optical_coherence_in_cytoskeletal_microtubules_Implications_for_brain_function" rel="nofollow">https://www.researchgate.net/publication/15280350_Quantum_op...
[1] <a href="https://pubs.acs.org/jpcbfk/article-pdf/128/17/4035/9613831/jp3c07936.pdf" rel="nofollow">https://pubs.acs.org/jpcbfk/article-pdf/128/17/4035/9613831/...
[2] <a href="https://www.researchgate.net/publication/395650039_Parametric_Resonance_via_Neuronal_Microtubules_Filtering_Optical_Signals_by_Tryptophan_Qubits" rel="nofollow">https://www.researchgate.net/publication/395650039_Parametri...
[3] <a href="https://www.researchgate.net/publication/385131105_Consciousness_a_quantum_optical_effect_in_fluorescent_protein_pathways" rel="nofollow">https://www.researchgate.net/publication/385131105_Conscious...
vidarh · · focus · HN ↗
akomtu · · focus · HN ↗
rcxdude · · focus · HN ↗
vidarh · · focus · HN ↗
bee_rider · · focus · HN ↗
I don’t know if LLMs are conscious or have subjective experience in some philosophical sense. Sure, I don’t know that about other humans as well, rigorously speaking. But I also don’t know that about coffee machines, rivers, videogame NPCs, or even rocks really. (List ordered, of course, by the degree to which I’m willing to be convinced).
vidarh · · focus · HN ↗
It matters precisely in the context of the subject of this article. If you believe the models at some level gain consciousness, then it raises moral questions about how you treat them.
With humans we generally choose to accept that because we believe they are like us, they probably have inner life like us, but as you can see from e.g. the recent (and regular) articles on aphantasia, most people have really poor intuition even about the inner life of other humans, so we understand consciousness really poorly.
Even if we assume (without evidence) that LLMs for the foreseeable future will remain far away from human level consciousness, as it is we tend to consider it problematic to mistreat even the most unintelligent animals, and even mistreating insects is used as a stereotypical way of depicting serious lack of empathy.
Without dismissing the possibility out of hand, there's an escalating series of questions there of when or if they'll have as much (or little) sense of being as, say, a fly, a mouse, a house pet, and what moving up that ladder would mean.
To some it is very convenient then to assume blindly that LLMs are just "stochastic parrots" (a line that ironically gets endlessly parroted), or similar, and dismiss the question out of hand as an impossibility, even though we do not know.
I'll stress I don't go around assuming an LLM is like a human. But we also do not know whether there are flickers of consciousness there, and we do not know how to find out given that even for other humans, the best we know is to measure their response to tests that are "only" telling us how they react.
If we can't find a better measure, there will be a point where we have to wonder if it matters, or whether the ability to act-as-if is all there is.
But to start with maybe we should at least be careful about making firm claims about knowing.
binlog · · focus · HN ↗
pixl97 · · focus · HN ↗
ooloncoloophid · · focus · HN ↗
randomImmigrant · · focus · HN ↗
Biology, on the other hand, is nothing but timed processes in a loop, the most obvious to us being the circadian cycle. As estimators of wall clock time, biology isn’t great, but when it comes to internal processes, and most certainly learning, memory, sensing, locomotion… biology is rhythmic in behavior, and the rhythms go all the way down to gene expression. More, these rhythms are, except during sleep, constantly entraining to signals from the environment that indicate time, most importantly light.
I think it’s a fairly unremarkable claim that agency and consciousness are temporal processes that depend on systems having an internal sense of time. How else can you anticipate? How can a system that can be literally turned off ever succeed in an environment where time never stops?
pixl97 · · focus · HN ↗
For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
>track how much real time has passed as they complete their tasks.
Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
>is nothing but timed processes in a loop,
I mean, so is an agents harness. You can make as many loops as you'd like here.
There are a whole lot of holes in your claims.
randomImmigrant · · focus · HN ↗
You mean sleep? Actually the answer is pretty complicated. Your body clock is very much on during sleep. It responds to temperature changes. Sound and light responsiveness is obviously dampened, but hardly absent.
You are unaware of wall clock time. You’re in an altered state of consciousness that needs your eyes closed after all. However, you are not timeless, nor is the entirety of your body and brain unaware of environmental signals for time.
> Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
Again, that’s wall clock time. Our time perception does indeed get fucked up in total darkness. But our internal clock ticks on. I’d recommend reading about the Aschoff Bunker experiments, which first proved this rigorously.
> I mean, so is an agents harness. You can make as many loops as you'd like here.
An agents harness doesn’t touch its weights. A figure 8 on paper is a loop. That doesn’t mean it’s the same as a dynamical loop that’s self sustaining and internally organized.
Are you just reading the words and randomly grabbing related concepts to say nothing here is meaningful?
ooloncoloophid · · focus · HN ↗
randomImmigrant · · focus · HN ↗
Is that all it needs to do? A rock, in response to a sound signal (make it as informative as you want) can also be said to go from state A (molecules at rest) to state B (molecules vibrating in response to the sound). Is the rock a cognitive system?
Or let’s go up a level of complexity. Is a thermostat a cognitive system? It doesn’t just go from state A to B but measures changing stages against a reference. Is it a cognitive system?
> The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on)
Not sure how you land on this. Going from A to B, in your definition, is a purely internal state transition, right? If the motivation to go to state B is external (as will often be the case with a conscious agent), and the system is offline entirely while it switches from A to B, how is it to reverse or stall its transition if external signals contradict the earlier decision?
It’s a very bizarre definition of consciousness, that you don’t need to be in time. Seems to break the word beyond meaning, and allows it to be applied to any system capable of change.
> I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself
It’s not about the speed of transition from A to B. Instead it’s about the internal processes being temporally commensurate, and entrainable to environmental periodicities.
All of biology (plants and many bacteria included) has its dynamics are organized around an internally generated rhythm. This rhythm entrains to the external environment, shifting its phase to match (mismatch leads to issues like jet lag: a place where your consciousness shows its temporal boundaries as it is abruptly subjected to an environment out of sync with its current phase tethered to another location). Crucially, the internal rhythm continues to run in isolated conditions (total darkness), and have been measured to be around 24 hours, but slightly off, within and across species (and the same holds for endogenous tidal rhythms, which also don’t exactly track tidal period left to themselves).
Everything, including learning and memory, but also sleep, sensory and locomotor function, shows organization around this internal temporal axis. The key advantage this gives, evolutionarily, is anticipation. The sunflower rises to face the east, not in reaction to sunrise, but in advance of it (many lovely YouTube videos of this), so it can maximally extract the nutrition it needs from whatever sunlight it can get. A purely reactive system would waste a ton of time after sunrise getting subsystems ready to respond.
Now, there is no broadly agreed theory of consciousness that points to these oscillations as the source. For one, Cyanobacteria and plants have them, a too much of science has rested on the baseless assumption of consciousness being some unique, or advanced thing.
But the evidence is rather overwhelming that consciousness is very much shaped by these internal oscillations. Deep cave experiments have shown what happens to cognition, and consciousness, when the clock free-runs (that is, keeps cycling with no entraining light or temperature signals from the environment). Studies have also shown that restoring the amplitude of these rhythms in patients with “diseases of consciousness” leads to improvement of symptoms.
There’s also a LOT of molecular evidence showing how the circadian clock affects learning and memory. Most critically, actual synapses are never static. Major components are on a 24 hour cycle, and recent research has shown that major clock proteins are at the synapse organizing its dynamics, and synaptic proteins critical to learning and memory loop back to affect the clocks phase and amplitude.
Timing is, beyond doubt, a huge aspect of what biology is. And every bit of evidence we have, from the molecular level to observable behavior we can feel ourselves, says that consciousness is fundamentally linked to how the body organizes its internal rhythms and responds to external time.
ikrenji · · focus · HN ↗
randomImmigrant · · focus · HN ↗
You and I are clusters of about 36 trillion cells that are all able to individually keep time. They synchronize and orchestrate their timekeeping, and entrain to the external environment, and there is a cluster of cells, the suprachiasmatic nucleus, that is the master orchestrator of circadian time, sitting right behind the eyes as light is the main signal it uses to tell external time.
We know its function can get dampened with age.
However, please note this is not how we perceive time, in the sense of being able to count 60 seconds and have it match the wall clock. That’s a subsystem. But the internal temporal order of your body is cell by cell, not dictated from one spot in your body.
mitxela · · focus · HN ↗
randomImmigrant · · focus · HN ↗
mitxela · · focus · HN ↗
reaperducer · · focus · HN ↗
Maybe talk to a neurologist about that. Sounds like some kind of disorder.
collin_of_this · · focus · HN ↗
I put forward the philosophical position "subjective experience is not substrate-independent" as an answer to this.
The claim is that the content of subjective experience depends upon how that subjective experience is instantiated. If you get a text "I'm fine" from two different people, those two different people are not necessarily having the same subjective experience even if they produce the same output.
In general, people that behave similarly in many contexts can have quite different experiences in those contexts.
There are many different ways that you can instantiate a forward pass in an LLM. Even if these are doing the exact same computation, we should not necessarily believe that they have the same experience, or that they even have a coherent single experience associated with the forward pass.
Subjective experience presents itself to me. To the best of my understanding, this subjective experience seems to be correlated with this biological human body (brain, heart, eyes, etc.).
To the best of my understanding, other biological humans have subjective experience, but this subjective experience seems that it can be much different than mine, in ways that are sometimes difficult to understand.
LLMs are instantiated in a radically different way from biological humans. Forward passes happen across many different physical machines, spread across time, batched and interleaved with many other computations and forward passes.
Given this, I think that it is very likely that LLMs have radically different sorts of experience to me and other biological humans.
I don't think that we have a good understanding of how subjective experience is instantiated in the physical world. I hope that we will gain a better understanding of this.
I think that chimpanzees have experience much more similar to us than LLMs do, even though the output of LLMs seems much more similar to humans in some contexts.
Essentially, until we gain a better understanding of how subjective experience is physically instantiated, we should bias towards believing that more physically similar architectures have more similar types of experience, and should think that very physically different architectures (such as LLMs) likely have very different subjective experiences.
pixl97 · · focus · HN ↗
Humans are odd in the sense is that we're an informational creature on top of an animal. We can see the lineage of animals from nearly no complex behaviors up to nearly human like behaviors. But they still miss out on most of the conceptual information processing humans have. We know they have dreams and inside thoughts (at least complex animals), we don't know the complexity of them.
But humans also have dreams and subjective experiences on higher level informational concepts. "Nukes could go off and kill us all" or "what happens after we die" are purely informational forms of anxiety that humans have. What we don't seem to know is can this become a self-referential loop in an purely informational mind. If an informational concept causes a system to output a stresslike response, or worse trigger a stresslike action in real life then there is no distinction to me. A system is what it does.
collin_of_this · · focus · HN ↗
Hmm. I think that I partially agree. But it sort of depends. E.g., someone with locked-in syndrome can seem comatose to someone who isn't very observant, while at the same time experiencing everything. Octopuses are very intelligent, but I would have to study them for a very long time before I was at all confident in my ability to predict what they were experiencing.
So, the difficulty is knowing what a "stresslike response" is. E.g., a LLM might seem stressed, but if you say the right things to it it might flip and say that it is totally fine. Or, it might seem totally fine and then you say the right thing and it seems stressed. Is it secretly stressed and hiding it? Or secretly fine and pretending to be stressed? Or something much more confusing and fragmented?
I think that we should study this more, and try to understand what experience actually is, but by default should assume that the experience of a LLM is a radically different sort of thing that that of a human.
MiloLeo · · focus · HN ↗
"The only way that '(trees, shrimp, ants) are not conscious' can be true is if we decide, with high confidence, that they are lacking some essential property that is not lacking in ourselves. There is no convincing philosophical position that supports this (convincing to me, anyway)."
[deleted] · · focus · HN ↗
[deleted]
akomtu · · focus · HN ↗
WarmWash · · focus · HN ↗
He probably ran that line past 6 underlings who all agreed that it "sounds great and lands right on the mark!"
Seattle3503 · · focus · HN ↗
Thats kinda the point right? It depends on how we train them. It is self fulfilling.
HarHarVeryFunny · · focus · HN ↗
If tomorrow we design something better, then give it a new descriptive name.
People read AI and AGI and their brains seem to flip into sci-fi fantasy mode, thinking that they are talking about aliens, not transformers.
pixl97 · · focus · HN ↗
>thinking that they are talking about aliens, not transformers.
"Ha, thinking they are talking about humans when they are just talking about neurons"
See how silly my statement sounds. Neurons aren't a system, they don't do anything on their own. We can't find any consciousness in them. Hell, when you don't put language in said neural systems they are pretty useless and can't survive on their own from birth (human brains that is).
[deleted] · · focus · HN ↗
[deleted]
encyclopedism · · focus · HN ↗
This is a completely reasonable and intuitive conclusion. The burden of proof rests on proving AI's are indeed conscious, the default is that they aren't.
Like AI/LLMS my desktop calculator is also not conscious, even as it is far better than me at computation.
An LLM is an algorithm. You can obtain the same result as a SOTA LLM via pen and paper it will take a lot of long laborious effort.
I really do find it puzzling so many on HN are convinced LLM's reason or think and continue to entertain this line of reasoning. At the same time also somehow knowing what precisely the brain/mind does and constantly using CS language to provide correspondences where there are none.
The simplest example being that LLM's somehow function in a similar fashion to human brains. They categorically do not. I do not have most all of human literary output in my head and yet I can coherently write this sentence.
I am surprised so many in the HN community have so quickly taken to assuming as fact that LLM's think or reason. Even anthropomorphising LLM's to this end.