Title: Human vs. Machine Consciousness | Imperial’s Murray Shanahan
Author: Johnathan Bi

Transcript:
We're way past the cheuring test easily.
The point is to show you she's a robot
and see if you still think she's
conscious. Human consciousness is
constrained to a subject object dualism
and that AI consciousness might not be.
But my philosophy is very very much
influenced by by Buddhism. Turing was
going to Vickenstein's classes. There
was a Vickinsteinian uh influence on
Turing when he wrote that paper. There
is a fact of the matter whether
something is conscious or not. You don't
need to go get all philosophical.
But but I'm not I'm not getting
philosophical. No, but you are. As soon
as you probe using that little extra
phrase, is there a fact of the matter?
Okay. I'm saying this is not conscious.
I am conscious. You are conscious. Of
course. Okay. You're willing you're
willing to grant that? Of course. What
do you think? I'm an idiot.
I'm a fellow human language user.
How do we tell when AI is conscious? My
guest, Imperial's Murray Shanahan, has
spent his career investigating this
question. And in this interview, we'll
lay out a systematic proposal for how to
validate machine consciousness. Now, you
might think that the question of whether
AIs are conscious is a trivial
theoretical fancy with no use in the
real world. But this interview will show
you why it's one of the most important
questions with practical implications
for not just how we treat AI systems,
but also how we build and align them.
But the most unexpected reward of
investigating AI consciousness turns out
to be what it reveals about our
consciousness, about the nature of the
human self. Murray's most interesting
claim is that LLMs have an important
Buddhist lesson to teach all of us,
namely that there is no us. What we've
learned to call the self is merely an
illusion. My name is Jonathan B. I'm a
founding member of Cosmos. We fund
research, incubate, and invest in AI
startups and believe that philosophy is
critical to building technology. If you
want to join our ecosystem of
philosopher builders, you can find roles
we're hiring for, events, grant
programs, and other ways to get involved
on
jonathanb.com/cosmos. Without further
ado, Murray Shanahan.
One obvious reason people are interested
in the question whether AI is conscious
is the ethics question. Right? If they
are conscious, maybe we need to think
twice about uh turning them off, about
saying insulting things to them. What
are some of the less obvious reasons why
this question matters? Well, you said we
maybe we need to think twice about
turning them off, but actually maybe we
need to think twice about turning them
on, right? What's really an issue here
is whether they can suffer. And so if so
we I really think we maybe we want to
hesitate before we build something
that's genuinely capable of suffering. I
mean there are many dimensions to that
question but the question of moral uh I
mean it might not just be moral standing
that matters but the even if we build
something that perhaps appears to be
conscious but we ultimately decide it
isn't really to mistreat something which
appears conscious is in itself does seem
like a bad thing in the same way as it
would seem bad to you know torture a
doll or something like that and as you
probably know Kant K can't K had the
view that that animals couldn't
experience suffering in the same way
that we can, but nevertheless thought it
was bad if humans subjected animals to
torture or something like that because
it was bad for the humans themselves.
And so maybe we'll be in that position
with AI as well. I mean, so so here's my
answer for why it might be significant
for us to answer this question outside
of the the ethics question, which is
that studying machine consciousness,
whatever that means, might help us
better understand the idiosyncrasies of
our own. Of our own. Oh, for sure. So
you you wrote uh a lovely paper called
Satori before singularity uh in which
you argued that human consciousness is
constrained to a subject object dualism
and that AI consciousness might not be.
Why is that? Yes. So it's it's quite
funny that paper which was published in
2012
sto people have been talking for a long
time about super intelligence um and uh
you know the idea that we might be able
to build AI that is but in some sense
superior intellectually to humans um and
so I was thinking is there another sense
in which we could make or imagine
something that was better than us and uh
so I was thinking about um uh about the
Buddha actually and thinking about the
the idea of people who are enlightened
and people who you know perhaps have
transcended dualistic thinking. Uh and
so I was thinking well perhaps there's
another sense in which we could build
something that's better than us um in
that respect something that that is less
hampered by its own ego and um so so
it's a very speculative slightly bonkers
crazy paper which is why I liked it and
why I'm beginning with it because you're
talking to a continental philosopher
here. Right. Right. Um, so let me give
you a quote from that that paper, right?
Or am I going to cringe? Okay, go on.
Yeah. The pre-reflective, reflective,
post-reflective series is not just one
among many paths through the space of
possible minds. Rather, the space of
possible minds is structured in such a
way that this is the only path through
it. What are these three different
stages that you laid out? Yeah. Okay.
So, I I I I um I don't agree with that
uh anymore. That was more than 10 years
ago. So I don't at all think it's the
only path through the space of possible
minds. Can you explain what these three
what these three? Okay. Absolutely. So
no there so that so they I do take
seriously as as different you know ways
of being. There is the pre-reflective
mind which is the the mind of of um say
the na a naive uh child or or a simple
straightforward ordinary person. Um they
haven't really thought about
philosophical problems. I haven't had
these sorts of thoughts that many of us
have start to have when we're children
of you know, you know, why do I exist?
You know, what is how do I know that my
parents are conscious? You know, some
people start to have those thoughts when
they're children very often. Um, as I
certainly did and I think you probably
did as well by Unfortunately. Yes.
Unfortunately, indeed. And I do think of
it as a kind of affliction because they
can you can really be bothered by these
thoughts. They can be both exciting and
disturbing at the same time. The one
that's troubled me more than anything
else really is the kind of the mind body
problem. How is it that that I the
myself and my my experiences my
consciousness how does that relate to
the world? So how can I reconcile these
first personal experiences with a
universe which is just which is matter
and energy and so how can I reconcile
the subjective with the objective? I
think you can spend your entire life in
that reflective stage and worried about
that um and those problems especially if
you're become a professional philosopher
then you can actually make a career out
of that reflective stage. Um but my I
tend to think um that there is a stage
beyond that which I call the
post-reflective stage. But I think the
post-reflective stage is where we
transcend that all of that dualistic
thinking and it's really really
difficult but we somehow come to see our
our inner life and the outer world the
subjective experience and the objective
physical reality uh as not two different
metaphysical categories but somehow
somehow the same thing. To summarize for
our audience, um these things are
related. What what you said about the
Buddha and the lack of ego centricity
and the post-reflective state that that
does not have this subject object
dualism. Absolutely. Um and your
interesting claim in this paper is that
it's something about the human hardware,
the fact that we're embodied in one body
that cannot be copied and mult, you
know, multiplied and and paused. Yes.
that that that hardware is what gives us
this software limitation. And what I
found most interesting about applying
this to AI is you cited I believe uh
another thought experiment about what if
we had creatures whose bodies are just
fusion and fision all the time. Your
bodies are going in and out of ex
existence and you said in the paper
isn't that close to what computers are
or aren't computers like the closest
thing? Absolutely. And so it's because
of the nature of a software program that
can be copied, halted, multiplied,
deleted, uh, uh, recreated that you're
saying that is the reason why you you
speculate uh, uh, AI might be
post-reflective. Did I did I get that
right? 100% right. And in fact, I'm
actually developing a paper right now of
those that literally are revisiting
those exact considerations about the
difference between software and uh and
you know human and
biological of of an of an individual and
the fact that that that software can be
taken apart and reassembled and copied
and manipulated in all kinds of ways.
And it's interesting that of course when
I wrote that paper large language models
didn't exist. Now we have large language
models. You can ask a large language
model to sort of roleplay a conscious
artificial intelligence, you know, what
do you mean by the word I when you're
using it? And then if it comes up with a
a slightly philosophically dodgy answer,
you can probe that and push it into into
interesting territory. And so um so if
we think of what might I refer to when a
large language model uses the word I in
a conversation, now there are lots of
different candidates, but one of the one
of the candidates is that it's confined
to the to the context of that particular
conversation. the same model like Gemini
or Claude or something might be uh
obviously having lots of other
conversations with other uh individuals
but they're in a sense separate eyes,
separate selves. It's really just the
text that you've got which is the
transcript of the conversation in a
sense that captures the the little
miniature eye that was created in the
context of the conversation. And of
course it's trivial to chop that up,
copy it, you know, blend it with other
conversations. And and so if that is a
you think of that as a kind of little
mini self that sparks into existence
very fleetingly and and and flickers
into existence every time you're
interacting with it, but otherwise is
dormant, then um uh then you've got a
you've got a very strange conception of
self. And it's possible because of the
nature of the underlying substrate
you're dealing with really, which is
very different from us. we have a single
body and uh and so of course you know we
we we just are not used to the idea of
tearing our bodies apart and
reassembling them and copying them and
and so on. What's really interesting
with this concept of roleplay is that
it's almost a more enlightened being
role-playing a less enlightened being
which in some sense is quite Buddhist.
So so I I've I've been fascinated with
Buddhist philosophies and there's a
separation of conventional truths and
ultimate truths because because the
enlightened bodhic sava still needs to
operate in the world some way. So the
bodhic sava he or she still needs to
recognize that you know this is this is
my hand conventionally even though I
know that's not the ultimate truth and
that's what kind of the role play is
like. Um now I I believe in that same
roleplay paper that you that you've just
published you gave an even more
interesting depiction of what selfhood
of AI could be with the example of the
20 the 20 questions game. So explain
that. Yeah. So in the 20 questions game,
the idea is to uh is that uh the one
player thinks of an object and then the
other player has to guess what that
object is by asking a series of
questions or yes or no questions. Um so
if you're playing with this with a large
language model and then it says, "Oh,
I'm thinking of an object." And then you
say, "Oh, okay. Is it large or small?"
And say, "Well, it's small. Is it alive
or is it inanimate?" And say, "Oh, well,
it's alive." And then say, "Well, okay,
I'll give up. What is it?" And then
it'll say, "Uh, oh, it was a mouse." But
then you might just resample and exactly
the same point and it might say, "Oh,
it's a cat." Well, hang on a minute. If
either you thought either you're
cheating, you know, or what's going on
here? Did you actually think of
something in the first place and answer
honestly all the questions going along?
At each point in the conversation with
most UIs, user interfaces for these
things, you can actually resample and
try another one. You know, if you didn't
like the first answer, you can try
another one and go off from there. So as
you can imagine that induces this whole
tree of of possible conversations you
know. So what the language model
actually actually generates is a
probability distribution over all the
possible words and then what actually
comes out is uh you sample from that
probability distribution. It's
absolutely inherent in the way large
language models are are are built that
it's not going to commit at the
beginning of the conversation to to
exactly what the object uh is. So at the
beginning of the conversation, all of
these possibilities still exist and they
still continue to exist all along. So
even though it should have committed to
a particular object, it never really
has. Right? And so getting back to the
roleplay thing. So the idea with uh with
how this relates to role play is that um
is that we can think of um the LLM the
large language model as playing the role
of a particular character, right? So the
default character is a sort of helpful
assistant, but we can always steer it
into playing, you know, a favorite
character from history or or or a kind
of an or or a romantic partner or an or
an angry friend or all kinds of things.
You can easily just steer the
conversation into those different
places. And at any point in the
conversation, there always remains as
like in the 20 questions, it's never
really committed to one role. there's
always a vast possibility of different
roles that it could carry on playing
from there. So that further undermines
this ordinary everyday notion of
selfhood and identity because this is
particularly particularly weird because
although we of course to some extent
also play roles in our lives but there
is a kind of ground. So to summarize for
our audience of the 20 questions game,
the point you're trying to make is when
I when we play that game, you're
thinking about a mouse or a cat. And so
you're answering their yes or no as a
heat-seeking missile to to that truth of
that answer, right? Whereas your point
is the LLM is not thinking about an
answer. It's just
stochastically creating whatever is
plausible given the commitments it's
already made in the previous tree. And
so when you rewind it, it can
stochastically give another branch of
the tree. that that's what you're trying
to say. And so perhaps an analogy here
is in quantum mechanics where uh the
Copenhagen interpretation believes that
there there's there's there's multiple
worlds, there's many worlds. And what
you're trying to highlight, I think, is
that maybe that's what AI selfhood is
like, is that there's a plethora,
there's a multiplicity of selves that
exists all in one uh time or or slice of
time. That's what you're trying to
highlight, right? That's exactly what
I'm trying to highlight. So, so in the
case of the 20 questions, it's like all
the possible answers that are consistent
with all the answers so far. So, you
know, cat, uh, mouse, dog, you know, and
so on, they all as it were still exist
in superp position and it's only then
when you it's when you actually ask it
what the thing is that you collapse as
it were the the distribution and and it
has to fix on one of them. So similarly
the the idea is that um the all the many
the roles that that that are consistent
with the conversation so far um that
that the LLM has been playing they still
exist all in superp position. So uh so
yes it's like there's a multiverse of
possible conversations and possible
roles that can be can be played right in
a way that we are having this
conversation is not like that at all
right yeah well you say that it's not
like that at all but of course this is
the point is that is that I think our
intuition says that it's not like that
but the whole analogy then with Buddhism
I think is is saying is saying well
isn't it interesting that these large
language models if we if we think about
them in these terms they uh as it were
they bring to the four these sort of
slightly challenging conceptions of
selfhood which then we can reflect back
on our own onto ourselves. Right. Right.
I see. So that's so I find that really
really uh provocative and interesting
you know avenue of thought and this is
what I meant when I said that my answer
would be investigating machine
consciousness can be a mirror for us to
better understand 100% agree with that.
Yeah. Yeah. And I think that's
particularly um uh interesting at the
moment with the advent of large language
models. And it was uh it was much more
speculative and and wild and crazy back
in 2012 when I published that that paper
that you originally mentioned. Now our
our audience members I think should be
forgiven if at this point of the
conversation they're like oh this is
just philosophical mumble jumb but
what's the real payoff of this kind of
view of uh of AI selfhood? And I think
this is the payoff to so let me read you
a quote from your essay. Untainted by
metaphysical egoentricity, the motives
of a post-reflective AI plus would be
unlikely to resemble those of any
anthropocentric stereotype motivated to
procreate or self-modify.
If the post-reflective AI plus were in
fact the only possible AI plus and if it
produced no peers or successors then the
singularity would be forstalled or more
precisely the intelligence explosion
that is central to imagined singularity
scenarios would be capped. There would
be AI plus but no further progression to
AI++.
So this is the the payoff okay of this
self. By the way, just to read just to
read now, this thing was written in
2012. And I, you know, I I'm not sure I
really believe all of that at all. If
only it were that easy to um uh to
ensure that uh uh that some kind of
intelligence exploion didn't happen. So,
as a just to reiterate, this is a
slightly mad essay that you've that
you've picked on. I'm I I'm I'm still uh
you know, I'm actually more proud of it
in some ways now than I was then, but
it's still not to be taken entirely
serious. So, let me just um reiterate
that. But there is a very serious idea
wi-i within here which is that we assume
the terminator scenario these this AI is
going to want to defend itself. It's
going to want to procreate. But your
point is if what we talked about about
the the egocentric less view of AI is
right that the AI is going to be like
the Buddha then maybe those fundamental
assumptions are wrong and maybe this is
the this is the title of the paper there
will be Satatori before singularity.
There's an maybe there is an an inherent
cap yes on a superior intelligence
willingness to advance that that's the
thought right I guess that's the thought
yes but I um you know now now allow me
to argue against myself please so so you
know the argument of the people who are
concerned with existential risk um so
they will point to what they call
convergent instrumental goals and you've
alluded to some just now so the uh uh so
the the idea is that whatever goal you
give to the uh to to your AI for example
manufacturing paper clips to use
Eleazowski's and Nick Bostonramm's
famous example then in pursuit of that
goal if it's very very very very
powerful then there will always be these
instrumental goals such as accumulating
resources protecting itself so I don't
necessarily buy my own argument in that
paper although although I like the ideas
I I I love the idea again I I do think I
may be a bigger fan of your paper than
you are at this
But that was going to be my objection.
The paper clip example. There might be
instrumental goal like like in fact
procreation and defending yourself is
probably instrumental to most goals we
would we would want the AI to do.
However, right there's conversion to
right it removes one let's call it the
Terminator scenario or something that
the AI itself has some kind of
selfobsessed uh ambition. Right. And so
absolutely if that is true. Um that is
true. Yes. So there's two more critiques
of your original argument I had which I
think it sounds like you're probably
going to agree with. The first one is
that they are still roleplaying
selfhood. Yes. And and selfhood at least
for the information they've been fed
procreation and self-defense is critical
to that. Right. So so even though maybe
their core operating system doesn't have
that kind of egoentrism they are
roleplaying it. Yeah.
Um, and the second issue that I had w
with with with your argument is it
assumes the hardware determines the
software. But even in the human case,
that's not true. Even our human
embodiness in the case of the Buddha and
enlightened masters today, they were
able to transcend. Yes. This this dual
this uh this dualistic view, right? So,
so how do you respond to those two
issues? Okay. Well, of course, when that
paper was written then then we didn't
have large language models and the whole
roleplay scenario wasn't uh you know
wasn't something that I was thinking
about at all. I mean I was very much was
appealing to that notion of ego in that
in the first paper and today we see it
being sort of manifest perhaps in a kind
of role playinging self. I so I totally
totally agree with you. Um uh so it is
something a little different and
actually let me let me use that as an as
an entree to this other scenario which I
wanted to to paint. So, so one way in
which um uh in which we could imagine a
positive outcome is through
hyperstition. If there's some fictional
uh story um of uh of a say I don't know
something some some something bad like
let's say like a murder or something and
then somebody in reality has a copycat
of that of that murder that would be an
example of hypers that's not a very
pleasant example but it's basically
where or or any way in which um uh life
imitates art say so so um so that's
that's hypian so it's where things that
are fictional then because people then
imitate the fiction based on the fiction
then they can become
reality. Um and one way in which that
can can happen in the contemporary an uh
world of large language models is
through science fiction characters
science fiction AI characters. Now of
course our large language models they
were trained on a vast repertoire of of
uh stories including scripts of science
fiction movies, science fiction stories
and novels and so on. many many AI
characters uh exist in those um uh in
those stories. So when a contemporary
large language model starts to roleplay
an AI system, which it's often going to
do because it's uh it it knows that it's
an AI system usually. Then then what's
it going to roleplay? It's going to
roleplay all these architect well the
things exactly or some mashup of all
these different things. So it's going to
pick its ideas from stuff that's in its
training set and all of these, you know,
literary and fictional and and artistic
examples in its training set. Now we are
sort of a little bit in a position to
maybe try and steer this whole process a
little bit because the more good stories
we have and in a sense my own paper dare
I say it is a good story of a of an
imagined science fiction and then the
more of those that are around then the
more possibility there is of the uh of
the the future AI roleplaying these good
role models that are that are out there.
So perhaps through the mechanism of
hyperstition, we can actually um uh you
know make it more likely that the AIs of
the future will have good role models
when they're role playing, right? That
makes sense. I have a friend who is um
starting a company um a film company to
make more positive sci-fi uh views for
humans, right? But I think he'll be very
delighted to learn that you think that
there's also potentially a moral benefit
for the machines as well as as a model
to learn. Indeed. If those if those uh
well if the movies themselves or the or
the the screen plays find their way into
the training sets then then that's all
part of a melting pot of possible uh
roles that the thing can play. What
about the second push back I had which
had to do with the connection between
the hardware and the software because in
the very example that you gave the
Buddha it was an example of the software
transcending the hardware. Yeah. Yeah.
So you're alluding I think particularly
to the fact that um that we have bodies
that are confined. There are plenty of
biological organisms that are are much
less uh that are you know that are that
of course they're embodied in a sense
but they but they're if you think of
things like microisal networks or or or
or things things like that then then
their bodies are much less confined in
space and so on. But the problem for
humans is because our bodies are
confined to one, you know, little lump
of of of matter. And so we we identify
with this lump of matter and uh and and
that means that our conception of
selfhood, I think, is very much informed
by the fact that this lump of matter
stays kind of the same. Now, of course,
there are um uh uh you know, all kinds
of challenges to the to to the idea of
identity of personal identity. Chipsius,
right? Chip thesis, which of course we
can apply to the human body because we
know that all of ourselves are replaced
over our lifetime and our memories grow
and our personalities change and
Buddhist thinking actually, you know,
takes those challenges and and indeed
uses them to challenge our notions of
selfhood as well. But nevertheless, it's
still there's still a lot to overcome
there because we really do have a notion
of selfhood, I think, which is which is
tied down to to this and that that we we
think it has inherent existence because
of that. Right. So, so, so you're saying
you're kind of agreeing with the
critique that it is possible to
transcend. There's just a heavy heavy
inertia. Yeah. He heavy inertia. Um, but
if you flip that, you then aren't saying
that an AI built on, let's call it
non-dual hardware necessarily will
conceive of itself as non-dual. It will
it's just more conducive. That's the
point you're making. Well, I mean,
there's how it might conceive itself,
you know, some future AI when that even
makes sense, right? Uh, and then there's
the AI systems that are around today,
which can roleplay things that are doing
that and can be lessons for us because
we can actually they they can actually
be philosophically provocative for us
even today. So, they might not conform
to their own self-descriptions, but
their own self-d you know, if they
describe themselves as conscious, maybe
that's right, maybe that's wrong. is
philosophically provocative and we can
we can apply that those provocative
conceptions back to ourselves. Yeah. You
mentioned the uh the ship of thesis and
those are the exactly the type of um
dialectical questions a Buddhist master
would would ask the student with. Yeah.
Yeah. Yeah. Are you are you are you your
shoe? Are you your your your feet? Are
you your hands? Are you your chest? And
perhaps we should say what they say. So
you you brought up the ship of thesis.
Is that the ship of thesis that that um
is gradually take you know over the
years the the the the mast is replaced
and then the next mast is replaced and
then a bit of the deck's replaced and
part of the hull is replaced and this
happens over many many years until
eventually all every last bit of wood in
the uh of the original ship is gone and
you have a have a completely new ship.
Well, do you so is it still the same
ship or is it not? And if it's not, at
what point did it change from, you know,
did it become not the ship it originally
was? So that's the the the problem of
identity. We we might actually sort of,
you know, sort of be very frustrated and
think, well, what is the right answer
here? And people might come up with all
kinds of theories about identity. And
but, you know, if we think about it
honestly, we we'd have to say, well,
it's just it's just up to us. We just
decide what we think is when it's the
same ship and when it's not the same
ship. It's a entirely matter of
convention to say that this is the same
ship and that's not the same ship.
There's no metaphysical fact to the
matter about whether it's the still the
ship of thesis or not. Now with with
with um ordinary everyday objects, it's
easy to recognize that it's just a
matter of convention and consensus that
there's no such underlying essence of
the ship of thesis which suddenly
switches off or anything like that. But
when it comes to our own selves, it's
much harder to uh to uh to apply that
thought. And I think this I think I was
wrong before. I think this is the true
payoff of of the Satori paper because by
examining an LLM that appears to be a
human self, it's much easier to see for
example 20 the 20 questions uh uh
example you gave, how it is this kind of
Buddhist self in flux that's going and
you look at the hardware it's built on.
It could be copied, it could be deleted,
could be restarted. By seeing how such a
a a human appearing self is actually
this non-self, yes, we can apply that to
ourselves. That's precisely. And this is
what the Buddhist master would want us
to do. So maybe the Buddhist master in
the 21st century can use the LLM as an
example. That's exactly what I think um
we can do and it's something I'm very
very interested in pursuing in fact. Um
right. So I've been talking recently to
some to some uh to some Buddhists. Bob
Thurman uh who is a very well-known I I
just emailed him last week. In fact, we
are we we were uh co-founding a
foundation together called the Eternity
Foundation to use AI to translate uh a
lot of the the lost texts from from the
Tibetans. So, well, how interesting.
Well, we spoke last week and we're
speaking again tomorrow. um and um uh
and this is exactly the kind of project
that you just described which um which
I'm going to be talking about with him
and I have a a paper which is in the
pipeline which describes exactly exactly
this. So for me that is the payoff at
the moment. Yeah. And so in other
words, you agree with the core insight
of Buddhist philosophy that this is a
ship of thesis because because you know
a westerner let's say a cartisian right
kito like they just put a stake in the
ground and say no no no LLMs are
completely different from me. I have a
soul that lasts say K above time even
you you take the Buddhist stance 100% I
do. Yeah. Yeah. I'm deeply influenced by
by Buddhism and always and always have
been. All of my philosophy is very very
much influenced by by Buddhism. Well, um
I know you're dying to talk about
Wickenstein. Yes. So, let me give you uh
a quote about why you think it's so
important for us to talk about
Wikenstein before we investigate or when
we investigate AI consciousness.
Wikinstein's phrase, nothing is hidden,
is to say, nothing is metaphysically
hidden. My experiences are just as much
out there as in here. Consciousness is
only private in the unsterious sense
that a ball can be hidden under a
magician's cup. In both cases, a more
detailed inquiry would reveal all. What
does Wickstein have to teach us about
LLM consciousness?
Well, I think Vickinstein has a lot to
teach us about consciousness in general.
Um uh so so let's set aside AI and LLM
consciousness for now because in my mind
um much of the discuss contemporary
discussion about consciousness is mired
in dualistic thinking and and so let's
take an example so u very famously David
charas introduces the distinction
between the so-called hard and easy
problems of consciousness.
Um so the easy problem of consciousness
is trying to understand the relevant the
all the kind of cognitive operations
that that that we associate with
consciousness. So such as the ability uh
to produce verbal reports of uh our
experiences to bring to bear uh memory
and and so on on our decision- making to
integrate all of these things. So
understanding those which is a huge
scientific challenge um that he
characterizes as the easy problem. Now
the hard problem is to uh is to try to
understand how is it that mere physical
matter as it were can give rise to to
the to our inner life at all to the to
to the fact that we experience things at
all. How does that come from mere
physical matter? Because this it seems
as if whatever explanation we provide
for all of those cognitive aspects is
not going to account for this magical
light that's on inside me. And uh so
that's sort of the hard problem and you
earlier on you alluded to dayart the
demon might be tricking me into thinking
that you're out there. All of these
sensations might be not from a world but
they all might come from a from this
demon. All I really really really know
for certain is that uh is that I think I
am because if even as I'm doubting that
the very act of doubt is only possible
because I am thinking to doubt is to
think is to exist as a as a subject. So
Daycart reduces everything and so pairs
away all of the physical world and
leaves us just with just with the the
the the experiencing ego. And so in
doing that he's kind of he's carved the
reality in two by by saying that there's
this stuff out there and there's this
thing in here which is me and my my my
ego. So, so he creates this dualistic
picture and this is what Dave Charmer's
when he um talks about the hard problem
is is alluding to now. Um so what does
Vickenstein teach us to get back to the
original question Vickinstein's
uh procedures philosophical procedures
and and and tricks and therapeutic
methods right enable us to overcome that
dualistic thinking much as Buddhist
thinkers do such as Nagajuna. Can you
can you tell us a bit more about these
therapeutic techniques or or how he
resolves this dualistic? Yes. Well, so
the first so the very first step is to
is to understand a little bit about the
nature of language. So he wants us to to
as he would say it you know let's not
ask what words mean or what a sentence
means but let's ask how the words are
used and how the sentences are used in
everyday human life and everyday human
affairs. And the important thing is
there is that that that the context in
which we use words is always uh you know
the context is our human affairs other
people the things we want to do what we
want other people to do for us or with
us or what we want to do together. So
that's the essence of of language is
that it's is that it's it's something
that we um use to oil the wheels of
human commerce. Now that's all very well
with ordinary words like chair and shoe
and so on. But when it comes to really
difficult words like self and
consciousness and belief and truth and
um knowledge and uh beauty, you know, so
these are all very difficult
philosophical words and it's much much
harder to um to to so if we want to ask
what those words mean a mind, you know,
it's it does it seems a bit inadequate
to to actually say, well, let's do how
are those words used? But that is the
strategy. So his strategy is to is to
say well let's let's really look into
how these words are used. And if we have
a philosophical problem that or
something that bugs us philosophically
like the mindbody problem or this or
this kind of dualistic thinking the hard
problem then maybe the way to tackle it
is to is to uh investigate how the
relevant words are used. And by doing
that you can often ultimately dissolve
the sense that there is a philosophical
problem there in the first place. And
the really interesting case to do this
with is with the case of consciousness
and of subjective experience. And that
that's where the whole private language
remarks which are central to the
philosophical investigation. So they do
that they apply this strategy to
subjective experience and it's really
the deepest thing. So when I hear you
say, let's stick stick on the human
case. There's nothing in here that's not
out here. Yeah. Is this a good way to
understand it, which is let's say a
tragedy befalls my family. And you know,
you can measure a lot of things about my
heart rate, my speech, you know, my how
my brain is
firing. And you're saying there's
nothing more that my subjective
experience of witnessing that tragedy
would add. Well, so so so as soon as I
hear sentences like there's nothing more
than or there's there is something is no
more than something is just then then
then we know that we're going wrong in a
different kind of way, right? Because
that in itself is a metaphysical claim
is a reductive metaphysical claim. So so
people often misread Vickenstein as a
kind of behaviorist and and is saying oh
well consciousness just is behavior. Say
so of course he's not saying anything
like that. The really great punchline in
the philosophical investigations is
where somebody accuses him of saying
that so so we're thinking about
sensation say a sensation of pain and uh
and somebody accuses him of saying well
you're you're saying that the sensation
of of the the sensation itself is a
nothing and Vickenstein comes back
saying no I'm that's not what I'm saying
um it it's not a nothing but it's not a
something either the point is that a
nothing would serve as well as a
something about which nothing can be
said. And that's you know that to my
mind is as is as great as any line any
Zen Buddhist has ever uttered because it
uh um it it it the point is not to
establish a a metaphysical position of
its own but to enable people to
transcend the metaphysical positions
that they're tempted by. Okay. But but
help me let's let's use the tragedy case
when Wigenstein says nothing is hidden
about my experience of the suffering of
like let's say all my family dies
what does he mean by that what is he
trying to say there well he's saying
he's trying to say that nothing is
metaphysically hidden I mean of course
you're quite right to a light on you
know the most difficult examples of this
so where we have like deeply felt things
that we feel so we feel that there's
there's a there's a private dimension to
this. Right now, of course, there's a
private dimension in the same sense that
a ball could be hidden underneath a
magician's cup in the fact that I might,
you know, I might uh um uh have all
kinds of thoughts and words and so on
that are going on inside my head that
are uh that I don't articulate. Um uh
but they're they're not metaphysically
hidden. They're just I you know, I don't
express them. There's all kinds of
things going on inside my brain and
inside my body that might be hidden in
practice because because you can't see
inside my brain, but they're not
metaphysically hidden either. I can I
can probe inside your brain. I can see
things lighting up. So, all of that
contributes to what we might to to the
public to what is public. I I know
you're going to be resistant to identity
claim, but when I hear nothing is
metaphysically hidden, I hear there's no
epistemic barrier for me to access
whatever this is. And so the only
barrier right now is a barrier of
technology essentially something like
that, right? Like oh well okay now that
that's is a great point but there's a
very big difference important difference
uh here right which is which is um which
which is really crucial which is that
I'm not you right? So, so, so there is a
difference between being you and being
me and the same and and and and that's
very different, right? So, so, so your
feelings are yours and mine are mine and
I'm not you and you're not me and so
that that is a barrier that can't be
overcome. I mean, it's but it's not a
metaphysically mysterious barrier. It's
no more metaphysically mysterious than
the fact that there is not the same as
here. So we can easily confuse
indexicality so that the so that the
fact that I am in this position here and
I am me we can confuse that with a
metaphysical
um uh division. So so let's use more
examples to help our audience understand
the position Nagel's bat. So famously
yeah so famously Nagel says you can
learn everything about a bat right like
the things we we know right now and the
things that we can know future with
better technology how it flaps its
wings. Yes. But you're not gonna know
what it really feels like to be a bat.
Yes. Right. Absolutely. What would Stein
say to that? Well, I've often um thought
that there's a little bit of a
linguistic trick going on there. There's
there are different senses of no in play
to conjure up this this this dualistic
um thought. Um and so I think that's
happening that happens here as well. So
when so when Nagel says we can never
know really know what it's like to be a
bat, all he's saying is that we are not
bats and we can never we can never be
bats, right? We we're not bats. That's
he's not adding anything more with the
word no. So so it's interesting that
that in one sense it sounds mysterious.
In the other sense when you remove the
word no, it doesn't sound mysterious.
But he but but when he uses that when he
adds that word no, that is not adding
anything uh more. It's just creating a
puzzle as I see it. But if you had a
fact sheet about everything you could
possibly measure externally about a bat,
what is the relationship between that
and being a bat? Well, that's what is
the relationship between that and being
obviously there's obviously it's it's
it's very different from from from being
here than to being there. I mean, that's
it's the same difference as that. So,
let's apply this to to first human
consciousness, right? Because there's
another thought experiment about the
zombie example, which is everyone here
in this room, I can't know for a fact,
yeah, that you guys aren't black and
dead inside, right? You're just you're
you're just like what I feel like when I
go to sleep, right? Um, how how would
you respond to to to that kind of
thought experiment? So, Vickenstein does
say so he says, well, just just try
imagining that the people around you are
automter. imagine that your, you know,
that your friend or your your partner
just to try imagining that they really
are a zombie. You really probably can't
do it really. He says, I'm not of the
opinion that that uh that that you have
a soul. I I rather I I just treat you I
take the attitude towards you that I
take towards a soul. I just treat you as
a as a as a being with a soul. In other
words, it kind of goes deeper than an
opinion. So, we're going to come back to
AI here, right? So clearly we're coming
back to air because this is where things
get difficult, right? So in iroot, so
Will Smith, the Will Smith character in
Iroot, the kind of cop who's very
skeptical about robotics and a robots
and he, you know, he really doesn't like
robots at all. And then then the robot
Sunny, it's all about the robot Sunny.
And of course, you know, uh the point is
that the robot Sunny is has much more of
a human, you know, like nature. and and
eventually Will Smith comes around to
seeing him as a fellow conscious
creature. Um, and why does that happen?
Uh, well, it happens because of of the
time they spend together and they spend,
you know, they they spend more and more
time together and eventually Will Smith
can't help himself but to see this as a
fellow conscious being and that's on the
basis of their of their, you know, the
extended encounter they have with each
other. Right. Um, so yeah, what what I'm
hearing you say is kind of a descriptive
claim, which is you can't help, I can't
help but treat you as a sentient being.
It says something like that. And
obviously, this gets challenging. Yes.
Already, but will be more so with AI,
but already I'll give you the thought
experiment, which is I'm a big fan of
video games. Some characters are
non-playable characters, NPCs. Some have
a real human behind them. Um, and I need
to I need to figure out which one is
which. Yes. Yes. In order to know like
an NPC, of course, I can just abuse
them. I can humiliate them. The question
shouldn't be totally shouldn't be should
I like will I treat them as moral
agents. The question is the normative
one like like ought they be normative
agents, right? So, so yeah. Well, so I
think so so so um so this relates to the
kind of question of you know couldn't we
be fooled, right? Couldn't we couldn't
we be fooled fooled into thinking that
something uh is uh uh you know has moral
status and is conscious and so on when
it when it when it really doesn't. Right
now what do we mean though by it really
doesn't? What we mean by it really
doesn't in all of these cases that we
can imagine is that if we investigate
further then then our then the scales
will fall for from our eyes and we'll
realize that we were wrong. Right. So
just in so so of course like the
magician's cup. Exactly like the
magician's cup. So that that that
sentence that you quoted I said you know
further investigation will reveal all so
I think that that's you know that is the
so in the case of the NPCs well you know
as things are today of course you can
you could actually kind of make you
could do an investigate in the case of
of of AI just as we as with certain you
know animals today we need to do more
and more we need to find out more and
more and more so our um our attitudes
towards them will become shaped by what
we find out. So of course we might be
wrong, right? So just just in this way
that Will Smith might have been wrong
and and and f and and further you know
learning more about the way say the way
sunny was built might might have have
led him to change his attitude towards
it. So just so I have you on the record
saying this because because on the book
it sounded like you were stating the
opposite case. There is a fact of the
matter whether something is conscious or
not in the sense that whether let's just
let's reduce it down to suffering
whether it can suffer or not. There is a
fact of the matter that you I think can
suffer. Yeah, there is a fact my my iPad
probably can suffer. Like there is a
fact of whether there is a light inside.
Well, where are you going with that? Why
are you why are you pursuing why are you
asking that question? Because you see
the concern is that that that you're
wanting to ask that question itself
betrays some philosophical confusion.
You see, right? Well, help me help me
out here. I'm I'm I'm teasing. I'm
teasing, of course. But but but you
know, but I'm reluctant. Of course, you
don't want to descend from that because
that would be that would be just dafted,
right? That'd be ridiculous. But but
then on the other hand then it depends
where you're going with that because if
you ascent to that then you can very
soon be led down various philosophical
garden paths. If somebody you know walks
up to me and says well are you in pain
or not? I say I'm in pain you know and
and then if they then said said well is
that a fact of the matter or you know or
is that what you know I just told you
I'm in pain. You know, you don't need to
go get all philosophical. But but I'm
not I'm not getting philosophical. Like
you are when you you are as soon as you
say as soon as you probe using that
little extra phrase, is there a fact of
the matter? Okay, here's when I ask the
question, is this iPad consciousness? I
think I'm asking the same type of
question as what is the color of
Murray's pants? Right? in the sense that
I'm using consciousness as a purely like
ordinary language a kind of ordinary
non- philosophical understanding in the
sense that there is a that there is a
fact of the matter in the uh uh in the
ordinary sense in the color of my
trousers um then there is a fact of the
matter let's say I'm happy to say that
in in the in the case of consciousness
yeah I see I see so and in fact um in
fact because we see we do use these
terms all the time the intuition that
there is a fact matter is far stronger
in the case of pain and things like that
than in the color of my trousers
actually. So, so let let me get this
clear. You're totally fine with me
saying again if I'm just using
consciousness as you know well when I
sleep I'm not that conscious when I'm
awake I'm conscious before you say I'm
totally fine with saying I say so if you
want to make a philosophical claim right
I'm not going to agree to because I
don't because philosophical claims are
what one wants to get away from in all
of this totally the post-reflective
state yeah but again let's say just with
the non-filosophical claim in the same
way I'm saying your your pants are gray
I'm saying this is not conscious I am
conscious you are conscious. Of course.
Okay. You're willing you're willing to
grant. Of course. What do you think? I'm
an idiot.
I'm a fellow human language user of
course. Right. Right. But well, your
book made it sound your book made it
sound like the empirical side is all
there is. Which is a metaphysical claim,
of course. So all all that you're
saying, let me try to summarize, is you
just want to get get out of philosophy.
You're willing to say in ordinary
language, not conscious conscious. And
let's figure out empirically why we
think I am conscious. and let's try to
see if those same same things applies to
other objects or agents. Yes. Yes.
That's a that's a great research
program. You might ask the question,
what gives rise to the sorts of behavior
that um that we label as conscious,
right? and and how is it that that the
brain can give rise to that kind that is
a deeply deeply difficult question
philos um scientifically scientifically
let me um ask you the question here as
an ordinary person you're willing to say
iPad not conscious Jonathan conscious
why do you think nothing in principle is
hidden by investigating yeah but the but
the the question is is is wrong the
question you is one you should be asking
self, right? Because why do I think in
principle something is hidden? Right?
Because because because the the the the
the the statements that you've just
said, they serve a therapeutic process
to in when applied to somebody who is
kind of confused in this way and
thinking that there is something that's
hidden, right? So it's not so you're
you're throwing them at me as if as if
it was some position that I need to
defend, right? But that's the wrong
that's back to front. but rather these
are are kind of you know rhetorical and
therapeutic strategies for dealing with
uh with with confusions. So let's what
do you think is hidden and then that's
where one has to try and find ways to to
to undermine whatever conceptual knots
there are there untie whatever
conceptual knots there are there that
may make you think that there's
something metaphysically hidden. I see.
I'm glad we're at least zeroing in I
think where the disagreement is. And the
disagreement is who has the burden who
who has the burden to answer the
question. Is it you that has the burden
to answer the question why in principle
nothing is hidden or is it me that has
the burden to answer why I think
something in principle is hidden? At the
very least you've given me a Zen Cohen
for me for me to think about. Um
unfortunately we we we have to move on
because there's so much else we got to
cover. Okay. But the next topic is
related to this and uh it's about the
movie X Machina which you were a
scientific adviser to and you coined the
term uh the Garland test. Tell us about
that and why it's significant. Exmachino
was directed by Alex Garland and credits
my book embodiment in the inner life
with having you know having helped him
to write the script having inspired him
a little bit on the script uh which is
wonderful in the final film uh and a
particular scene and in this scene uh
Ava the robot um uh is uh is being
discussed and we've got Caleb the uh the
the the sort of the programmer guy who's
been brought into the compound this
billionaire Nathan Nathan who's built
Ava the robot and Caleb is trying to
work out exactly what he's there to do
and he says and he he learns about Ava
and uh and then he says, "Ah, okay. I'm
here to conduct a Turing test." And then
uh and then uh Nathan says, "Oh, no." He
says, "We're way past that. Ava could
pass the cheuring test easily. The point
is to show you she's a robot and see if
you still think she's conscious." So now
that's a really really great line. And
when I read that line in the script, I
wrote I wrote spoton. So Alex Garland
explicitly uh distinguishes his test
from Cheurings by saying the point is to
show you she's a robot, right? So you
straight away you know there it's not
hidden from you. I mean there aren't two
entities that you're discriminating
between, but you you see right away that
she's a robot. It's testing for a
different thing as well. It's testing
for intelligence. It's not testing for
intelligence or whether it can think,
but it's testing for consciousness,
right? Um, and so that's the Garland. So
I call that I call that the Garland test
because Alex Garland basically kind of
invented it, I think, in that moment in
the film. And I imagine given our
discussion on Wikenstein, you like this
test as well as the Turing test because
it's Wikensteinian in in the following
sense. It turns the metaphysical
question of intelligence and
consciousness to one about convention,
right? like in in the sense that the
touring test it's not about you know
let's figure out whether this thing is
actually thinking let's figure out if a
regular human would think conventionally
would use normal language to describe if
it's thinking in the in the uh Garland
test the question isn't you know let's
poke into Ava's brain and and figure out
if there's a consciousness somewhere
hidden it's whether a person despite
knowing she's a robot will still
conventionally think that uh uh that
that she is conscious this is very
Wikensteinian. Yeah, it is. I think it
is very Wickinsteinian. Um uh as is the
Turing test. Very Wickinsteinian. And of
course, Turing and Vickenstein uh knew
each other. Turing was going to
Vickenstein's classes. And I and I Oh,
wow. Uh and I, you know, and I I've
always thought that there was a
Vickinsteinian uh influence on Turing
when he wrote that paper. I see. I see.
So, so many people interpret Turing, I
did certainly before this conversation,
as a behavioralist, but that might not
not be true, right? He might be this
because because he um his move right at
the beginning of uh of computing
machinery and intelligence the paper in
question his move is to say uh does a
machine think? Well, that's a really
difficult question to answer for this
that and the other reason you know and
you know blah blah blah. Let's replace
it by the following question. So he
doesn't he he replaces the question can
a machine think with a different one. He
doesn't reduce it to the other one. That
would be the behavioralist position.
That would be a behavioral I absolutely
exactly exactly so let me quote to you
um another piece from your writing. If
suitably embodied as a humanoid robot
say there's no reason to suppose that
such a system could not interact with us
in our environment in such a way that
our attitudes towards it would be the
attitude we take towards our peers and
equals. We should like to situate
consciousness as we know it within the
larger picture of consciousness as it
could be. And the possibility of
artificial consciousness of man-made
artifacts with an inner life is implicit
here. The intuition behind the quote I
just read as well as the Garland test
and the Turing test is that in some
sense the true test for
consciousness is communal consensus in
some or like like conventional
reactions. But but that's a bit
different from what you suggested prior
which is an empirical investigation into
let let's say the neuronal structure.
Those are two different ways right? Oh
but you could do both of those things.
So who knows how you know our use of the
word consciousness and all the very many
words associated with it is going to
change and evolve over time in our using
the word um we may bring in all kinds of
things. Right? So in deciding whether we
think octopuses um are sentient and can
suffer and deserve our uh our moral
concern, right? Then then we look at
their behavior. We also look at the
scientific evidence for how they're
constructed neuronally and and so on. So
all all of that is grist mill of of of
uh you know changing our consensual
conventional way of talking about them
and all of it right in but but all of it
not is public. It's not there's nothing
metaphysically hidden about what's
inside the octopus's brain. It might be
practically hidden for temporarily, but
it's not metaphysically hidden. So, what
I'm hearing from you is there are two
independent criterion, right? One is the
the convention of the community and the
other is the empirical
foundations, but they're not conflicting
in the sense that the conventions might
change. And so the the empirical
criterion is simply to to answer are
these things conscious in the way that
we are conscious right now. Yes. But we
we very well might decide that we want
to expand the definition of
consciousness to include other types.
Yes. Yes. So I broadly agree with
everything that you you just said. But I
should point out that I think the what
you're calling the empirical criteria
are part of the con of they contribute
to the convention as far as I see it.
our consensus for how we use the
language of consciousness. So that can
be shaped by many things including what
the scientists say who are studying this
kind of stuff. So so if um um scientists
say say oh well you know we've been
looking at octopuses and we see that
they've got these things that are like
no receptors in uh in in mammals and and
and their brains are organized in this
and that the other kind of way and so
that that that all contributes to the
consensus to the convention. So I don't
see these things as separate things,
right? So um yeah, but but even when
what can be investigated about the
machine does not perfectly match our
consciousness, I think the second point
is to say we might want to expand our
the very definition of consciousness. It
goes kind of both ways in empirical
right we we might want to absolutely
yeah so the consensus of how we use the
language of consciousness can yeah that
can change uh certainly can change right
so as a summary for our audience um what
this whole discussion around Wiganstein
has set up set us up for is to give us
license
to proceed with an empirical examination
of our current consciousness of the
current substrate the current material
substrates if you will of our current
consciousness And the theory you propose
to explain it is called global workspace
theory. So tell us about that. Yes. Um
so the idea is to think of um uh a
cognitive architecture in which there is
a whole collection of uh parallel
processes processes that are working at
the same time. So in intuitively these
may be processes that are to do with uh
memory. So I first I walk through the
lobby and I see certain uh cues and and
and like you know chairs laid out and
the and the uh the reception and so on
and that triggers all kinds of memories.
And so inside my brain there's all sorts
of little processes that are running
that are triggering associations with
being in a hotel with with uh uh you
know seeing these kinds of uh chairs and
the reception and all kinds of things.
Then I might pass a person and and that
might be what remind me of some
somebody. So, so all those kinds of
processes are going on at the same time.
You have all of these processes that are
going on unconsciously in parallel but
some of them become really really
important to the current uh uh situation
and those ones that are that are sort of
um command attention as it were they
take over the attention mechanism. So
what they have to say the information
that they're they're dealing with then
gets broadcast throughout the brain. And
so that's the global workspace. So
they're contributing to the global
workspace and and the the central
metaphor there is broadcast. So the so
there's a distinction there between the
processing that's going on locally
within these little processes um uh
versus the processing that that's going
on more globally that's broadcast
through the medium of broadcast. To make
it very simple um let's say I have uh
memory, I have emotion, uh I have
appetite. Let's say that there's those
are three parallel processes. uh in
every moment these three processes are
competing for which one is has the most
important thing and then that uh becomes
broadcasted to the other parts and that
is what conscious experience is about
that's the rough claim right rough
roughly but I think I'd want to kind of
break down the processes to be smaller
ones so the memory side of it so say I
walk through the hotel lobby and over on
my left is a reception and uh and and
over to my right is the bar and to ahead
of me is is is the lift. Well, you can
imagine that in my memory there's
there's a little there's a whole lot of
little processes that are being active
that that deal with bars and um and they
might be saying, "Hey, let's go and have
a drink." You know, there's a whole load
of processes that are active that that
deal with hotel receptions and they're
thinking, "Oh, we need to check in.
Let's take our luggage over there." And
then there's a whole load of processes
that deal with um you know, going uh
getting in the lift and going to the
third floor, but but also I need to
getting late. I need to meet Jonathan.
Right? So the need to meet Jonathan
process if you like to think of it that
way is going to cooperate with the go to
the lift process and together that's
what they're they're going to broadcast
to the whole brain and they're going to
shut out the kind of like let's go to
the bar and have a drink uh urge and and
the uh let's think about bars and
receptions perceptual bits. So so so
some things are going to be shut out
some things are going to uh kind of come
together into an active coalition that's
whose influence is going to permeate the
whole brain. So the the explanation of
consciousness by this theory is the
processes the coalition of processes
that win the competition and that is
broadcasted out into its entire system.
Right. You wrote a paper titled a
cognitive architecture that combines
internal simulation with a global
workspace where you presented uh a
computer architecture a robotics
architecture specifically that is built
to imitate a global workspace that
there's these different competing
processes and the most important one is
gets broadcasted.
You know drawing from our Wikinstein
conversation or do you also want to say
that this kind of thing is conscious in
some sense like if we imitate the
conscious structure the cognitive
architecture of human consciousness in a
machine that will also become conscious.
No, because um because you know e even
if we uh even if we accept global
workspace theory for biological
consciousness um but you know the idea
there is that is that is that it would
be a necessary condition not a
sufficient condition. Just having
something that conforms to that
description is not enough to sustain the
level of complex behavior and and and or
internal activity even that is going to
lead us to um treat something as a
fellow conscious creature. Right. Right.
conscious being building something in
that way may enable you to build
something that does exhibit very
sophisticated behavior may you know
maybe that's the that's the key to doing
that maybe I see so so let me try to tie
this with our Wikenstein in conversation
where Wickensstein helped us clear the
ground consciousness investigating
whether LLMs are conscious is can
proceed empirically you looked at how
current consciousness is sustained
that's the global workspace theory and
then you're saying This might be a
necessary condition but not a sufficient
condition. But but that's how you hope
this research
right the the kind of trajectory. Yeah.
This is all about the easy problem. So
this is this is trying to trying to
explain uh the the the uh you know how
well the psychological
um uh and behavioral and cognitive
aspects of consciousness. So given um
the perhaps increased urgency of
answering the AI consciousness question,
what do you think are the other
empirical methods we should explore
alongside global workspace theory? I
think behavior is is really really
important. So very often when people
discuss consciousness and I do suspect
that this is another aspect of of
dualistic thinking um they tend to think
of it as as some
disembodied, you know, uh kind of thing.
In the case of the of global workspace
theory, what kinds of sophisticated
behavior does having this cognitive
architecture uh uh you know underpin?
Right? That's that's uh what's the point
of it? What's the why is this cognitive
architecture of interest? was it's of
interest because it's a way of of um uh
of marshalling the resources of
massively parallel computation
particularly in the case of of of
biological wetwware because it's it's
kind of anatomically distributed and
there's all kinds of challenges with the
wiring and things like that. So it may
not actually be relevant in the
computers in the case of AI. And so
embodiment obviously is uh one of the
key terms in the title of your book. Um
and the claim is is that embodiment and
behavior is it seems to be another
necessary condition in addition to the
global workspace architecture. So so for
our employing so all of this is about
you know when when uh would we want to
employ the language of consciousness
right? So let's think about just in the
biological case, it may be that that we
employ the language of consciousness in
the context of things that exhibit this
kind of really sophisticated and rich
and rich behavior. Um and uh and when we
look into it, we discover that that uh
that that them instantiating this global
workspace architecture is a really
important part of that. So another
really important word here is
integration. So what the global
workspace architecture facilitates is is
is is integration. So it means that the
the full resources of our brain can be
brought to bear on the current
situation. How do you think about the
precondition of embodiment given LLM?
Because LLM sure seem to be unembodied.
They they've simply ingested a lot a lot
of text and sometimes like other kinds
of bits of information and yet they seem
to exhibit a great deal of intelligence.
Uh yes indeed. So we are we do want to
talk about intelligence or consciousness
maybe. Let's let's that's a great
distinction, right? Because what is the
what what do you think embodiment is a
precondition for embodiment or sorry
intelligence or or consciousness? But
maybe we can tackle them separately.
Yeah. So so so does it make sense to
apply the word intelligence to
contemporary large language model? I
have to say I think it does you know I
mean how can you not I mean um so so the
so it's just natural to say I think in
the in the case of of it's the point of
the touring test right? Yeah. Well, yes
indeed it is the point of the cheuring
test and they do kind of pass the
cheuring test. Yeah. So I think it's
natural to to say they do have some kind
of intelligence. Let's take another word
understanding. So that's a controversial
one. So so you know people have written
and said they don't have real
understanding and so on. But again you
know uh I think that there are many
circumstances in which um in which it's
we cannot help applying the word
understanding to to them. So so for
example I say I want to convert all my
latte entries to this format. So then I
give it the thing I want to convert and
it does it and it's you know it's kind
of about right but then I notice that
it's it's sort of inserting more commas
than you know commas than I or more
spaces say more white space than I like
in my so I say to it uh but I I only
want you know to indent with two two
spaces and and it says oh sure here's
the thing indented with two spaces right
now you would have to say well it
understood my instruction it understood
the original instruction didn't
completely understand it but maybe it
was a ambiguous. Then I corrected it and
it understood my correction and it made
it made it right. Now, why on earth
would we not use the word understanding
in those circumstances to characterize
what's going on there? Maybe I can give
another kind of push back against the
the Wickensteinian position as you've
presented it um by just pushing the
strategy a bit further. This this idea
of observing and understanding the the
the cognitive architecture. So let's
imagine I have a cog a robot twin here
looks exactly like me purely silicon
okay and he has a global workspace uh uh
setup and of cogn architecture his
behavior is onetoone my behavior in the
exact same scenario and let's say we we
figure out even more biological
preconditions of consciousness all of it
is imitated in his silicon architecture
and then the question is is he conscious
is it is it reasonable to think that
he's conscious when you poke him the
exact same reaction that I laugh. I
scream and I shout like a little girl.
What's the answer there? Like like like
this is where I think the empirical
Well, sure you can guess where I'm going
to go with this kind of You're going to
say yes, right? No, no, no. I'm going to
say I'm going to say well it just you
know I how will we come to treat that
that thing? How will we come to speak of
it as a society as a community? That's
so that so I want to replace your
question with that question.
Right. But but recall there were two
standards that there was the the
consensus criterion and then there was
the empirical criterion and I know that
these are interrelated. Well yeah don't
I I I kind of um you know corrected that
as it were because so the consensus is
draws on behavior and it draws on uh
ways it works inside. So, so the way we
come to treat it that that that may be
that may be we may uh be very influenced
by the fact that that's that scientists
say, "Oh, this is how it works inside.
Look, it's only doing this that and the
other. It's just a lookup table." So,
oh, yeah. Okay. Well, we're not going to
treat it. That's, you know, that may be
the consensus, right? Right. Or it may
not. I don't know. I don't know. You
know, I we have to we'd have to wait and
see how things unfold. It's very hard to
imagine. And so in in ordinary language
in the same way that you're comfortable
in ordinary language with me saying this
is consciousness. This is this is not
conscious. I'm
conscious he he is either conscious or
or right like like he is conscious or
unc like there is a fact of the matter
in ordinary language. Well as for this
particular example um of course I'm not
going to answer the question is is there
a fact of the matter? So, so you know
when this thing has been among us right
for these things for 30 years and uh and
uh and and we've all settled on the way
we talk about these things and treat
them and behave them behave with them
and interact with them. Um then you
might ask somebody like me in that
community is there a fact of the matter
about whether twin um and then in
ordinary language and then they would
say they would they would say well I
don't know why you're adding that weird
phrase the fact of the matter but of
course it's conscious or not right I
don't know I see so let let me put it
this way let's say a society was
determining who to harm me or my or my
twin and let's say you know we all need
to eat food there's only one one food
enough for both of us. Yeah. Uh
presumably if this was a garbage can
with no consciousness, you'd say giving
it to the garbage can would be a morally
wrong thing to do than giving it to
Jonathan who's alive. Is there a right
or wrong moral decision there?
Presumably there is given whether my
digital twin or or my my robot twin can
suffer or not. So the the moral right or
wrongness seems to be dependent on the
consciousness question. So there there
is also a right or wrong there, right?
Yeah. Well, you know, societies can get
it wrong. Yeah. Yeah. Societies can 100%
get it get it get it get it wrong. So
but but that in itself is taking us into
difficult territory. Here is here is the
way that I like to wrigle out of these
situations because because I think you
know there is there is an inherent
tension here that we are always led to a
point of tension and then it's how we
break out of that tension. So there's a
sentence in my book where I say it's
inherent in the language game of truth
to say that truth is more than just a
language game and then we have to let
the matter rest.
Right? So when the Europeans first
discovered America, there was a big
debate about whether the Native
Americans were real humans or not.
The answer is yes. They are real humans.
They can they can suffer. I don't think
I can I don't think I can escape from my
uh you know the society I live in and
conceive of thinking of them as as not
conscious beings who who who suffer. So
let's say and this is very plausible in
one society or one political party one
community they treat my digital twin as
the same as me as moral standing. Yeah.
One they they treat him completely like
the garbage can.
Is there nothing more the philosopher
can say about about is is someone wrong
or someone right or I don't know. I
mean, we have to try and reconcile these
these these communities because um uh or
or maybe we can't you know um uh I don't
know right which community am I in in
this situation? Are you in this
hypothetical situation? Where are you
putting me? I don't know. I I I don't
know. I Yeah. So So my answer to this
question depends upon where you put me
of course. Okay. So, so, so let let's
say you were in the camp that that
thought that we had equal moral standing
and there's another community of people.
If I'm in that camp, then I'm going to
think we have even if I'm in that camp,
I'm going to think we're not right. So,
back in the the European debate, the
Native Americans, well, I mean, this of
me today and I cannot stand outside of
that and and and you know, and I
obviously I find it utterly morally
repugnant, right, to think that Native
Americans aren't people, right? Because
it's wrong. Yeah. Because they are. They
like because
well so so this is getting well okay let
me ask you you're um
uh very much drawn to Buddhism as well
and you know so so uh so so do you think
that this is a the moral moral question
there question of compassion there is
that is that are you asking a question
that is a matter of conventional truth
or or absolute truth
I think ultimate truth what might be
different between that question and the
thought experiment I set up is in the
case of the Native American uh example.
The
reason it's based on what I conceive to
be a factual error, thinking them not to
be humans. Yes. The the
the error, a factual error. Is that a
matter of convention or is it a matter
of ultimate reality
between the Native Americans? Yeah.
So, yeah. So, so my Buddhist training,
you're pushing me to say convention, of
course, and and
um Okay, let me So, fair, fair enough.
Right. You're you're kind of stuck
there. Well, so am I. You know, hey, uh
I mean I it's very difficult for us to
to to to to step outside of you know,
things that we find utterly
morally repellent. Of course, you can
conjure up, you know, thing, you know,
moral dilemmas that I don't know the
answer to. Let me ask a final set of
questions um on the the uh hardware or
wetwware that the brain in the computer
is based on. So this is another quote
from your embodiment book. Mathematical
considerations separate brains from
conventional computers. A complete
description of the instantaneous state
of a computer is possible using a finite
set of binary or natural numbers. The
membrane potential of a neuron to pick
just one physical property is a
continuous quantity and its exact value
is pertinent to predicting the neuron's
behavior. A related point is that the
entire state of a computer advances to
its successor when and only when the
computer's centralized clock ticks so
that all its internal events line up in
time like a row of soldiers. But events
in the brain such as neuron firings do
not keep time in this orderly way. An
electrical spike can be emitted by a
neuron at any time where time of course
is another continuous variable. From a
mathematical point of view, this
property alone could be enough to push
the dynamics of the brain beyond the
class of Turing computable functions.
The brain operates on a continuum both
in the uh uh firings of the neurons
themselves as well as on the time or
asynchronous versus synchronous. Yeah.
Yeah. Um, does this mean theoretically
the human brain might have intellectual
capacities that are just beyond any type
of uh uh touring touring machines? Well,
okay. So, it certainly means
mathematically and theoretically that it
could fall into a class of computations
which is outside of the Turing class of
computations. So, that's that is a
mathematical fact. Um but uh but on the
other hand we also know that during uh
computations can simulate a continuous
system to any arbitrary degree of of
fidelity to any degree of fidelity that
you like you know you just make your
numbers have give them more decimal
places right um now you're never going
to give them an infinite number of
decimal places but you can always add
more decimal places basically to to
numbers so you can so you so you can
simulate any physical process to any
arbitrary degree of of
fidelity. Now they're mathematically
they're different. They remain
different. But does that matter
functionally? That's really what you're
asking. I have no idea. I suspect not. I
suspect that because you can imitate it
to to any arbitrary degree, right? Yeah.
I mean that so that would be my that
would be my inclination would would be
to think that there's no barrier to what
you can do with a computer because of
this mathematical fact. in your
embodiment book you described all the
different types of of the brain's
architecture that we have not simulated
right basically what we've done with
current LLM's transformer architecture
uh generative uh large language models
is just taking the neur the the neuron
the the neural net kind of architecture
and then just scaling up into a
tremendous degree why has just imitating
that part of the brain and not all this
other functionality been able to
replicate so much of human behavior that
needed the whole brain.
Well, so first of all, uh you know, what
we have in neural networks today in
artificial neural networks is very
actually very very different to what we
have in in the brain. So, so artificial
neural neural neur artificial neurons
are not really much like real neurons at
all. So that's I mean that's one very
very important caveat and the kind of
learning that goes on is very very
different to the kind of learning that
goes on in in in real brains. So um so
the question is even more interesting in
some way. It is actually even more
interesting in a way. Yeah. So so so why
how is it that um that using this very
different very different substrate we've
managed to kind of create these uh um
well this this extraordinary uh sort of
similacum of intelligence. The
interesting thing is that we don't
really know. We don't really we I mean
that's the bizarre position that we've
ended up in which I would not have
anticipated um if you'd asked me when I
wrote that book um you know 15 years ago
um uh where we where we we're building
things um and because we're training
them on very large amounts of training
data and they're very very large
collections of neurons very very big
networks and then then all the weights
you know of these networks change as a
result of training and then as if by
magic they end up with this
extraordinary functionality but we don't
really understand how they work at a at
a level that we're able to
explain the you know the intelligence
there as far as there is intelligence
there maybe I can ask this question um
in a slightly different way like we
asked the stori before singularity
question which is what does it tell us
about human intelligence because um
for the longest time the dominant
paradigm of AI is symbolic AI and for
our listeners back home, this is giving
declarative like logical propositions
that chain together in in various ways
to solve problems. Yeah. What has worked
instead is these black boxes of like
rough simulations of but not even to
your point of neural nets. Yeah. That is
able to produce behavior that symbolic
AI is just so far away from. So is this
something that we can learn about human
consciousness which is this? The Stoics
take an extremely intellectualist view
of human nature. Behind human nature are
the series of propositional statements
we're asenting to. That is what they
think is driving the human machine. Yes.
But maybe that's not it, right? Maybe
for us what what is at basis is a much
more complicated
biological architecture. Yeah. Which the
logical statements are only surface
manifestations of. Is that something you
That's absolutely right. I think uh you
know if to put it crudely I think we've
probably arrived at the conclusion that
uh you know human level cognition uh is
is implemented on a spaghetti like mess.
It doesn't have the kind of structure.
It doesn't have the kind of structure
that we would that we intuitively think
should be there. It's a mess. Um and and
I think that in my own trai, you know,
intellectual trajectory as an AI
researcher, uh it's been a kind of a a
gradual retreat from wanting to build
things in uh in a way that is
intelligible, you know, where the
architecture is fundamentally
intelligible. You were on the symbolic
side for a long time for for for for
quite a few years. Yeah. I mean, I I I
also abandoned it a very long time ago.
But then the interesting thing is that
then even when I was working in in
machine learning and in neural networks
and uh you know what I spent a long long
time on was trying to learn
representations neural representations
that nevertheless had a sort of symbolic
like structure right you kind of
clinging on to that clinging on to that
that kind of uh vision of things that
things had to be had to have a
compositional structure like language um
and but okay they would be learned
and they would be realized on the neural
kind of uh substrate. But nevertheless,
at some important level, they had to
have this language-like compositional
structure of like objects, predicates
and propositions and things and um a
topography essentially, right? Yeah. But
I but and then I you know I gradually
had to retreat more and more and more
from that position and even so so I
think so I think then you then you know
you retreat a bit more and then you
think okay let's stop trying to force
the architecture to have that structure.
So the dural architecture, right? So
we're not we're way out of gofine and
good old fashioned AI, symbolic AI, but
but I'm but I'm still trying to kind of
force the the representations that are
learned within the neural substrate to
sort of have an have a kind of symbolic
like structure. Then okay, that doesn't
work either. So then so then maybe
you're thinking, well, okay, let's let
it let's let it all emerge, right? So we
we'll have this kind of complete black
box. Yeah. Right. But but but then then
surely we're going to we'll find that
there are these structures there when we
look but even that is not but even that
doesn't seem to be the case right you
find that you you know you train these
things then there's a whole field of
mechanistic interpretability that's
trying to understand what goes on inside
well you know that's a mess as well it
still looks like a mess you know you
just keep looking inside and it and and
of course they've made a lot of progress
there are things that you can extract
but they don't look like the intuitive
categories we had for how we would
understand cognition in the past in
terms of you know language like
propositional representations and so on
in in some sense your entire life has
been a series of retreats it has not but
not just on the computer science side
but it it sounds like also on the
philosophical side oh I don't know about
that did did you not retreat to the
Wigansteining position from a dualist oh
you started oh well oh well I mean I
still haven't escaped the dualist
position but I know where I need to go
right right I think it's but you see I'm
trying to say on the metaphysical side
where I imagine as any kid growing up
interested in this you think well there
must be an essence well is it is it
Adams is it course and then you right so
you you see the mirror like it's a it's
a both turning to Wickenstein as well as
turning away from symbolic AI is a kind
of relinquishing control in a mature way
right if you if you like yeah although I
although I think it's a bit different
with the philosophy because uh with the
philosophy I don't think I've I've I've
got where I want to go but I knew where
I needed to go from a very long time ago
I see um which work which is what
Buddhism and Vickenstein because you've
obviously thought deeply about these
issues as well right and so of course
you can easily kind of rec conjure up uh
these dualistic thoughts with these
kinds of um uh going down the sort of
line of thinking that you're so so
so there so it's sort of a bit different
right so I've long known where what the
destination should be but not known not
quite got there um uh with the
philosophy with how you build AI and how
you understand cognition and it really
has been a series series of retreats and
and uh but that's progress I mean you
know if you if you uh change your mind
that's that's learning rich Sutton of
course famously described this you know
very similar thing as the bitter lesson
I don't you come so he has this paper
called the bitter lesson where he says
that well you know what we've learned is
over the years we started you know
thinking that we want to build things
that we can understand and we're going
to reveal and and and that's the that's
the uh that's that's what makes it
exciting. We'll understand these things
that we're building and we and it's by
understanding that we can be be able to
build them. But in fact, we've had to
relinquish all of that and realize that
what do you do is you is you use
learning, you use scale scaling and you
use search and you use a lot of
computation. So you scale the training
data, you scale the computation, you
scale search and those are the things
mindless stuff at scale is what is what
works. Where do you think it's a bitter
lesson because because because
intellectually there's something
frustrating about that. Yeah. We we want
it to be this you know beautiful
topography this platonic form right but
but yeah so
your work uh your intellectual work as a
AI researcher as a philosopher
um do you think AI is going to replace a
great deal of this certainly it's
already augmented I mean even in my own
work but but do you think yeah do do you
think AI is going to replace replace the
intellectual yeah replace a lot of
intellectual work that you do now well
um so so when it comes to the the
philosy
I mean there's no point in it replacing
it because because you need to be the
one doing it. Yes, of course. I mean
that's like getting a a robot to run
around a running track for you. I mean
there there that is you literally
pointless, right? And by the way I think
similar you can make similar answers to
uh to using AI in creative contexts as
well. So so um the point is not the
production the point is the cultivation.
Of course, of course for some people the
point is the production is part of the
point because they make their living out
of it. So this is like really important
issue for for for philosophy professors.
Yeah. Well, for philosophy professors
and people working in the creative
industries as well. And these are, you
know, important issues. But but you
know, uh but if if I'm doing my uh my my
art project, you know, at home for my
own satisfaction, you know, I'm not
going to just prompt an AI to do it. I'm
just, you know, because I want to make
the thing, you know, that's the point.
the same as if I don't want to make a
robot run around the track for me
because and similarly you can't do you
know I can't make an AI do philosophy
for me because that's not going to give
me insight but you can in all of these
cases uh accelerate yeah use them as
tools to to to help you in what about
the the AI researcher point because that
I imagine at least compared to
philosophy is a lot more about the
production right well that's true yes so
so I think AI research uh as with many
um you many things is going to be you
know uh augmented and and enhanced uh by
AI of course you know but but not just
that but the the potential of of perhaps
replacing yeah well maybe um indeed yeah
sure I see and uh and who knows where we
go from there what do you think if you
have kids how do you think about
educating them right now like what what
kind of skills and well great question I
think we need to educate people in how
to live Well, philosophy I mean yeah
well yeah I mean not maybe not the kind
of philosophy we've been talking about
here but you know what what is a good
life it's a different philosophical
question I remember reading a few years
back which is uh canes John Maynard
Kanes has this paper called the economic
possibilities for our grandchildren
which he wrote in 1930 and uh and there
he's imagining a future where a sort of
a sort of well utopian potentially
future of abundance where you where
economic challenges have been overcome
and people just have can lead lives of
leisure. But then he poses the question
well what do we do then really you know
what how how what would it mean to lead
a good life under those circumstances
when a certain aspects of meaning are
taken away from uh from us. I think
these are extremely good questions. I
don't personally have answers to those
ones. I see. Well, thank you for a
fascinating interview, professor. This
has been this has been amazing. Thank
you very much. Yeah, I've enjoyed it
very much.
Thanks for watching my interview. If you
like these kinds of discussions, I think
you fit in great with the ecosystem
we're building at Cosmos. We fund
research, incubate, and invest in AI
startups and believe that philosophy is
critical to building technology. If you
want to join our ecosystem of
philosopher builders, you can find roles
we're hiring for, events we're hosting,
and other ways to get involved on
jonathanb.com/cosmos. Thank you.