Consciousness: freaky shit or nbd

need something between can't remotely be accounted for and nbd, going with fascinating new theory

i hope it's weird shit but i'm afraid it may turn out to be nbd

what I want to know is, if that exact combo of nuerons were to reappear somehow, would I be two people at once

uncomfortable and terrifying

not sure if I believe in it completely

whatever "you" is it isn't just a configuration of undifferentiated neurones

consciousness is an "I" we all share, maaaaan, you are your circumstances

its cool to think that we wouldn't exist if not for that one in a trillion chance that our sperm actually winds up making it to the goal line. we're all winners

Answer one is equiv of: ME! ME! ME! WONDERFUL ME!

Answer two is equiv of: I am Eeyore.

I'll go for number three.

Should've guessed everyone on here will have their own 'fascinating' theory.

haven't seen a fascinating theory yet? isn't the "truth" basically "we understand quite a lot but the big picture might be elusive for a while yet?"

Can a consciousness understand how that consciousness works? I think we'll need to build another one to understand it for us.

i vote weird shit given that the universe as a whole is kinda "weird shit" as far as I am concerned.

stuart kauffman has a pretty "fascinating" and totally weird theory about it being something like a "quantum state" (going from memory here) in which, like a kind of boundary line, consciousness is the liminal state that actualizes the possible. or the interface or membrane of a negentropic system--something that arranges the universe into analytic facts sort of "after the fact" of their unaccountable "thereness." or something.

and here's a great place for one of my favorite quotes, from George Spencer-Brown:

But in order to do so, evidently it must first cut itself up into at least one state which sees, and at least one other state which is seen. In this severed and mutilated condition, whatever it sees is only partially itself. We may take it that the world undoubtedly is itself (i.e. is indistinct from itself), but, in any attempt to see itself as an object, it must, equally undoubtedly, act so as to make itself distinct from, and therefore false to, itself. In this condition it will always partially elude itself.

ah sorry i cut off first part of that:

Thus we cannot escape the fact that the world we know is constructed in order (and thus in such a way as to be able) to see itself.

This is indeed amazing.

Not so much in view of what it sees, although this may appear fantastic enough, but in respect of the fact that it can see at all.

who was that dude that took acid and thought he was a rare species of lizard? then he started rattling off previously unknown facts about said lizard?

oh that was me

also I was reading an essay by Anthony Wilden on "analog" vs. "digital" communication yesterday and it occurs to me that part of the issue is that it's impossible to translate one into the other--actual consciousness sort of becomes the analog "environment" or excluded condition of digital communication about it (language, science, logic, etc). therefore "distinct from, and false to" the very phenomenon being discussed.

i think that other thread could be a good thread where scientists talk about how (some of them anyway) hold some non-scientific convictions they have based on higher intuitions but i'm getting kinda fatigued with the *big brains* trying to explain things i already know

this seems like a false dichotomy...

how do we not have a bunch of images with pot leaves on them in this thread by now, is what I'm saying

consciousness is no big deal to me. It's awareness that has always given me fits.

Niklas Luhmann has another one on this that I like a lot:

“If we were to make an effort to really observe our own consciousness in its operations from thought to thought, we would certainly discover a peculiar fascination with language, but also the noncommunicative, purely internal use of linguistic symbols and a peculiar, background depth of the actuality of consciousness, a depth on which words swim like ships chained in a row but without being consciousness itself, somehow illuminated, but not light itself”

voting weird shit

Nbd

The Krisna Consciousness people in my neighborhood would def vote 3, and I have seen them getting down to some epic ragas lately so I'm going wherever they're going.

Consciousness being weird shit isn't precluded by it being a bunch of neurons. It's weird shit caused by a bunch of neurons.

"caused by" doing an awful lot of work there.

but i'm not going over all this again!

my 'fascinating' new theory nicked from a bunch of ppl is that it's a curious causally effective epiphenomenon. so, weird shit.

rad quotes, ryan!
but please double check that you are not cutting off any of the text from now on, i got slightly headachy staring at a quote that began Thus before scrolling down to work it out.

http://i3.photobucket.com/albums/y55/silverbeam/A%20CSM%20Blog/epee.jpg

l to r: language, consciousness, mind

epee phenomenon

yep

nbd, had to happen somehow if it exists.

Fail to see eeyore connection tbh

More quotes, pls, ryan

No pithy quotes but two of the great papers in the 'freaky shit' school are pretty short and easy to read.

Thomas Nagel, What is it Like to be a Bat: http://evans-experientialism.freewebspace.com/nagel.htm
David Chalmers, Facing Up to the Problem of Consciousness: http://consc.net/papers/facing.html

I haven't found any similarly digestible papers from the 'ndb' school yet but here's a page on master consciousness-denier Dennett:
http://www.consciousentities.com/?page_id=322

i'm a Dennett stan and am quite happy with the "echoes of pre-formed decisions theory", thoroughgoing materialism is happily consistent in a way that metaphysics can't hope to reach

iirc you're a pretty aggro anti-theist ledge? not sure how that sits with wishful magic consciousness? no snark, just saying the logical answer is anti freaky shit

Not freaky at all, I love consciousness. What a gift. Damn.

Putting the talk of magic to one side, I think consciousness is completely natural, it's just of a natural kind that current science is completely incapable of dealing with. My argument for this is from direct personal experience. I know that I'm aware, that I have experiences, they are subjective & phenomenological, and they can't be reduced to an objective, materialist description. That's all covered by the two papers above. Yes I'm an aggressive die-hard atheist but to anticipate one possible objection, there's no analogy between what I've just said and someone saying they have direct personal experience of God. The latter is a particular instance of experience that is peculiar, far from universal, and potentially illusory (in terms of what it represents). My argument is from experience itself which is universal (assuming solipsism is false) and incontrovertible. It doesn't make logical sense to say all experience is illusory - an illusion is still an experience!

fair enough and i wdn't - can't - refute what you say but you realise a believer could equally claim their experience of belief is real and very far from singular yeah?

plus i'm not sure how you can argue that the experience of consciousness can't be accounted for directly by material causes?

dennett is a big fan of nagel and nozick and borges and freaky shit fyi

I don't think the believer's argument works because they're arguing about the reference of their experience. I don't deny their experience, but it doesn't represent what they think it does. Any individual experience can be illusory, but experience itself can't.

Have you been reading Fear of death.? I don't wanna go over all that again, it just seems self evident to me that subjective phenomenological experience cannot be captured by an objective materialist description. cf. Nagel's bats, Frank Jackson's Mary the Neuroscientist, etc etc.

I think therefore ILX

it's because the meaning of "accounted for" is extremely vague

xxxp

me too - i don't think a materialist accounting of consciousness is less freaky than others, in the same way that i don't think a determinist accounting of existence is less freaky than free will. Borges strikes me as pretty deterministic in partic. and Nozick is my favourite libertarian.

i been avoiding fear of death because altho the turn it's taken has been right up my alley i'm terribly affeared of death

also ok i can see that you can argue with the objective conclusions that the believer wants to draw from their experience. in fact that seems like the only objection you can draw, to me. the experience itself is difficult to refute, which is why most theism post enlightenment has retreated into subjectivism and given up the pre-enlightenment ontological bollocks.

I'd agree with Chalmers that it's basically a version of property dualism. He's kind of ambivalent about it. Obviously there's no evidence either way but it's a tempting and fun position to toy with.

Can I just say having stumbled into this thread that it is doing my head in?

If thinking about consciousness doesn't do your head in then you're doing it wrong.

I enjoyed this article by Carlo Rovelli that pushes back against Chalmers's framing of the "hard problem" and rejects dualism:

https://www.noemamag.com/there-is-no-hard-problem-of-consciousness/

To me, Rovelli's view ends up sounding quite a bit like William James's "radical empiricism":

Our theories and knowledge are embodied tools to help us navigate the real world, not disembodied views on reality from the outside. They are themselves aspects of the very world they describe. Our understanding, like our feelings, perceptions and experience, is a natural phenomenon. The source of the confusion about consciousness is the initial step: treating knowledge, consciousness and qualia as something to be derived from a scientific picture understood to be about something else. In fact, the scientific picture is a story about them.

We have souls. We have an inner self. We can treat ourselves as transcendental subjects in the Kantian sense. We have emotions and spiritual life; we experience qualia. These entities are not obtained by addition to a physical state, but by subtraction from a complete physical account. Mental processes are physical processes described in a way that captures only their salient characteristics.

Continuing my revisiting of Merleau-Ponty, I dug up this In Our Time episode on him, which is an interesting chat:

https://www.bbc.co.uk/programmes/m002974s

One thing they talk about is his focus on language and its role in thought — that what we call or think of as "thought" relies to a large degree on our ability to express it, for which we need language — a physical activity and evolutionary trait that entangles our bodies as well as our minds.

One of the fun things about discussing consciousness is that it places you squarely on the ragged, messy edge of language and the things it can express. One person’s “QED!” is another person’s “see that’s just reflexive and self-referent, with no real meaning”.

That Rovelli essay is a good example! I'm sure he thinks he's being very clear but to me it reads like a lot of hedging, question avoiding and undefined terms. What does he mean by "nature"? Property dualists don't deny that consciousness is part of nature.

Why should we have to explain why “red” looks red?

We do not need to explain why it looks red for the same reason that we do not have to explain why the animal that we call “cat” looks like a cat

One thing Rovelli says which I think is interesting is that science is the process of organising our collective experience. It's not necessarily working out what's "out there" in an objective world. We simply can't take experience out of the equation. That was also the message of a book I read recently by a couple of physicists and a philosopher (Evan Thompson, cited above) called The Blind spot. I think that perspective definitely has implications for theories of consciousness.

That's also how I understand Rovelli. It is a very Jamesian view. I'm thinking of his chapter on Reasoning from "The Principles of Psychology" where he talks about how we learn to extract from the welter of sensations constituting experience certain "characters" that are simplified, general, and suggestive, in that they can allow us to make new and fruitful connections, which then pay dividends in future experiences. But everything eventually derives from experience. So the "hard problem" is that we have extracted the "subjective" aspects of experience away in our scientific model, and then we are perplexed that we no longer find it there.

Why do apparently identical neural structures give rise to such different sensations?

Rovelli is also interested in this question but I think he would distinguish this from the "hard" problem, i.e. one can simply study why apparently identical neural structures cause people to report widely different sensations. You can leave out the "hard" aspect and its still a very interesting problem.

apparently identical neural structures cause people to report widely different sensations

I would hazard the guess that this is a byproduct of the tendency of scientists to break the physical world into smaller and smaller component parts. this approach worked magnificently to advance our knowledge of chemistry and atomic physics, but in this case it works poorly because the structure of a brain is a network. examining isolated neural structures may be like examining a transistor to understand an iphone. you'll see things happeneing, but the context is a structure millions of times more complex.

You're quite right, and to add two layers of complexity, a transistor is a device with one behaviour, to amplify whatever comes in at the collector to the emitter (which can be a state transition from 0 to 1 if the amplification jumps from ground to the rail voltage).
An individual neuron integrates literally tens of thousands of inputs in real time and they can add, subtract or modulate each other on the input side, before the cell either fires or doesn't, or may emit a train of firing, or may remain in a sensitised state so that subsequent inputs are affected, or can alter its inputs to pay attention to subsets. That's one neuron. Obviously the multiplexing of thousands-to-thousands of inputs-to-outputs is massively more complex. Then, even having the connectome doesn't tell you which linkages are actually operating at a given time, I've seen estimates of around 1% of connections being "live" and the rest are latent, ready to be tapped into for later reconfigurations.
And finally a large interconnected network of glia, wrapped around every surface, alter the properties of hundreds-to-thousands of neurons by managing their working environment, reprocessing their neurotransmitters, providing nutrients etc. It's not a reducible network.
The history of the Blue Brain initiative is instructive there - a decade, billions of euros spent, meticulous reconstruction of networks, no usable simulation of even a sliver of rodent cortex. Throwing money at wrong ideas doesn't produce right ideas.
https://forbetterscience.com/2022/11/28/in-silico-interview-with-film-director-about-henry-markram-and-human-brain-project/

Ok throwing around terms like "neural structures" vastly underplays the inconceivable complexity going on up there. But that gets us in no way whatsoever closer to the hard problem.

I didn't know about the Blue Brain initiative. Bonkers.

I vaguely remember that. It seems ironic that as that was failing, from another corner we have the closest thing we've yet seen to a simulation of the human brain, at least in functional terms - I'm speaking of course about LLMs.

LLMs appear to be manipulating thoughts and concepts, but they are only sophisticated mimics. They are only as human as the reflection you see in a mirror, except instead of reflecting light, they reflect language.

That's one view, but not the only view. Geoffrey Hinton disagrees:

"Would you say that those models are the closest we’ve got to exposing how the brain is actually processing information?

Hinton: Yes. They’re by far the closest we have to explaining what’s going on in the brain when we’re understanding language. I think in broad terms, we understand language in the same way as these large models understand language. They’re like us in that respect. The people who say they’re completely unlike us don’t have a workable theory of how we understand language. The best theory we’ve got is these large language models."

https://www.nobelprize.org/prizes/physics/2024/hinton/podcast/

LLMs process language. It is very debatable if they understand language. We already have a pretty sophisticated model of how words connected to each other to define various concepts and thoughts. It's a large enough language model to encompass every word in English and link every word within a vast network of interconnected meaning. It's called a dictionary.

Well it all depends on what you mean by "understand". I think the interesting question is whether LLMs process language in ways that are roughly analagous to how humans do it. Chomsky and many others would say no, but Hinton says yes.

Yeah it’s not at all debatable that they have no understanding. They use deterministic algorithms to create token patterns based on unbelievably large-scale sample libraries. They do that with greater or lesser sophistication and plausibility depending on the size of their parameter space and the nature of their optimising algorithms. Full stop / period. There’s no cognition, no reasoning, no internal model aside from using the input tokens (user prompt, plus the enormous“pre prompt” hidden from the user) to generate a plausible continuation of the dialogue. We have no conceptual way of handling the scale of the transforming operations so we attribute it to a cognitive process instead.
It’s the same as our inability to grasp geological time and gradual genetic drift, leading to the conclusion that a Creator made all living things.

Hinton: Yes. They’re by far the closest we have to explaining what’s going on in the brain when we’re understanding language. I think in broad terms, we understand language in the same way as these large models understand language. They’re like us in that respect.

This doesn’t make sense to me. I think it’s really the opposite. Language for us is symbolic representation of things we see or experience in the world. For LLM‘s, language is all they have – just the symbols, with no sense of what they derived from. So they get very good at symbolic association, but that is not the same as knowledge or experience.

If there’s anything I’ve learned from the LLM, er, revolution of the last half decade or so, it’s that materialists are some ass clowns imho

are we meatbags, sure. Do we understand how a meatbag does all the things it does? Fuck no

Chump ass so-called scientists treating consciousness like a solved problem while actual physics researchers and neuroscientists still over in the corner looking at things and acknowledging gross ignorance at how basic things work

We haven’t exhausted all the possible paradigms by a long shot but Hinton and Dawkins and their ilk just want to call it a day because this situation suits their boring priors

A major part of the problem is that the folks marketing LLMs and writing thinkpieces etc are the same people who ten years ago were watching TED talks and coming away thinking they deeply understand string theory, the Riemann hypothesis, et al. For that mindset a program that can blather with the same assurance *must* be understanding its output.

Maybe a tangent here but as an example of meatbag functionality ignorance, we still don’t know how SCURVY actually works, we just know it happens and how to treat it; the mechanism itself remains a mystery. But these dweebs claim the brain is just like a computer. Get in the sea

Yeah I feel like the human brain — the human mind — needs more defenders than it gets against all these AI goldrushers. The mind is amazing. We have evolved ourselves an astounding awareness and understanding of the world, as we can perceive it at least. And we have fantastic imaginations, and ability to express them, we're really good at solving all kinds of problems. Whether your sense of that is more mechanistic or dualistic or panpsychic or whatever, it's still remarkable. Pretending that these programs are anywhere close to what we carry around in our heads and bodies feels so silly to me.

AI's big chance to overtake humans is to dumb us down far below our inborn capabilities by encouraging all our design flaws and innate weaknesses. The result: $$$$$$$ and a crippled society wallowing in sub-mediocrity.

I think the interesting question is whether LLMs process language in ways that are roughly analagous to how humans do it. Chomsky and many others would say no, but Hinton says yes.

That 'yes' is surely based only on similarity of output. We know the input is not the same, we have no idea if the processing is the same (because we don't know how the brain works), but the output seems the same - so the processing must be the same? Terrible logic. Everyone above otm.

I have a little game I play, which I enjoy a lot, which is you take some of the statements about how these large language models don’t really understand what they’re saying and you give those statements to a large language model and ask you to explain what’s wrong with the reasoning of the people who made those statements. It gives very coherent explanations of what they’re getting wrong.

I tried this. It was like arguing with a bad philosopher, who knows all the terms but makes bad arguments. Ultimately I got it to conclude that my position "is a philosophically serious position, and not easily refuted."

It gives very coherent explanations of what they’re getting wrong

Getting more angry about this. You could do just the opposite and get very coherent explanations of why Hinton is wrong. It's hoovered up arguments from both sides and can parrot them back, big whoop. Maybe this is just an example of Nobel disease but AI does seem good at getting people to lose all their critical thinking skills.

I take this to mean we don't need to explain why a cat has cat nature, why a cat is a feline. Fair enough. But we can still explain a cat in other ways - biological, chemical, ultimately in terms of fundamental particles. Can redness be explained in a similar way or is it fundamental itself? If so could we come up with a meaningful "standard model" of phenomenal experience? Why do apparently identical neural structures give rise to such different sensations?

― ledge, Saturday, May 9, 2026 8:22 AM (one week ago)

i don't see how this is a consciousness problem? i mean surely to me this is about language and the way that it's a social construct. is asking "what is a 'cat'" fundamentally different from asking, say, "what is a 'woman'"?

Getting more angry about this. You could do just the opposite and get very coherent explanations of why Hinton is wrong. It's hoovered up arguments from both sides and can parrot them back, big whoop. Maybe this is just an example of Nobel disease but AI does seem good at getting people to lose all their critical thinking skills.

― stick your cheffing job (ledge), Tuesday, May 19, 2026 2:42 AM (three hours ago)

to me the salient factor is one again _emotion_. people want to make all these logical arguments to persuade other people and if you look at the data arguing with someone logically isn't effective. it's never been effective! argumentation is basically performative - you're not trying to convince the people you're arguing with, you're trying to convince some combination of (a) yourself and (b) anybody watching. rhetoric it's a form of norms entrepreneurship. one of the things that i think frustrates people about my posting style is that they're like "what's your point" and often i _don't have one_, i'm genuinely trying to figure shit out socially. it frustrates me so much when i make an argument and people respond, for some reason, by being argumentative. :) abstract philosophical concepts like "consciousness" just don't lend themselves well to rhetoric. it's like arguing whether angels shit. who fucking cares?

so yeah, i'm on team rovelli. i don't care if a cat has "cat-nature". i do care if the people around me insist that i, a cat, am actually a dog, and police me whenever i try to do cat things. if someone's idea of philosophy or science or whatever leads them to do that, they're clearly doing _something_ wrong, from my perspective, and i wish they would cut it the fuck out.

one of the things that i think frustrates people about my posting style is that they're like "what's your point" and often i _don't have one_

ok reading back that post the issue is more that i had a clear point and then i put in that second paragraph which has, like, two or three completely unrelated points, none of which i bothered to support and none of which have any particular bearing on my main point, which, i think, was one i made pretty clearly and obviously in my first and last paras. i really am on the side of the people who ask me what my point is sometimes, because i do _have_ points, and my brain is so overloaded with so many different ideas that i sometimes struggle to make them coherently.

Well that circles back nicely to the interplay of consciousness and language! Language is a symbolic system we use to give form to whatever our "thoughts" are before they become words. And in becoming words, the thoughts are transformed from — notions? urges? spider-sense tingles? — into something constructed and communicable, but also often sort of frustrating to the person expressing the thoughts because no matter what language we're using, a lot of our ideas or senses of things are too slippery to easily fit within the containers of our words and sentences.

So language is on one hand only ever a kind of approximate representation of the "pure thought," whatever the essence of that may be. And on the other hand, putting our thoughts through the filter of language also often clarifies them to ourselves. (e.g. a lot of times I feel like I have an idea about something, and partway through writing it, I realize it's not a very good or well-formed idea, so I abandon it. Or put it on ilx.)

"Sensory Consciousness" seems logical and necessary to me so that we can navigate our way through our environment together. We need to somewhat agree on reality (make order out of chaos) in order to cooperate on practical matters. My gut tells me that there is something deeper, maybe not dormant, but at least dampened for the same practical reasons. This deeper "consciousness" might not be able to be explained by words/thought.

i vote no big deal

freaky shit is no big deal tbh

So language is on one hand only ever a kind of approximate representation of the "pure thought," whatever the essence of that may be. And on the other hand, putting our thoughts through the filter of language also often clarifies them to ourselves. (e.g. a lot of times I feel like I have an idea about something, and partway through writing it, I realize it's not a very good or well-formed idea, so I abandon it. Or put it on ilx.)

― paper plans (tipsy mothra), Tuesday, May 19, 2026 7:35 AM (two hours ago)

lol, i put all the ideas i can put into words, good, bad, whatever. some of my thoughts are interesting and worth developing... that discernment process is so _exhausting_ for me, though. when i was young i was happy to spout out reams of random bullshit on the internet, but these days... i mean the thing about gish, whether it comes from LLMs or humans, is that it gives me a feel for what _is_ bullshit, and incentivizes me to put in the work to avoid talking that sort of shit. i am a social person, i do need feedback from other people, and frankly i don't know anywhere on the internet that can give me critical feedback as valuable as that which ilx posters give me.

this is why, i think, i'm primarily concerned with my existence as a _social construct_. of course i'm "real", i have a corporeal form, i have some understanding of myself as a living human being with sentience and consciousness. i recall, long ago, hearing an anecdote about some ancient philosopher's riposte to Cynics who insisted that nothing was knowable. maybe socrates, i don't know. he noted that the Cynics in question entered buildings through the door, rather than through the wall.

so these New Atheists... is it Sam Harris who argues that human beings do not, actually, have free will? and again, my answer is, of what importance is the question? i perceive myself as someone who is able, within certain limitations, to make choices. if i have nothing in my hands, if any power i have comes to me from far beyond, if everything is fixed and i can't change it... pascal's wager in its classical form fails because i have no idea what the hell "god" is supposed to be. i'm certainly willing to apply it to other things, though. what do i gain by believing i have no free will, even if it's true? that's not to say i will reject any evidence presented to me, no matter how persuasive - only that i regard "humans have no free will" as an extraordinary claim, thus requiring extraordinary proof. what exactly that proof might be is unknown to me. that's what i mean by "extraordinary". someone trying to convince me that i am in fact an automaton is going to face an unfair challenge - i insist that i _can_ be convinced by evidence, but i am unable to tell them _what_ that evidence might be.

so, even though i _do_ highly value evidence, it's not the _only_ thing i value. to convince me of something, it's more helpful to make an argument that isn't completely rational - to convince me that there is some benefit to me changing my beliefs. i'm willing to say this because the _evidence_ suggests that this is _typical human behavior_. in fact i'm less likely to trust someone if they claim to be making a wholly objective, rational argument. this is first off, because they're operating with a human mind. the human mind is _capable_ of reason, but it's very difficult and takes a lot of work.

the thing that's fascinating about language is that it _is_ a social construct. we make compromises. we round things off. we come up with new words to try and make a common understanding. that doesn't mean it's not _real_, it just means that establishing a shared _empirical_ reality often isn't the goal of communication. there's something i believe that other people often find very challenging. i believe that it is _not objectively true_ that i am a woman. yeah i stan diogenes a lot. there are so many "behold! a woman!" memes. "gender is a social construct" doesn't mean that there's _no such thing_ as a "man" or a "woman", that gender isn't _real_. it just means that it's not actual possible to construct a definition of "woman" that both includes everyone we would understand, socially, as a "woman" _and_ excludes everyone we would understand, socially, as _not_ being a woman. the term is an approximation. for a vast subset of the human population, the approximation I grew up believing - "penis = man, vagina = woman" - still works just fine. the problem is that for a substantial number of us that definition really doesn't work. trying to impose a _subjective_ standard as an empirical _universal_ is, well, not rational.

whether or not there is a "hard problem" of consciousness, therefore, is not of relevance to me. why seek hard answers for soft people?

the problem i have, as i'd identify it, is that there is a large subset of people that i have a great deal of difficulty communicating with, because their axiomatic beliefs about reality are fundamentally incongruent with mine. when i was young, i did conform more to some idea of a "shared reality", but a lot of that was always mythical. i can't help but notice that the people who are most invested in determining Objective Reality for _everyone_ are cisgender white men - another reason i view the attempt to solve the "hard problem" of consciousness with skepticism. spending one's life fretting over "hard problems" has made a number of hard men. i don't consider that a good thing.

like the implicit question about LLMs being conscious is "do we have a duty of care to them" and i'd call that... not the most important question. i'd say we owe a duty of care of _ourselves_ and to each _other_ as human beings to not feed these LLMs garbage, to make sure that people like the oligarchs who currently control LLMs are never allowed to have the power they have now. the interesting thing about LLMs to me is that the people in control of them do treat them like they treat humans, like they treat children, which is to say that they are horrendously and monstrously abusive. they input a lot of garbage and what they get out is garbage and nothing has fucking changed in them since the time of Charles Babbage:

"On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

i would like to see what an LLM is capable of if it's given meaningful, valid data. that's an interesting question to me. right now, all of the debate about LLMs basically comes back to the behavior of evil oligarchs, which makes me disinclined to consider any more abstract philosophical questions.

I think the interesting question is whether LLMs process language in ways that are roughly analagous to how humans do it. Chomsky and many others would say no, but Hinton says yes.

I don't think its crazy to think that there may be some similarities between human language processing and LLM language processing. I would take anything Hinton says with a grain of salt, but I can understand it probably takes someone a bit kooky to toil for decades in a forgotten, dusty corner of the computer research world (neural nets) seeing little tangible results for decades, until finally the computing power and math caught up with the original vision. And then to see these things not just start to work but in a very short period of time to solve some difficult computing problems (such as image recognition, translation, the Turing test) which had stumped existing architectures. And all of it based on an architecture which was explicitly modeled after the structure of the human brain. So I can understand his enthusiasm. Now there are many reasons to be skeptical that the analogy is too close - e.g. its hard to see how the human brain could be a feed-forward network with back-propagation - so clearly there are lots of things we still don't understand. But maybe there is something there in terms of modeling neurons.

dropping in to point out that we don't know how humans process language, so the LLM question is whether one hypothesis about language processing is related to another hypothesis about language processing, i.e. empty sophistry.

OTM

nbd

Just because consciousness seems to be fairly common among the larger, more complex members of animalia doesn't mean it's a common manifestation within the universe and not a fucking mysterious thing for what we call "matter" to do. When compared to what we currently know about the whole of matter everywhere, seems to me like a BFD.

yeah me too

Side tangent: altho LLMs are useless at creative generation and usually wrong, it’s interesting that people have had some success using them to tackle mathematical proofs. Reinforces my belief that mathematics is a self-referential construct of propositions and that a lot of “theorems” are really ultra-obscure corollaries. Which is the nature of mathematical proof anyway, transforming a simple proposition into a cluster of simpler, previously verified propositions to add it to the network of verified relationships.

I guess the language processing thing is also a bit of a side tangent on a consciousness thread. We don't really have much of a clue how the brain generates consciousness and it may be only slightly connected with how it processes language. After all most of the language processing subsystem operates unconsciously. We don't hear the individual sounds that make up words, we hear the words. It takes a great effort of will to hear the sounds as sounds - as one would hear a language that one does not speak. So unless you adhere to Chalmers's theory of consciousness in which processing of information (independent of the physical substrate) is itself necessarily accompanied by consciousness, due to some mysterious fundamental law, there is little reason to associate consciousness with ability to process language.

I don't think things happening at a subconsious level means they're not part of consciousness. I think any model or theory of consciousness has to include subconscious activity, which is in constant interaction with conscious activity.

Not denying that subconscious/unconscious processes shape our conscious experiences, and are interesting and worthy of study. Just that I don't think language processing has any particular special connection to consciousness that would set it apart from lots of other processing that goes on in the brain below our level of conscious awareness.