x28's new Blog

Once again, the discussion comes up if AI can have a consciousness, and so I’ll try to find my own stance.

I’ll try it by asking myself: if an AI told me a bit about his consciousness, what would it take that I’d believe him, that his consciousness feels sufficiently similar like mine?

Note: Since I am not philosophically trained, I cannot talk about what consciousness is but only how my own one feels and how I imagine someone else’s. (We cannot ‘grasp’ it, anyway, like things from the outside world, but only experience it from the inside.) And I mean ‘consciousness’ in the simple sense as the opposite of fainting — I don’t mean something like ‘consciously thinking’ (about something, which is kind of meta, and again from the outside).

Descriptions of consciousness often involve a distinction between the outside world and some inner world. For example, seeing a scene and remembering it, seem cleanly distinct. But in practice, the sensory part and the memory part are much more diffuse and blurred: outside a small spot of sharp vision (fovea), we need to add a lot from memory to get the impression of a consistent sharp picture.

(See my previous blog posts here Human zoom and Working memory.)

For me, these added memory bits are an important ingredient of consciousness.

So when papers are talking about the physical experience of an AI, I don’t think of giving it cameras and gauges of battery fill-states and link these to something simulating its amygdala.

If an AI’s consciousness should ‘feel’ like mine, it should not feel like a tin box fixated in a corner, but it would in particular need a similar sense of space as we have. Vision trumps all other senses, and our own place within the visible environment is IMHO crucial.

Furthermore, it would need a sense of significant time spans instead of its notorious lightning response times. Already the visual memory surrounding the focal point is time-dependent since it quickly fades away. Likewise, audio depends on time. If we want to even think of AIs gaining ‘wisdom’, we associate that with especially much time.

So I think, a sense of time and a sense of place would have to be implemented somehow artificially before I could believe that the AI feels something like a consciousness — albeit ‘artificial’ would still mean ‘alien’. Without time and space, it could not feel motion — and that’s probably why the archetype of a non-conscious being is the rock who cannot move.

The sense of space and time would align with the cited paper which says “physical reality is inherently continuous”. (The paper also speaks a lot about ideas like abstraction, isolating ‘invariants’ and concepts, and ‘alphabetization’, which I cannot sufficiently judge, but they seem to me as biased towards the fixing & isolating mode of brain operation, rather than the all-at-once mode of a network configuration.)

Anyway, compared to AI’s ‘understanding’ of language and concepts, their sense of time and space would feel much more artificial and alien.

But the most important ingredient, IMHO, would be a sense of subjectivity: feeling as an individual among its peers, not as a copy of a standardized series. For thought experiments creating conscious AIs, it would probably be necessary to raise them by other alien individuals (which again needs a sense of time).

This interesting post by Nik Bear Brown (via OLDaily) talks primarily about students’ trust in AI, as a “variable”, to be “calibrated” — which causes Downes to “digress”:

“I do have an issue with defining ‘trust’ in any useful way. But I digress.”

I agree that there will be no universally accepted criteria of what counts as trust. I think this is very similar as trying to define ethical behavior for everyone. It is something that cannot be defined, but everyone can recognize it on their own.

It also depends on the indiviidual’s varying inclination to trust someone, and it is not a numerical variable to be gauged, but it is present or it is (beyond a certain ‘threshold’ mentioned in the post) broken.

Bear Brown’s post elaborates on the problems that students have with their institution providing the AI, on Self Determination Theory, on dishonesty and surveillance, so it it also about how much the institution trusts their students. In this context, I dare to point to an old CCK08 post of mine which may be anachronistic but mentioned how much Humboldt’s idea of education had to do with trust.

As far as the trust in the AI itself is concerned, I need to reiterate my distinction between a tool and an assistant: trust in a hammer is much a less delicate thing than trust in an assistant.

I’m still grappling with the idea of ‘salience’ and ‘relevant similarity’ from a recent comment and my previous post.

I understand the salience and relevance of overlapping ‘features’ and the resulting maximized similarity when it comes to recognizing a given object as a ‘something’ in the ‘somethings’ category, or for determining its essence, family resemblance, or even for recognizing a pattern and context from tacit knowledge.

But I think there is also another type of salience: striking and outstanding, and ‘prominent’ in the sense of protruding, not like competition and ranking, just noticeable as a contrast from the surrounding items, or ‘leaping’ (as its etymology suggests) into one’s attention.

In the diagram below, most rectangles share some ‘features’ with many others: the shape, position, and number of their chopped corners. These relevant similarities constitute 6 classes which I have colored for easier comparing. Two rectangles, however, stand out from their surroundings because of their skew slope: a blue one on the left half, and an orange one on the right.

The similarity between these two outstanding rectangles is not being measured and compared with other similarities — it is just recognized as striking, and in this sense ‘salient’. The similarity is not actively searched for and not evaluated by calculations, but it is discovered with a more passive sort of attention, often in a serendipitous way. The different ways of associating might be like different ‘temperature’ settings in AI, i.e. more or less creative, or wide vs. narrow context.

Furthermore, the distinguishing feature (the tilt) seems relevant because of the suddenly recognized association. So it is not always easy to say what came first: the relevance or the association. Maybe it is ‘chicken and egg’ kind of problem?

The striking salience type of associations is IMHO important for connections between different knowledge domains (as indicated by the two halves of my diagram). While classes and categories use detailed and precise comparisons, the distant associations are more like metaphors, fuzzy and illustrative.

(And while the classifying type of associations works with ‘abstractions’ which ‘strip away’ some less relevant features, a metaphor connects two concrete domains and brings to mind both of them instead of none.)

I keep emphasizing these distant associations (both similarities and distinctions) because they are important for critical thinking, see here (5 page PDF).

Recently I had several occasions to think about being ‘conscious of ‘ something or ‘consciously’ perceiving or experiencing something. Most recently here and here, but in particular in Stephen Downes’s 1990 opus The Network Phenomenon (TNP)

(which is a difficult read because, as a doctoral thesis, it complies with the rules of the ‘craft’ and bothers to refute all sorts of objections — some of which sound like their advocates raise them just for the delight of a ‘witty’ debate, or maybe just defiance. Mostly they are about concepts and rules that were innate instead of learned; for laypersons like me I would recommend to initially skip most of this debate and start with section “A. The Structure in Review” in Chapter VIII.)

I came there to learn more about ‘relevant similarity’ (more on that later) but already in the chapter on Perception, I stumbled upon a discussion about what it means that we are “not consciously aware of the actual input activation” of neuronal units, and “what does it mean to separate a person or a consciousness form that person’s experience?”.

1. Aboutness

Now when I imagine me consciously perceiving a single phenomenon, i.e. being aware of my perception in some sort of ‘meta’ mode, I don’t think of the whole of consciousness. which feels much more immediate and immersive and like the unique indivisible interface to the outside reality context.

(Maybe this distinction is stronger in German where ‘conscious’ is ‘bewusst‘ (from ‘wissen’ = ‘knowing’) while ‘consciousness’ is ‘Bewusstsein‘ (from ‘sein’ = ‘being’), and the simple noun for ‘consciously’ would be ‘Bewusstheit‘, something like ‘*conscioushood‘, or perhaps ‘conscious–ness’ with a hyphen).

Probably my difficulties arise because consciousness is a notoriously difficult concept (IMHO because we want to grasp it like all other real things, namely from the outside, but we only ever feel it from the inside).

Anyhow, it is surprising to me that this term appears very early at the very low level of neuronal activation (BTW same with ‘cognition’ e.g. in “cognitive cells”). And this surprise is not because I were expecting something like an invisible “mythical ‘eye in the sky'”, as I repeatedly emphasized.

Rather, the aboutness, or intension, that is expressed by ‘conscious of ‘ something, sounds to me like one of the two fundamental ways of attending, and ‘consciousness’ sounds like the other. One: isolating, fixing, focussing, grasping from the outside — the other: holistic, simultaneous all-at-once, immersive, from the inside. One: single data points — the other: multipoint patterns. That’s why I think consciousness is on a ‘higher’ (multi) level than ‘conscious of ‘ (single) perceptions.

2. Relevant similarity

Now similarity, as I understand it, happens on different levels, too, and as TNP chapter VI explains it in detail, overlapping vectors of neuronal activation play the bottom role here. And ‘relevant‘ similarity is illustrated as a measure of how strong the similarity to another item is, in a given context (leg of a dog, here), or in particular, in the context of creating categories, prototypes, or ‘essences’, in the philosophical TNP text.

I have seen similarity on a more superficial level which is easier to understand. Instead of measured relevance, I think of ‘salience’ in the sense of something suddenly striking (as its Latin root indicates which means ‘leap’), because this is how it feels when an association between two concepts suddenly happens, sometimes even with an Ah-ha effect, and sometimes in a reversed way in that a difference becomes apparent.

In a comment on my blog post about Partial Patterns, Downes describes the “similarity calculus” in terms of “features“, and it makes perfect sense to me that such subsymbolic units must be involved both in human neurology and in AI’s weights.

Now I wonder if these features are on a conscious level and can be identified? Despite these parts of a concept cannot be named, because they are subsymbolic?

My guess is that we can scent them out through their similarity connections which they are part of, and we can be conscious of the various similarities and distinctions even though we may not become conscious of the connections’ end points (or ‘ports’, as I called them previously).

I think this would align with the connectivist idea that knowledge resides in the connections, and I wonder whether a bit more of ‘explainable AI’ would be possible (at least for general-purpose language), if these connections between concepts were not only buried in giant black boxes but were revealed and could consciously be accessed by us humans.

Trying to respond to two recent questions from Stephen:

“What does the world need me to do?” (here) and
“Thinking more about why there’s been such a strong reaction against AI in academia and on the left…” (here)

‘The world’ has a problem with human intelligence. Many don’t see it as a gift that brings obligations but as an asset and a privilege that they need to defend against the ‘stupid‘ mass, and now against the machine competitor.

The ‘left’, properly so called, should of course fight against the capitalist AI’s power on the labor market. But being intelligent, many are distracted by the above defense of their role.

Familiar concepts like intelligence, thought, and cognition are tumbling, and need intuitive adjustments, compatible with actual capitalist AI, not with theoretically thinkable AI. Interested parties of both sides of the polarization do their best to sow doubt and confusion — as in the recent claim by Riley (via yesterday’s OLDaily) that “human thinking is largely independent of human language”.

Philosophers are experts in precise distinctions. We need one who makes them understandable, and robust against populist redefinitions.

Inspired by Stephen Downes’s questions about cognition as pattern recognition, I think I should visualize two important mechanisms: recognition of similarities and of distinctions.

For details, go to my ‘ontology’ attempt where the concepts are depicted in greater context, but with the drawback that they had to be squeezed into the RDF triples of subject – predicate – object.

Please tell me if you have suggestions, or create a Pull Request on the Github repo..

As my readers will probably know, my favorite question about an AI application is whether it is an instrument or a service interface. Another possible distinction is similar: when we work ‘with’ Claude, is it

• the ‘with’ in the ‘instrumental case’ (as in ‘with a hammer’),
• or in the ‘comitative case’ (as in ‘with my colleague’)?

But people’s work styles are varying, and some will call the former a ‘tool’ but mean a service which does the work for them, and some will call the latter a ‘collaboration’ and mean that the other one does all the work for them. Our attitudes to collaboration are particularly different. Some love the great deal of dialog that is necessary, while for others it is a challenge to verbalize all the expectations and approaches, and this is more than ‘prompt literacy’.

Now working with Claude feels indeed much like a collaborative dialog. And unlike other generative tasks, programming does not suffer from the problem that the artificial dialog partner does not bring a subjective individuality beyond the averaged stuff.

When we put together a bespoke collection of features, the whole project can indeed feel like the human is in control as (s)he is wielding the small delegated service parts like hammers and screwdrivers.

In this scenario, offside of all distractions of the polarized discussion, I think the centuries-old question becomes much clearer: Should a technology do a task that it could do?

Of course, a dredge could transport my breakfast bag, and then the dredge should not be blamed for its energy consumption if someone does that. I won’t argue about the absolute amount of climate cost because it seems that few people know it exactly and misinformation abounds like with previous denialists. I still have the impression that a lot of waste is happening relative to what might be saved. Already the fact that the entire behemoth of language recognition is done in the same brute force approach as the domain specific details, seems inapprehensible to me.

But useful things can warrant some cost; it just depends. After all, the human brain, too, costs. It’s ~ 20% of the total body’s consumption — how many sandwiches more do I eat if I do the coding myself?. And the opportunity to create ‘vibe’-coded solutions for long tail needs, sounds very tempting.

Practical experience with the current (gold-rush flavored) reality is not (yet?) so optimal. In particular, too much delegation will probably lead to a lot of technical debt in that the code will be less maintainable (if I revisit my own code after some time, it is hard to recall enough, and code designed by partner Claude is already difficult to navigate right away). It will also lead to much mindless trying because it is so quick (how often did I try by flipping a binary option instead of thinking). And finally, it will produce tons of throw-away pieces of very limited value.

I have great fondness for individual niche coding and idiosyncratic solutions. But as I said re Long tail needs: the users representing the ‘long tail’ should leverage their power of being many. They could press the tech barons for better solutions. Instead of inventing them countless times anew. (In Kin Lane’s car analogy, demand better bus services).

As a reviewer, Daniel Buschek noticed some issues with papers where LLMs have been asked to help. I particularly like the fourth and last one: “Stretched Result Summaries” which contain “interpretations [that] come across as surprising, unfounded, or far-fetched.” because it resonates a lot.

Firstly, it resonates with my own observations and thinking about emphasis in summaries (see my picture here). Original emphasis and subtle nuances may get lost, and as his post confirms, nuances are created that don’t exist in the original.

Secondly, I know this problem from long before AI, when we tried to get our docs ‘polished’. I know my weakness that my writing is often too clumsy, not only in the foreign language, and we tried to get help from other colleagues, including a language specialist with a doctorate. The problem was just the same: the rewritten texts were not just polished, but significantly altered, like ‘varnished’, so to speak.

My wish then for the colleague was, and now for AI polishing is: don’t instantly alter the text, just mark the passages that are bumpy or make readers stumble or slow down.

Of course this wish aligns with my previous suggestion that the summaries should give me the topics and themes but not full propositions. And furthermore, it aligns with my general stance about tools that they should be instruments that I wield like a hammer and not a service interface that does the thinking for me.

Someone on LinkedIn is drafting (in German) a Science Communication FAQ for the “general public” about: How does a chatbot actually work, psychologically speaking.

He uses the usual “statistical patterns” belittlement, and compares the “impression” of understanding with the Chinese Room Scenario.

If Science Communicators don’t understand the difference between ‘rules’ and ‘patterns’, and the similarity of human ‘statistical patterns’ and AI, they won’t convince the ‘General Public’. Maybe they just widen the gap, and eventually, they may contribute to increasing Science Denial.

During the past week, I watched presentations of a free online conference, the Global Visual Thinking Conference called “Sketch your Mind”. While I knew more than half of the speakers and own books of 4 of them, I still learned a lot.

But in particular I realized how different the ‘visual thinking’ is that they all must try to cover for their clients and readers:

• two kinds of users: object visualizers vs. spatial visualizers (see Kosslyn for the difference),
• two stages: sensemaking vs. presenting,
• and two expectations that people have about their thinking tools: instrument (which they wield like a hammer), vs. service (which thinks for them).

For the “world-class leaders in visual thinking, personal knowledge management, design, and more”, it is obviously necessary to cater to all of these 8 variations.

For me, by constrast, only one little cube is interesting, the bottom left foreground part marked with red outline: spatial viz, sensemaking, thinking myself with tools just as instruments.