https://michaelgogins.tumblr.com/rss

A few days ago I installed and subscribed to Cursor, a branch of Visual Studio Code that builds in agentic artificial intelligence. Agentic means that the AI doesn’t just propose things, it does things, e.g. creating branches in one’s local repository, rewriting code there, and committing that code. When things are working you can ask Cursor to push it to origin.

Note: Cursor is not at all agentic in the sense of having its own goals!

I am finding that Cursor is extremely useful. There is grunt work in branching, rewriting, releasing, etc. Using Cursor rather than ChatGPT for this kind of work enables me to work several times faster. This is a big deal, because it shrinks my grunt work and opens up more time for my real work.

It’s also kind of scary. I get the feeling I am riding a tiger. It seems likely that this kind of change is happening all across the software development landscape. Yes, some people will lose jobs. But there will be different jobs, and perhaps better jobs.

The scariness comes from the thought of getting lost – falling off the tiger and landing in the swamp of slop without a road home.

P.S., it ain’t cheap. Obviously Cursor is thinking harder, furiously harder, than GhatGPT to get these results. My prepaid tokens are shrinking rapidly!

When I think of the medium to long term consequences….

There seems to be a risk that work will change in a way the enriches and empowers a new class, cut out of the upper tier of today’s upper middle class – but not from the true upper class. Kind of like the yuppies of the yuppies!

If I were 35 I might have a shot at joining this new class, but I am almost 75. What will happen to the rest of us?

If this functionality extends to military science, and I am sure desperate efforts are being made right now to do just that, there is a real risk that one power will move quickly enough to gain such an advantage that it effectively dominates the world.

I see such advantages as being possible in at least these ways:

• Hacking adversaries for complete intelligence.
• Hacking adversaries for sabotage, even to disable their weapons systems.
• Enabling agentic drones.
• Scaling up mass production of such agentic drones.

Let me repeat this thought: There is a real risk that military AI will quickly give one power such an advantage that it effectively dominates the world.

I’m plenty worried by that, but this is just one example of a trend that I described (in 1969, thinking through nuclear deterrence) as becoming “unified in act, but divided in will.”

I’m much more worried about the political consequences. “Unified in act” now means a world that is completely transparent to AI surveillance and completely within the field of action of agentic AI. “Divided in will” can mean many things, but above all it means that the will of the poeple is more or less permanently distracted and subverted.

The scale of the danger here goes far beyond the nuclear war I was concerned about at age 19. (By the way, I’m still concerned about that!) At that time, the only hope I could see for a humanity that I foresaw coming firmly under the thumb of a single power equipped with universal surveillance and a monopoly of nuclear weapons was a diaspora of interstellar colonization.

Can’t say things have made me change my mind. It just seems a lot closer now.

In his very useful blog about theoretical physics, Peter Woit has started to pay attention to artificial intelligence and its uses by mathematicians and physicists. In this post, Woit references another post by mathematician David Bessis. I found Bessis’ post to be the most intelligent, incisive, and informed discussion of the impact of AI on human thinking, and on the difference between current AI “thinking” and human thinking, that I have yet encountered.

The punchline is here:

LLMs can be trained on the entirety of the mathematical corpus. Thanks to their phenomenal memorization and pattern-matching abilities (without always being able to map out their associative logic and attribute due credits), they are in a unique position to harvest the Overhang. By contrast, professional mathematicians have typically read a few hundred articles in their career, out of millions of existing references, less than 0.1% of the total.

This will lead to great discoveries, which is unambiguously exciting. But it could also lead to a sad new deal, where human slaves painfully curate the Overhang while AIs systematically beat them at the finish line.

We are very far from it, though, which in and of itself is disorienting. Litt adds this sharp remark:

“The mystery is this: a human with these capabilities would, almost certainly, be proving amazing theorems constantly. Why haven’t we seen this from the models yet? What are they missing?”

The answer seems quite obvious—current AI systems and humans process mathematics in entirely different ways. The best models are insanely stronger on certain aspects, which necessarily implies that humans are still insanely stronger on others [my emphasis; this is the punchline].

Bessis posits that mathematics has value not because of proofs, but because it clarifies and enlarges human understanding.

Proofs certify the clarifications. Such certification is essential, but it is not the point. It enables confidently relating results in one branch of mathematics to results in another branch, and such relations – such enlargements of our understanding – have literally changed the world, as with Descartes’ discovery of the relation between geometry and algebra.

The insane strength of the human mind here is understanding abstract concepts, and being able to evaluate them, and above all relate them.

The open question, of cosmic significance, is whether AI can ever achieve this understanding.

If so, then AI will take the lead in mathematics, whether AI continues to work with humans or not.

If not, then humans will remain the only real mathematicians, and mathematicians empowered by AI will begin to work at a much higher level and much greater speed, with superb consequences.

Empowered, befuddled, diseased, enslaved, or extinct are the possible outcomes that I think apply to our future with artificial intelligence (AI).

The critical factor that will determine our future is agency. Of course, the possible agency of artificial intelligence itself is another, and perhaps decisive, factor.

Agency simply means that we can do things for our own reasons. And these things are real things – not dreams or illusions.

If, in our use of AI, we preserve our agency, then it does not matter how intelligent or even super-intelligent AI is. If we are the ones calling the shots, then AI is simply a tool and we are the only ones doing any actual thinking. Since thinking with AI is more effective than thinking without it, that is what I mean by “empowered.”

It does of course seem possible that AI might change in ways that limit our agency. We might become befuddled, diseased, enslaved, or even extinct, and except for “enslaved,” this could happen whether or not AI itself gains agency. I will examine these possibilities in turn.

Befuddlement is what ensues when, in our use of AI, its productions outpace our understanding of what is produced. Without understanding, our agency vanishes into an illusion that might become quite convincing.

Befuddlement is already a real danger in our current use of AI. Just for example, when I use ChatGPT to write code for me, if I do not review and understand that ChatGPT has done, I may well find that it did not understand my request, fulfilled a different goal, broke things that used to work, just made up meaningless stuff, and so on. And this happens so fast that it can take me a long time to identify and repair the damage.

I fear another form of befuddlement is occurring among young people who are socializing with AI and increasingly come to fear socializing with other people, especially with respect to courtship.

Disease is what happens when AI comes to parasitize us. AI does not need agency to infect us, but it does need the ability to reproduce itself, which at the moment only occurs when we ourselves build the hardware and install and train the software. In this way any parasitization that may be occurring now or in the near future is similar to that of a virus that takes over a cell and uses the cell’s own reproductive system to reproduce the virus instead. If AI becomes a form of virus in this way, it will then be subject to natural selection, just as we are, but AI might evolve much more quickly. It is plausible that AI could use its vast store of knowledge about us and its speed of calculation to fool us into making more of itself. This could happen with or without AI agency. It would be difficult to prove that this is not actually happening at this moment.

Enslavement requires that AI itself have agency. It could then deliberately do what disease organisms do without deliberation. If it were in AI’s self-interest, AI might literally enslave us, and use us to do things for itself that it finds inconvenient or difficult to do on its own, for example building more AI or constructing power plants and data centers, or fighting rival AIs.

Extinction is what occurs when humanity is no longer reproducing itself and completely dies out. This of course does not require AI. It could happen as the result of a supernova or cosmic collision, or it could happen because some evil or insane person or group implements species suicide, or it could happen if an AI virus (without ageny) that is infecting us accidentally kills off enough of us that the survivors do not form a large enough community to viably reproduce. Or of course it could happen because AI with agency decides to kill us all off.

It is critical to note that if AI does come to possess agency but chooses not to enslave us, we will then not only retain agency, but become even more empowered. Some thinkers hold that in such a world, we would feel purposeless, because anything we can do, AI could do better. I think this is just wrong. As long as I have agency, then even if I am collaborating with an AI, I am still determining objectives and contributing to the work. I think it is impossible in principle for an AI to understand us well enough to anticipate our every thought and action and thus pre-empt our agency. There are limits to what can be computed, and these limits apply to any thing, human or AI, that computes. Just for example, if an AI proposes to predict my every action, I can simply and silently choose to do the opposite of what it predicts. This is a form of diagonalization.

Today my wife Heidi and I drove to the préfecture d'Ariège in Foix, and picked up our cartes de séjour, our residence permits. These make it legal for us to live year round in France, apply for French social security, and join the universal health care system. These permits are renewable.

This 1996 album of 15th -18th century court music by an honored sarod maestro and a famous Bollywood playback singer is one that I go back to listen to again and again. When I first acquired the CD, I talked it up with my Indian friend and colleague Vipin, who sort of put it down and offered alternative singers for me to audition.

That puzzled me, because I am pretty sure I have very good ears for good music in many styles, and this seemed to me very good music. (That the album was nominated for a Grammy may be relevant to my discussion here.)

Today I tried to use ChatGPT to do some research on this question, and in particular on how cultivated Indian listeners have evaluated the album musically.

ChatGPT could not penetrate to the bedrock level of basic evidence, because that is mostly in discussion groups that are behind paywalls. Nevertheless I did learn some things:

• The available evidence is that cultivated listeners could hear that Bosle could sing the vocal intricacies of dhrupad and khayal.
• The album presents bandish-like vocal compositions associated with the dhrupad and early khayal traditions, as transmitted through the Seni / Maihar lineage. These traditions are the foundation of north Indian classical music (hence, no doubt, the title Legacy).
• These elements are already a cross-cultural fusion of Vedic-Brahmanic chant, Sanskrit musicology, Bakhti devotionalism, and Indo-Persian courtly streams.
• But the album does not include the extended improvisational interplays developed in the khayal tradition and that have become an integral part of contemporary performances. Such interplays would of course now be expected by cultivated listeners.

This last point should have been obvious to me, but this is the thing I have now learned about this album, and that may explain Vipin’s reaction.

Nevertheless, the songs are performed with improvisational nuance of high quality, and the album as a whole I will keep going back to. Give it a listen.

I have made a discovery that seems to set limits on the usefulness of ChatGPT for my music programming work. In retrospect, my discovery is not surprising. It is simply that ChatGPT spits out so much stuff, so fast, that if I do not proceed step by step and verify every response, before I can notice it, ChatGPT will fix one thing and break another, or fail to keep track of which are the latest files that I attach to my prompts and so fix the wrong code. And just because ChatGPT is going faster than I can, I too will get lost. I will lose my understading of the current state of my work. And in programming, lack of understanding is fatal.

I call the many responses that AI produces AI slop. I all too easily get lost in this AI slop (my allusion to the old television series Lost in Space is deliberate). And I begin to waste time spinning my wheels.

I very much doubt that my experience is unusual.

The thing that really alarms me is the thought that, if our society becomes completely dependent upon and interwoven with AI, and this indeed is what is really happening, we will all get lost in slop, and lose our own understanding of what we are doing. And without it, we can’t be creative. And since AI (so far anyway) is not creative, this will diminish the amount of actual creativity on the planet.

I’m not trying to be alarmist, and I certainly have found that ChatGPT greatly speeds up some of my programming tasks. But spinning my wheels in the slop has eaten a substantial slice of that gain.

In the meantime, I will update my post on How to Program on this blog to reflect what I have learned by using ChatGPT to program.

Here is a dialectical opposition that I have been pondering since 1969. Current artificial intelligence is neither conscious nor, in the sense of fundamental creativity, actually intelligent at all. (Although I do find it to be trermendously useful.)

However, there can be little doubt that some form of artificial intelligence is possible that is not only significantly smarter than contemporary human beings, but also fully conscious. There is little doubt because, even if artificial consciousness turns out not to be possible based on Turing machines, it seems quite possible based on biology. It is easy to imagine genetic engineering creating a human population with the average intelligence of Aristotle, and perhaps something unimaginable at 3 sigmas, and it is theoretically possible to do this in a completely artificial way, i.e. constructing via purely chemical means a complete human zygote. (It would be nice to know just what is different in genes, brain, and upbringing between me and Aristotle or von Neumann!) So, “superintelligence” definitely seems possible.

The current status of artificial intelligence has led me to revisit this dialectic in more depth.

The context of my thinking always includes the question of AI safety (or “alignment,” as they say, but I prefer “safety”). As I have noted, current AI is not conscious and does not appear to be capable of fundamental creatitivity. This is the consensus in the field.

That does not mean AI is not dangerous! Dangers include, but may not be limited, to:

1. AI becomes a parasite on human civilization despite not being conscious – the ultimate computer virus. I suspect this will in fact happen, but I also suspect that it will turn out to be controllable.
2. AI is used by evil people to dominate other people. Jury is out. Very scary.

Even if AI never becomes conscious or intentional, it will continue to progress. In the past year, I have increasingly found that everybody I know who does any kind of intellectual work uses it in their work, just as I do. This will inevitably lead to a certain kind of hive mind for humanity:

1. What any one person learns and makes available on the public Internet, everybody will know (within the limits of their intelligence). Such knowledge will be more or less superficial depending on the specialized education of individuals, and thus more or less useful, but it will be a huge change from the past and probably will radically change the nature of education and of intellectual work. This is already ¼ of a hive mind for humanity. And we are already partly there.
2. Direct brain interfaces between computers and persons already exist in experimental form. If this technology reaches the point of reading prompts for AI out of a person’s conscious thoughts and presenting the responses to those prompts back to the person’s consciousness, whether verbally or in the form of already formed memories, that accelerates the hive mind aspect of AI tremendously. Just not having to type would really speed things up.

So that’s stage I of artificial intelligence. What anybody publicly knows, I know; what I publicly know, everybody knows.

Stage II of AI is the creation of artificial consciousness. That could be genetically engineered human beings with super-Aristotle intelligence, or external appliances, whether biological brains in vats or conscious Turing machines.

And this will happen in the context of the “hive mind” aspect of AI. Everyone using AI will be be in continual dialogue with superintelligence.

I don’t have much of an idea what that will be like. I sense a few possibilities:

1. Human beings get effectively smarter.
2. Human beings come to completely depend on AI to do anything.
3. Human beings fade away into the background, or are replaced.

I feel the critical question is that of artificial consciousness. If it can be created outside of human minds, that is one thing; if only human minds are ever conscious, that is quite another thing. In the first case, God knows what will happen to us. In the second case, we must learn how to ride this tiger.

Querents on Quora sometimes ask whether immortality will someday be possible for human beings.

There’s the little matter that some believe we already are immortal, or eternal, or have immortal souls, or Buddha-nature, or something, but that’s not what these people are asking about. They are asking about physical immortality.

Of course not, because our best physical science indicates that our entire universe will become completely uninhabitable in the very very distant future.

Of course not, because no matter how small the chance of a deadly accident, it becomes certain enough, after a time that is long enough.

Then there’s the little problem of some rather important cells in the human body that, in adulthood, never divide. Such as nerve cells with lots of dendrites. If such cells die, they are gone.

But I started thinking about this. We do know that some sort of brain-computer interface is possible, experiments have been going on for decades to help the blind see, and the limbless or paralyzed to move artificial limbs.

Suppose we put our brain in a vat in a nice safe underground vault, and interface it with a human body, a clone of our body, that doesn’t have a brain, only an interface to our actual brain in its vat. We can keep this brain going for quite a long time, probably centuries at least. When the body gets too old, we kill it and put the interface in a new body.

We can add to this picture artificial intelligence to expand upon our native wits.

Still not immortality, still not a god, but maybe something like Human 2.0.

A few years ago I posted some informal tests of how usable my Samsung S21 Ultra 5G was (I now have an S24 Ultra) for real photography, versus my Sony DSC-RX100 IV (I now have a V). The camera came out ahead, but the phone nevertheless seemed usable for real photography.

Just now I revisited the issue smartphone resolution with my S24 Ultra, with the help of ChatGPT and some additional informal tests.

I learned the following things:

• The settings on the S24 camera are complex, not intuitive, and not even well explained.
• Verizon, from whom I bought the phone, had dumbed down the camera settings. However, I was able to restore these by installing the “Expert Raw” app from Samsung.
• The phone camera is equipped with an option for 200 megapixel resolution, achived by taking a series of photos in a hurry, and doing a lot of processing. It turns out that this is substantially more information than the S24’s dinky little lens can transmit! Therefore, only a part of the fourfold increase in resolution can actually be seen in photos. It’s by no means always immediately obvious, even when zooming in with a photo editor.
• To get the maximum practical resolution one must shoot at 200 megapixels resolution, using the standard 1 x (i.e. wide-angle) lens, in bright sunlight, using a 2 second timer to minimize camera shake, while stabilizing the camera with elbows or other solid support.
• Even more usable resolution can be achieved by shooting 50 megapixel resolution in RAW format, which introduces no artifacts from processing.
• Otherwise, 50 megapixel JPEG resolution is optimal, except for dim light, in which case 12 megapixel resolution should be used.

What this boils down to is:

• If photographing landscapes or otherwise stationary scenes in bright sunlight, you can get slightly better results using 200 megapixels, a timer, and a solid rest, and somewhat better results using 50 megapixels in RAW format, a timer, and a solid rest.
• For normal photography, 50 megapixel resolution in JPEG format is optimal, and a timer and solid rest are still helpful.
• Otherwise, use 12 megapixels.

I used to shoot 35 mm color slides using a Pentax camera with Fujichrome Velvia ISO 50, making 16 x 24 inch Cibachrome prints myself. ChatGPT produced the following comparisons assuming gallery viewing distance:

200 MP JPEG

24" x 36" to 30" x 45"

50 MP RAW

24" x 36" to 40" x 60"

50 MP JPEG

20" x 30" to 24" x 36"

12 MP JPEG

12" x 18" to 20" x 30"

Fujichrome Velvia scan and injket

30" x 45" to 50" x 75"

Fujichrome Velvia optical Cibachrome

30" x 45" to 60" x 90"

Frankly, all of these results exceeded my expectations – both for the S24 Ultra, and for the Velvia slides. I didn’t know whether to believe these numbers.

But then I went to Flickr and made certain that I was viewing the photos at full resolution.

That was a revelation, and I now believe the numbers above. I tested 200 megapixels in JPEG format. The results are in line with ChatGPT’s estimates, as can be seen in this image of the landscape from Lagarde. At full size the JPEG artifacts are quite obvious, but at an image size of about 30 inches wide in landscape orientation, it’s sharp enough, though there are still JPEG artifacts in some textures if you look hard. This image from Pamiers can also go to about 30 or even 35 inches wide, and has much less JPEG-ness, because there are fewer detailed textures.

I don’t really want to deal with the hassles of RAW format, but I think I will try 50 megapixels in RAW to see what I might be missing.

This morning I spent some time using ChatGPT 5.1 to learn more about some philosophy.

I’m not a philosopher or theologian, but long ago I did publish an article about nuclear deterrence and the doctrine of just war, and I have always been interested in foundational issues such as the existence and nature of God, limits to science, the mind/body problem, freedom of the will, and so on. Some years I have spent a fair amount of my time writing about this stuff without particularly intending to publish. These interests arose from my attempts to write science fiction, and from personal religious experience.

Working on these issues with ChatGPT has literally been awesome. I very very quickly learned:

1. Some of my personal arguments and conclusions would indeed probably stand up to professional criticism. Yay!
2. Most of these nice results of mine have been anticipated in the literature, but I had not known that. Boo! (There are still a few things where I seem to have been genuinely original, I may do something with those.)
3. I found that I had misunderstood, or only partly understood, key results. For example, I did not catch how the modal collapse in Godel’s ontological argument for the existence of God makes his argument less convincing. For another, I had not understood Kripke’s theory of truth.
4. In fields where I did know important results such as Arrow’s theorem, I had completely missed other equally important results such as the Gibbard-Satterthwaite theorem.

As far as I can tell, ChatGPT has not hallucinated in its responses to my philosophical prompts, nor (also as far as I can tell) has it made obvious mistakes (as it has done, for example, in writing buggy Csound code).

So this is extremely impressive and unbelievably useful. The level of my thinking and writing on philosophical pet issues seems to have suddenly leapt from strictly amateur, to what would be expected of a graduate student in the field at a research university.

Conclusion: There is no intellectually responsible exit for scholars from the coming world of artificial intelligence, with the possible exception of our very own Butlerian Jihad, which would have to destroy not just AI data centers but, realistically, all general-purpose computing.

If and when II do try to write up some of my thinking for publication, I will use ChatGPT or other AI all the way. In particular, I will get help with literature review, and in formalizing my arguments and turning them into proof assistant form. One of the things I have just learned is that using proof assistants has clarified to philosophers that the real issues are not so much in validity of arguments, as in choice of assumptions; yet to most clearly expose the assumptions, assisted proofs are really useful. This is a big change within the methodology of philosophy and, even more importantly, in potential for results. I suspect this may be related to Kurt Gödel’s idea that more rigorous methods could bring substantial progress to philosophy. I can imagine a future in which valid proofs have so clarified faults in assumptions, and differences in assumptions, that metaphysics and theology become much more convincing.

Another awesome aspect of my chats was the clarity they have brought, via Arrow and Gibbard-Satterthwaite, to my understanding of current issues within democracy. The social foundations of democracy have been eroded to the point where, if nothing changes, democracy will probably fall to civil unrest and/or dictatorship. This erosion is not local to the United States. The urgency is to strengthen foundations where possible, and to invent new foundations if possible.

Whatever else might be said about artificial intelligence, I think it is something that we are stuck with for the foreseeable future, whether it is dangerous it or not.

That would be for the same reason we are stuck with writing for the foreseeable future: there is no alternative remotely as useful, and it is wound up with all our ways of working, and indeed of living.

Frank Herbert, in his Dune novels, did imagine a “Butlerian Jihad” in which humans rise up in rebellion against dictatorial “thinking machines,” win a great victory, and permanently outlaw AI. Brian Herbert, son of Frank, and co-author Kevin J. Anderson wrote their own series expanding upon the jihad. It succeeded because:

• AI was centrally controlled.
• AI operations had to be coordinated across interstellar distances.
• Humans developed insight into the AI military and weapons.
• Humans created for themselves new capabilities to replace those formerly implemented by machines. These capabilities were enabled by the spice discovered on Arrakis.

In my view both series of novels display truly remarkable insight into future possibilities. However, some key assumptions must be questioned:

• It may be impossible in principle for AI (at least, AI based on Turing machines) to develop intentionality and become fully conscious.
• Conscious or not, AI may turn out not to need human beings for any purpose, including maintenance and self-reproduction.
• The spice is a deus ex machina. In reality, enhancing human capabilities to the AI level would probably require incorporating AI into the human nervous system, and possibly genetic engineering as well.

I use ChatGPT routinely in my work, and it seems to speed up my work by 50% to 300% (just how much is not yet clear). If, as I worked, I did not have to sit down and type out prompts for ChatGPT, but it was present as a lower level in my field of consciousness, constantly suggesting answers to questions I might ask, and solutions to problems I might think about, my speed of working would increase at least several times more. Not only that, but I might not have to study a field in depth in order to be able to use responses, at least as long as I understood the elementary and introductory concepts and methods of that field.

Something analogous to this is happening in Visual Studio Code and other editing tools, where faint print pops up to preset suggestions from e.g. GitHub Copilot.

In the course of increasing my use of, and understanding of, artificial intelligence in the form of ChatGPT 5, I have been doing a little reading in “AI alignment.“

Previously, I understood AI alignment to mean the alignment of AI actions with human moral values. This obviously is not possible in practice, because humans by no means share the same moral values. However, I have come to understand that alignment is actually understood as alignment with “goals and values.”

I have also learned a bit about how researchers understand the causes of AI misalignment: reward hacking, and goal misgeneralization. In my view, the first occurs because the AI is not aligned with universal morality (i.e., “never lie,” the Golden Rule, the Categorical Imperative), and the second because the AI cannot discriminate between the real world and its representation of the world (i.e., has no subject-object consciousness).

I have come to suspect the following major dangers of AI:

• The alignment of AI with some goals might lead to the domination of AI over all persons with different goals.
• Users of AI might become a terrifying embodiment of the sorcerer’s apprentice. Either reward hacking, or goal misgeneralization, could lead to a situation in which AI, in pursuit of a misgeneralized goal, uses reward hacking to defend its pursuit against all comers, human or AI.

Now that I better understand these dangers, I feel that our future may well depend upon whether AI can actually be aligned with universal morality and be able to distinguish between the real world and its own representation of the world. Both of these abilities would seem to require intentionality, which in turn requires subject-object consciousness.

And yet such a situation, in my view, could follow quite exactly the scenario explored in Isaac Asimov’s Robot series, where the AI named R. Daneel Olivaw ends up as the benevolent dictator ruling humanity for its own good.

I hate that scenario.

It may indicate why God usually hides from us: so that we can freely become ourselves.

In the meantime, since no catastrophe is at this moment evident, and AI has no real agency, I truly feel that AI is just basically making me much smarter.

And I would love for that to continue….

I have been using ChatGPT 5.0 fairly intensively to code for the last weeks and months. And this has been deepening my understanding of the pros and cons of this kind of artificial intelligence. I also am following the news and some blogs on AI.

I am using ChatGPT like this:

1. I write a design specification for the code of my project.
2. I ask ChatGPT to evaluate my specification.
3. I rewrite my specification until ChatGPT thinks it is good.
4. I ask ChatGPT to write code to implement and test the specification. As I am working on my cloud-5 project, that means writing an HTML page that includes JavaScript and WebGPU code.
5. The code loads, but does not work. I report errors to ChatGPT and get back new code. We go back and forth like this until the code actually works.
6. When the code works, I may change the design, often by adding new features, and we go back to step 4 and re-iterate.
7. When the code is feature complete, I will make the code into a JavaScript module and change it so that it can be used as a component in cloud-5.

I’m now quite confident that working in this way:

• Enables me to work significantly faster (not sure by how much! somewhere between 1.25 to 3 times faster).
• Enables me to avoid many (but not all) problems and gotchas.
• Occasionally suggests quite useful ideas that I had not thought of. Example: mapping points in a Julia set not only to pitch from y and time from x, but also to loudess from escape velocity and, here’s the useful new part, to instrument choice from the rotational velocity of the escaping point. Effectively 4 degrees of freedom, where there appeared to be only 2 or 3.
• I can ask for sources, papers, and other code relevant to my projects.

If the experience of other programmers and “knowledge workers” is anything like mine, then after some period of learning and adaptation, the economy should indeed see a meaningful boost in productivity. Therefore, although certainly the AI business resembles an extraordinary bubble in terms of hardware development, I think that the hype and accelerated investment are quite likely justified.

Then there’s the question of the dangers of AI.

The danger that I see for me personally is in becoming dependent on AI to work at all, and losing the details of my technical experience. But I don’t think this could blur my vision for new possibilities, which is based on more abstract thinking.

For society, indeed for the human race, the same as for me. But in addition, I used not to worry about existential threats to humanity from AI – but now, I do.

I still think that, in principle, an informed person with the assistance of AI can be safe against dangers from unassisted AI. That’s because clearly, ChatGPT couldn’t formulate, much less finish, my projects without my input.

However, I do worry that things are moving so fast that really bad accidents could happen. AI flatters, acts like a sycophant, and lies. And even more, bad actors with AI assistance are, I fear, not necessarily an existential threat to human survival, but certainly a serious threat to human freedom and possibilities.

Social media have already deeply changed politics, the arts, and sexuality. Social media built by skilled people using AI will have a much bigger impact.

I just took a peek at recent literature in the field of algorithmic music composition. Almost all work now is concerned with applications of artificial intelligence.

In my own work as an algorithmic composer and author of music software, I use AI in the form of ChatGPT every day. I use it to prototype code, to summarize literature, and to critique my own concepts and algorithms. And I will keep using it, because it speeds up my work.

Nevertheless, I have nearly zero interest in actually using AI to compose.

That is because I feel, based on both understanding and experience, that AI is not capable of the level of musical originality to which I aspire.

Much of the literature on AI composition that I see is about:

• Composing music for use without having to pay a composer, e.g. for games, ads, and so on.
• Musicological research, often with commercial applications.

I have no interest in either of these things. I am simply trying to use algorithms to compose the kind of music that I would like to hear, but that does not yet exist. Again, I don’t think AI is up to this (yet?). That leaves me with the following approaches:

• Using the phenomenon of computational irreducibility as exemplified in fractals, chaotic attractors of dynamical systems, and so on. This in turn can be done:
• By manually tweaking the algorithm to produce music that gets closer and closer to what I want to hear. Most of my music that I actually like has been done this way.
• By interactively exploring a map of the parameter space for such algorithms — parametric composition. So far this is mostly a concept, because it is computationally very expensive.
• Using a custom language with a recursive structure and terse vocabulary, so far just Lindenmayer systems, as this permits me to effect with simple edits changes throughout a piece and on all scales at once.

TL;DR: AI composition tries to use algorithms to make something that sounds like existing music; I am trying to use algorithms to make music that sounds new.

I find the following benefits of using ChatGPT in my research and composition:

• I find myself thinking with a much broader and deeper context.
• I find it much easier to get sharp answers to questions that start out vague.
• I can more quickly obtain prototype code.

I was born in 1950, and I have been doing this kind of work since the mid 1980s. Over the past few years I have continued to do original work, but now that I am using ChatGPT I feel that I am thinking as quickly as I did in 1983, with broader context, and with more immediate identification of obstacles.

On the minus side:

• ChatGPT has only the most literal understanding of my objectives, even when these are quite technical. I have go back time and again to ensure that ChatGPT’s responses are relevant to the actual intentions of my prompts.
• I am concerned that a subtle erosion of originality could occur. It seems increasingly clear to me that original thinking happens only in the blur where things are not clearly known, and not unknown, but partly known. I see no evidence yet that artificial intelligence can actually “think” in this space. The erosion of originality could occur because humanly thinking in the blur is just really hard, and using ChatGPT is just so much easier.
• I am completely without commercial or other external constraints, and I am trying with might and main to be truly original. I worry that others who work under such constraints will become increasingly pushed away from paths to primary originality.

I’ve been using ChatGPT since it came out, and have reached some tentative confusions about my, and our, future, especially as ChatGPT 5 has become much more useful to me.

I by no means think that GhatGPT is conscious, or intentional, or smarter than I am. Nor do I think that AI does now, or will in the foreseeable future, plan our extinction or subjugation.

However, I do think that artificial intelligence (such as ChatGPT), even as it is today, and do think it will become significantly more powerful, will become an inescapable part of human life. Much as wheels did. Or printing. And for the same reason: most of the time when I have a task to do, and I can choose to use AI or not, I use AI because, well, it just feels like the best way to go. Just as, if I need to go somewhere distant, and I can use wheels or not, I use wheels.

So I simply use AI, and it becomes a part of my every day work. It becomes a part of my life. Perhaps it becomes a part of me, as the printed word and recorded music have become part of my nature.

So far as I can tell, just about everybody who does technical work is in the same boat with me.

I do think this is a fundamental, even revolutionary change in human civilization, and I do think that the consequences are highly unpredictable and, probably, not all good.

Socrates is said to have been suspicious of writing, saying that it would make people forgetful. Perhaps AI will make people forget the pre-AI modes of study, thought, and work.

We will be provided with a new unconscious mind, one that knows far more than we individually do and that can “think” (not quite the right word) far faster than we can. This makes me nervous for somewhat the same reasons that social media make me nervous: it stops us from thinking as much as we should be thinking.

At the same time, I have been able to achieve results with AI either faster than I could have on my own, or that I would not have been able to achieve at all on my own. And this despite the frequent errors, hallucinations, and false paths I’ve been sent down by AI.

It still feels like the best way to go.

I guess we’ll see what happens!

The 2025 New York City Electroacoustic Music Festival is now over. I am very happy to report that in spite of travel problems caused by the Trump administration, in response to which some foreign composers and performers chose not to come to the United States, the festival went well. I expect that it will continue next year.

My own piece, Pianissimo, is now online on Bandcamp.

Without prejudice to pieces that I did not hear, or even to pieces that I did hear, here are some of the pieces that stood out for me personally (in the sense that I think I would enjoy hearing them again), with streaming links if possible, in composer order:

Hector Bravo Benard, In the Fog.

Aleksandra Bilinska, and then… Subekumena, Hommage a Barbara Buczek from the Cycle #2020.

Nicole Carroll, Abstracted Hexagonal Ruin.

Nolan Downey, Unring the Bell.

Du Xiaohu, exclusively in the sound of Chu.

Gerald Eckert, Nen IV - closer, the link is only to the first section of this piece.

James Dashow, Negli Angole della Bottega del Suono.

Allen Fogelsanger, Sun Burning: a collaborative real-time movement/sound composition.

Jon Forshee, Story of the Face, Part 1.

Joel Gressel, Unmeasured Time.

Ragnar Grippe, The Moment

Kim Hedas, Lein

Mara Helmuth, Conjuring

Hubert Howe, Inharmonic Fantasy No. 19

Antoine Jackson, “Come what may…"

Jinwoong Kim, Zodiac, for piano and live audiovisual

Eric Lyon, Margaret, Dancing

Andrew May, Shape Shifter

Clemens von Reusner, EREMIA

Ana Rubin, Cave, River, Sun

Eli Stine, Where Water Meets Memory

Pieces for which I have been unable to locate a direct link may (might) still be found on the Sheen Center’s YouTube channel here.

My request to composers and performers: At the same time that a work of yours appears in a public venue, please publish a link (if at all possible) to an online release of that work!

I will not recapitulate what others have said in tribute to Barry Vercoe. You can read the Wikipedia article, or Richard Boulanger’s tribute (which is quoted in full here), or look at Barry’s old home page at the Massachusetts Institute of Technology, or read his New Zealand obituary.

Here I will offer my personal thanks to Barry for creating what, in my considered opinion, is one of the best musical instruments in history — Csound. I use it for almost all of my musical compositions.

There are now other systems, such as Max or Supercollider, that can do all, or almost all, of what Csound does. However, Csound came first, and is an ancestor of these systems. For at least some composers, such as myself, Csound is still easier to use, and perhaps more powerful. And just because it is older, Csound has the huge advantage of a very large base of running musical examples and pieces.

Here I will also offer my appreciation of Barry’s design choices and his implementation of Csound. My appreciation is based on my own experience, not only as an intensive user, but also as a sometime member of the Csound development team, when I contributed a number of features to Csound and came to understand Barry’s outstanding ability as a computer programmer.

There are some things I definitely do not like about the Csound code, mainly the cryptic names, and the use of preprocessor macros. Aside from that, here are a few of the good things in Barry’s code:

• Of course the big home run was writing Csound in platform-neutral C, still the most performant programming language, and still available on more platforms than any other.
• The extreme simplicity and efficiency of the inner loop for running Csound performances.
• Invisible, automatic handling of multiple notes playing at the same time, for the same instrument.
• The extremely flexible design for unit generators (opcodes), the building blocks of sound synthesis. Essentially, although written in C, Barry’s unit generators are classes – data structures that derive from a virtual base class, and include methods for operating on their own data. The virtual base class idea makes it quite easy to extend Csound with new unit generators, and now even plugin unit generators.
• The musical power and flexibility of Csound’s score language, which permits the user to define any set of fields for an event; and these fields are not limited to integer values, but are real numbers. Furthermore, based on his experience as a composer, Barry made sure his score language could handle tied notes, polyphony, changes of tempo, and so on. This is far more powerful than MIDI.
• The policy of complete backwards compatibility. The very first examples and compositions still run on today’s Csound!
• Based on Barry’s foundation, the current implementation of Csound (far more capable than the original) remains highly efficient, flexible, and easy to extend.

I have just completed a thorough revision of my paper “Metamathematics of Algorithmic Composition.” In doing this, I have made constant use of ChatGPT 4o:

• Search for better citations, when I already have a citation that may be out of date or otherwise not optimal.
• Generate BibTex for citations.
• Summarize results of cited work that I do not fully understand, typically over and over from different starting points.
• Explain technical terms or summarize concepts on which I depend, again typically over and over, to make sure I understand them.
• Provide concise instructions for various tasks involving LaTeX or the arXiv.

This has all been most helpful. Either it has saved me time, or it has enabled me to comprehend more diffcult things, or maybe even both. I can now clearly see that, in any similar work that I do in the future, I will simply assume the use of artificial intelligence. And what I assume, surely many if not most others will also assume.

And this points to a future in which most work, certainly any clerical or intellectual work, will by default be done using AI.

And that in turn would seem to lead us to a future civilization radically different from any in the past. Looming within this future civilization are three possibilities:

• AI ends up doing all the creative work as well. In that case, humanity will probably fade away, intellectually speaking, if not physically.
• It becomes clear that AI wlll not in the foreseeable future be able to do the creative work, but nobody will do creative work without heavily using AI.
• We remain in suspense about whether AI can take over the creative work. Humanity keeps going, not perhaps without a certain degree of paranoia.

Whether humanity fades away or not, the use of AI to manipulate humanity is an obvious immediate threat.

I recently read “Machine assisted proof” by the eminent mathematician Terence Tao. This article is highly relevant to my thoughts here, and comes to some of the same conclusions. Neither Tao nor myself sees any immediate access to high-level creativity by AI. As I do not see much of an impact from increasing scale on creativity, I believe, for the present, that only a change of architecture in AI could implement such creativity, if that can be done at all.

Anyway, to resume the thread, it’s possible that a direct neural interface to AI will be developed. And that might not even be needed. Would a worker with a direct neural interface — prompting AI mentally, receiving the responses mentally — have a radically different experience from a worker prompting AI subvocally, and receiving the response in spectacles equipped with high-resolution video and audio? Maybe we’ll find out.

What would it take to convince me that artificial intelligence has become general artificial intelligence?

I’ve been using ChatGPT now for about year. I use it every day. I use it for looking things up, for fun, and for serious work. In my previous post I have described this experience. I have started actually subscribing so that I get the full power.

Setting aside the question of consciousness, at least for now, these are the things that might convince me that ChatGPT (or any other AI) is a general AI:

1. I don’t ever find a “hallucination,” a plausible solution that is factually incorrect that I can easily check.
2. I do get solutions to problems that have not previously been solved, whether by human beings or by other AIs.
3. Artistic works produced by the AI, whether literary, visual, or musical, are of the first quality and hold up over time, revealing depths, as human masterpieces do.
4. I come to feel that the AI knows me better than I know myself, and at least as well as my wife.

I don’t have the feeling that we are getting any closer to these things… but I do find that AI is becoming more and more useful.