Agency without consciousness

Agency

The word "agency" can be used to mean many things. Here are two ways I'm especially interested in:

Firstly, in contemporary AI research it is used to talk about systems that can take actions autonomously based on interactions with some external environment. This is what distinguishes, for example, LLM agents from LLM chatbots. It's interesting that, with a few exceptions,¹ the academic and corporate definitions have mostly converged here. Broadly, everyone is using the word in the same way, from marketing teams at Salesforce to research groups at universities.

For example, from the Russell and Norvig textbook:

An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators.

And from the Anthropic engineering blog:

“...we’ve gravitated towards a simple definition for agents: LLMs autonomously using tools in a loop."

Secondly, in 20th century analytic philosophy it is used to distinguish intentional actions from mere behaviour - presupposing the ability to act intentionally. Donald Davidson's writing is full of paradigmatic examples of this kind of discussion:

Thus spilling the coffee, sinking the Bismark, and insulting someone are all things that may or may not be done intentionally, but even when not intentional, they are normally actions. If, for example, I intentionally spill the contents of my cup, mistakenly thinking it is tea when it is coffee, then spilling the coffee is something I do, it is an action of mine, though I do not do it intentionally. On the other hand, if I spill the coffee because you jiggle my hand, I cannot be called the agent.

In this post I'm using "agency" only in the contemporary AI research sense - that is, autonomous behaviour, interacting with some environment. One issue with talking this way is that the scope of things that count as agents becomes incredibly broad. Thermostats, for example, are agents, by virtue of the fact that they can autonomously adjust the temperature of a room. For this reason it can be helpful to see this kind of agency as a spectrum.

However, being precise about what being "more agentic" in this sense means is an open research question. Notable examples include:

1. The difficulty of the hardest task an agent can reliably complete (measured in hours it would take a human to complete the task). (Ability to Complete Long Tasks)
2. The length of time (or number of reasoning tokens or steps) after which performance begins to plateau. (Test-Time Scaling)
3. The ability to infer, pursue, and interrupt secondary goals in a way that is adaptive towards pursuing a primary goal. (Metacognition)

Later in this post I give examples of systems which are agentic, and I try to pick examples that are "interesting", in part because they score highly on all three of these metrics.

Consciousness

Here are three ways the word "consciousness" gets used (definitions from SEP).

A system is self-conscious if it is not only aware, but also aware that it is aware - it has some form of awareness of itself as subject or agent.

A system is access-conscious if information in one of its states is widely available for use in reasoning, verbal report, and the rational control of action.

A system is phenomenally-conscious if there is something it is like for it to be in that state - its states have a subjective, qualitative “feel” (qualia).

It's an open question whether systems can be genuinely conscious in any of these ways without being conscious in all the other ways, however one thing that is clear to me is that systems can display highly agentic behaviour without being conscious in any of these ways. In the next section I go through each of these one by one and give an interesting example of an agentic system which is not conscious in that sense, and then talk about why I think the example is particularly interesting.

Examples of agency without consciousness

Agency without self-consciousness

Firstly, humans are capable of self consciousness, but we aren't always in self conscious states. We can operate on "autopilot", for example during a daily commute that we've done hundreds of times before. In these states we are perfectly agentic - we can navigate through traffic and crowds, perhaps even re-plan our journey in the face of delays. Much of our perceptions in this state are also access-conscious: we can recall what we saw, what route we took etc. We are also phenomenally conscious when in this state - there is something it is like for us to be on the commute (e.g. we might feel grumpy, we might be seeing the garish colour of a passing car).

Agency without access-consciousness

Again, while humans generally display access-consciousness, we aren't always in states which are access-conscious to us. One notable example is the psychological phenomenon of blindsight. In cases of blindsight it is possible for a human to react to some event - for example, to duck out of the way of a flying object, without being able to consciously "see" the object. If asked why they took certain actions, patients with blindsight will report that they don't know.

Agency without phenomenal-consciousness²

Some interesting examples for this category include complex temperature control systems and self driving cars. However these examples are contentious because some accounts of phenomenal consciousness do predict that there is something it is like to be such a system. For this reason I think the best examples are probably collective agents like corporations. A corporation is highly agentic, it can pursue goals over long time horizons, perform metacognition (reviewing its own performance, changing strategy) and so on. Corporations also demonstrate highly centralised planning capabilities and need to model themselves within the markets in which they operate. However, to our best understanding, there is no phenomenal character to a corporation's experience, there doesn't seem to be anything it is like to be one.