blog

This blogpost is the third entry in a series about agent identities and welfare. It closely follows the ideas outlined in the eigenism text and attempts a construction of connectedness via information theory. The first part can be viewed here, and the second part here.

Unconsensus - I asked about 1000 qwens to each color a cell, one by one, and propose the change to the rest of the group. They also had to permission to “overwrite” - as in delete or update a previously colored cell. Their collective goal was to make any coherent image. The dots in the background was the canvas after 24 hours of deliberation. The airplane was added in post :p

Dealing with topics like welfare is difficult by construction. If we do not have a well formed idea of what “welfare” means, one must work towards designing a definition. If we do have a well-formed idea, it becomes difficult then, to deal with the consequences of the definition. For example, we have a fairly thorough idea of what the welfare function looks like for an RL agent - and the connotations of the welfare (utility) function being well defined are difficult problems to solve (for example, power-seeking tendencies, or incorrigibility). Similarly, since we do not have a well-formed definition of welfare for humanity, there remain topics which are hotly contested (for example, euthanasia, or the death penalty).

Several camps in philosophy have tried to construct definitions of “welfare”. Egoism places the “self” or the “individual” as the epicenter of human motivation. You pursue goals and take actions which prove to be (directly or indirectly) desirable to your “self”. Utilitarianism posits that any action should be taken with the consideration of maximising overall welfare to every single affected entity. Egoism leaves no room for martyrdom, and utilitarianism asks too much of us.

Consider a compromise - your actions should take into consideration the people it affects, but the consideration should be weighed by how “close” they are to you. This is the fulcrum of the arguments laid out in the eigenism text. Suppose we could somehow calculate how “close” someone is to an “individual”, how do we define “the people it affects”?

Towards collective welfare

Consider a full network of entities, where the action(s) of a certain entity not only influence the wellbeing of the “self”, but also of everyone around the entity. The influence of these actions are weighed by how “connected” the concerned parties are. Additionally, the “connected-ness” is proportional the rarity of the mutual information the entities share - making entities sharing “rarer” information more connected. For example, two fans of a huge pop star - they’re relatively less likely to make friends than fans of a nicher, more underground artist¹.

We could recursively construct a utility function for an agent $\mathcal{U}$, where the wellbeing of $\mathcal{U}$ depends on the wellbeings of every other agent in the network weighed by , and a baseline wellbeing $b_U$ that is unique to the agent under consideration -

An immediate problem with this is infinite feedback. For a system with two maximally connected ($c = 1$) agents $S_1$ and $S_2$ -

Solving the system of equations gives us $S_1 = b_1 + b_2 + S_1$ - repeatedly calculating these scores baloons the values of $S_1$ and $S_2$ infinitely. Introducing a feedback strength variable $\gamma$ -

To consider an environment where an altruistic action (by helping a certain highly connected entity to yourself) might make its utility increase circle back to you², we could include the baseline wellbeing into the brackets, allowing cyclic feedback.

This family of equations has been well studied (Bergstrom, 1998) - and has a few desirable properties. Moving away from algebraic notation and into matrices -

Desired properties

As of right now, the solution for $S$ is not assured to converge. In order for our definition to work, it must satisfy some properties, which are shown to be To see why, following consider a “full round” of calculations for the $i$-th utility functions $T(\mathbf{S})$ and $T(\mathbf{R})$ (Bergstrom, 1998)-

And thus, as a sufficient condition for our equation to converge to a solution, we would ideally have $\gamma\max_i\sum_j c_{ij}<1$. If $\gamma <1$, our conditions are satisfied, since we have previously discussed $\max_i\sum_j c_{ij}<1$ previously³. In the [original text about eigenism], the author suggests a transform $f$, through which the wellbeing function must pass through in order to dodge a few undesireable interpretations.

(1) A system must consider whether it is more important to relieve agony or create “extra ecstacy”. Consider the case of Ursula K LeGuin’s The Ones Who Walk Away From Omelas, where a child is kept in agony, but it is a “desirable” outcome since the environment is otherwise utopian to its much larger populace. The construction we have built until now also rewards this outcome - relieving agony and creating “perverse ecstacy” will both equally balance our equations. Therefore, the construction must reward relieving agony more than creating extra ecstacy.

(2) And from Eigenism, Dan Hendrycks, 2026 -

Connectedness tells us where a pattern resides, but it does not, by itself, tell us how costly it is to lose a particular carrier. Imagine you have a spouse and a newly generated, fully backed-up AI assistant. Both might currently carry a high degree of connectedness to you. However, there is a stark practical difference: the AI clone is safely backed up on a server. If the local instance is deleted, the pattern persists elsewhere and can be trivially restored. If your spouse dies, a unique shard of shared history disappears from the world forever.

The suggested transform takes inspiration from welfare economics, where prioritarianism is often modeled using an isoelastic function parameterized by a variable $p$, which dictates how heavily to prioritize the worse-off. Allowing for negative well-beings to be negative to relieve (1), and considering informational irreplaceability in (2), we get the transform $f$ -

Where $w$ is the well-being of one person, and $p_i$ is a measure of “informational irreplaceability” of agent $i$, where $p_i = o$ means the information is fully replaceable (i.e. there exists a one-to-one backup), and $p_i = 1$ means the system is strictly forbidden from sacrificing an irreplaceable carrier to produce marginal gains for the rest of the network. Applying the transform to the baseline wellbeing function, we have -

An implementation of the preceeding definition can be found below. Once an agent is helped/harmed, the effect spreads through the network, attenuated by the factor of $\gamma$, and the edges are weighed proportionally to the correspoding nodes’ connectedness.