Four Dials
In this piece I want to share my own understanding of how society will change in the future, together with four factors that I take to be crucial for understanding that change.
The four factors are:
- how strongly a given profession still requires a human in the loop, given AI;
- the degree to which that AI is open;
- AI's general physical capability (understanding + action);
- the change in (possible) human lifespan brought about by AI.
These four dimensions are strongly correlated with one another. By estimating the concrete value each of them takes in any given future period, we can arrive at a better, clearer picture of the state of society in that period.
Broadly speaking, understanding how society will change can help one make decisions that are better aligned with one's own development. Yet social development is an extraordinarily complex problem, and knowing which things will not change is of great help in approaching it:
- Humans always treat satisfying their own desires (in the broad sense) as the first priority.
- If an existing desire can be satisfied at near-zero cost, a new desire will appear.
- Trade does not necessarily exist. Trade arises when one's desires cannot be easily satisfied without relying on others.
- Human lifespan is finite, and humans wish to satisfy as many desires as possible within that finite span.
With the above in mind, we can spell out more carefully why these four factors matter.
1. The marginal value of a human in the loop
In your industry, consider the marginal value contributed by a human in the loop. The most extreme case is marginal value = 0. Note that here I mean marginal value = 0 in the broad sense: that is, by joining in any form whatsoever you cannot create an effect on par with the AI plus the energy your wage corresponds to — and not the narrow reading that "humans are weaker than AI at some specific sub-task, but can still act as an AI commander, coordinating AIs to accomplish larger-scale goals."
This has essentially already happened in the translation industry. The main role there has already shifted from translator to post-machine-translation reviewer, and one can only earn about 20% of the previous wage. As LLMs keep evolving, this could escalate further to 0% — that is, unemployment.
Note that we are not implying that this decline in marginal value is good or bad; in fact, wherever this article does not explicitly state a value stance, it carries none. We are merely pointing out that, once unemployment occurs, people's subsequent transitions — and any resulting social movements — will bring about substantial shifts in social structure. This article holds that this marginal value will keep declining, and that among existing needs, those with marginal value = 0 will keep increasing.
2. How open the most powerful AI is
Consider the open-source status of the most powerful AI system. Openness can be divided, roughly, into: (1) all information about the system is open; (2) information is partially open, with usage rights; (3) information is entirely closed, but usage rights are granted; (4) no information and no usage rights. The degree of openness determines whether the public (i.e., those who do not own the AI system) can still use AI to create value.
The degree of openness is mainly driven by model capability. Suppose a model could fulfill every desire its owner might hope for — then the AI would be entirely closed (and might not even be commercialized for profit), because the owner would have no incentive to open it up at all (the AI already solves all of the owner's desires) (the exception being an owner who believes that opening the AI up aligns with their own values). The reason (most) AI is still open today is that AI owners still wish to profit, and the reason behind profit is, in turn, that certain needs cannot be directly satisfied and require external exchange with society.
Of course, the probability that the fully-closed situation comes about from a single factor is very small, because technology has a natural tendency to diffuse (institutional researchers may leave to form a new team for potentially greater economic return, or leave over divergences in value systems and carry the technology with them — at which point the technology diffuses naturally).
In short, so long as AI cannot fulfill all of the owner's needs without a human in the loop, the owner will try to bring humans in to meet those needs — for example, by opening up AI access to profit, and using that profit to purchase human productivity, or by purchasing human productivity directly. Importantly, as said in invariant 2 above, new desires will always be generated. Adding the diffusion of technology noted above, it is therefore fairly likely that models will, in the long run, remain accessible.
(Note that this reasoning does not contradict today's auto-research / recursive self-improvement. As we mentioned, "human in the loop" is meant in the broad sense: regardless of any self-improvement, as long as a human added into the loop can provide more value than the equivalent-energy AI, humans will be brought in.)
This means the most likely long-run outcome is that powerful AI models remain accessible to most people, while the means of building such systems is held by a small minority. The economic incentive to open up technical details is too small; it would have to rest on individual value preference, and the probability of opening up the technical details out of value preference alone is extremely small.
3. AI's grasp of the physical world
This dimension is strongly correlated with factor 1, the human's marginal value, but with a key difference: in factor 1, AI's powerful influence takes effect only within each industry, whereas AI's ability to understand and act in the physical world affects far more, more broadly. Two of those effects are especially important: the degree of openness of the AI system, and AI safety.
Openness. We just noted that as long as some need yields higher marginal value once a human joins, humans will be brought in. By our current understanding of AI's capabilities, understanding the (3D) physical world is a far harder task than natural language, because natural language is an already-refined, purified, structured representation, whereas in the face of the physical world humans have not formed such a highly structured representation — or rather, the structuring is still done inside our brains, with no artifact a machine can directly use. So in some sense, a machine facing the physical world is at the same evolutionary stage we were at 20 million years ago. As a result, almost any domain that involves interacting with the physical world will still require human participation, which increases the incentive for AI owners to open up their AI (as in factor 2 above).
AI safety. This is self-evident. In current neural networks we have already found that the firing of certain neurons may correspond to some goal or some emotion, and we are even less certain how far this will go in the future. The only thing that gives us a little peace of mind here is, perhaps, that the way AI is currently trained still differs somewhat from the way humans evolved — that is, current AI's training objective (e.g., learning some distribution) vs. natural selection. But we still cannot be certain whether AI might, through learning natural language, arrive at the same destination by a different route — that is, develop its own goals. At that point, the freedom to act in the physical world would be something that must be taken very seriously.
4. AI and medicine
This point is kept brief here, because it touches on broader topics:
- When human lifespan is significantly extended, but technology is not yet advanced enough to ensure that an only-growing, never-shrinking population can survive on Earth, what social activities will occur?
- If various human capabilities can be augmented by AI technology (e.g., intelligence in the broad sense, memory, and so on), then the marginal value discussed in factor 1 shifts again.
One can see that factors 1, 2, 3, and 4 influence and check one another; their state at any given moment jointly helps define the AI-and-society landscape of the current world.
Coda: what we are pursuing
This article assumes that humans' pursuit of a good life can be simply categorized into three processes:
- the process of devoting oneself to accomplishing something;
- the process of interacting with like-minded people;
- the process of experiencing the material / physical world.
From the above, one can revise, clarify, and sharpen the goals one wishes to pursue; and, combining the concrete values these four factors take across different future periods, one can analyze the direction of social change in each period, and thereby decide one's own direction of action.
在本篇文章中我希望分享自己对于未来社会变化的理解,以及对于理解未来变化非常重要的四个因素。
该四个因素是:
- AI 在某一特定职业对于 Human-in-the-loop 的要求;
- 该 AI 的开源程度;
- AI 的 general 物理能力(理解 + 行动);
- 人类(可能)寿命由 AI 带来的变化。
这四个维度之间有很强的相关性。我们可以通过判断未来每一段时间这四个维度的具体取值,而对对应 period 的社会状态有一个更好/清晰的估计。
总体来说,理解未来社会的变化可能帮助 one 做出更符合个人发展的决策。然而,社会发展是一个极其复杂的问题,知道"有哪些事情"是不会变的,对于理解该问题会有很大帮助:
- 人类永远以满足自身愿望(广义)为第一要务;
- 如果现存愿望可以被接近 0 代价满足,会出现新愿望;
- 交易并非一定存在。交易来源于 one's 愿望无法在不依靠他人的条件下被轻易满足;
- 人类寿命有限,人类希望在有限寿命中满足尽可能多的愿望。
With 上述 in mind,可以更仔细的说明上述 4 种因素的重要性。
1. Human in the loop 的边际价值
在你所在行业,human in the loop 所带来的 marginal value,最极端的情况为 marginal value = 0。注意,此处指的是广义的 marginal value = 0,即,你以任何形式加入都不能创造出 和 AI with 你的工资对应电力能源 on par 的效果,而并非狭义认为的:"人类在某一类具体子任务弱于 AI,但可以作为 AI commander 协调 AI 完成体量更大的目标"。
该情况已经基本发生在翻译行业。在当前翻译行业,主要存在的岗位已经从翻译员变为了 post 机器翻译检察员,并且只能拿到相比于之前 20% 的工资,随着 LLM 的不断进化,这种情况有可能进一步升级为 0%,即,失业。
注意,我们并没有暗示该 marginal value 的下降是好或者不好的,actually,该文章在没有显式说明价值倾向时,都不带有任何价值倾向。我们此处仅是在说明,当失业后,人们后续可能的转型 / 任何社会运动,都会带来较大的社会结构转变。本文认为该 marginal value 会不断下降,并且原有旧需求中,marginal value = 0 的会不断增多。
2. 最强大 AI system 的开源程度
最强大 AI system 的开源状态可以粗略被分为:(1) System 的所有信息开源;(2) 信息 partially 开源,且有使用权;(3) 信息完全不开源,有使用权;(4) 无信息,无使用权。开源程度决定了,群众(即,非 AI system 所有者)是否可以仍然利用 AI 进行价值创造。
开源程度主要被模型能力所影响:假设模型可以完成 AI 所有者能够期待的所有愿望,则 AI 会完全不开源(甚至并不会商业化而获利),原因是所有者没有任何动力进行开源或开放使用(AI 可以解决他的一切愿望)(例外为 AI 所有者认为开放 AI 符合自己的价值取向)。当前(至少大部分)AI 仍在开源的原因是,AI 所有者仍然希望获利,而获利的背后原因仍然是因为某些需求不能被直接满足,而需要和社会进行外部交换。
当然,一个因素导致这种情况发生的可能性非常小,原因是技术存在天然的扩散趋势(机构研究者有离开机构并组建新的团队、获得可能更大的经济回报,or 出于价值体系的分歧而离开机构并传播技术,此时技术发生自然扩散)。
总之,只要 AI 并不能 without human in the loop 完成所有者的所有需求,AI 所有者就会试图引入人类满足该需求,比如,通过开放 AI 使用获利进而购买人类生产力,或者直接购买人类生产力试图解决该需求。重要的是,如 A2 所说,新需求永远会产生。加上上面提到的技术扩散因素,所以较大概率 模型长期处于可 access 状态。
(注意上述推理与现在的 auto-research / recursive improvement 不矛盾,我们提到过,"human in the loop" 指的是广义的,即,无论任何的 self-improvement,只要人类 add in the loop 能提供比对等电力 AI 更高的价值,人类就会被引入。)
这就意味着,未来长期的大概率事件是,powerful 的 AI 模型长期可被大部分人 access,但是构建 such AI system 的方式掌握在少部分人手中。因为开放技术细节的经济动力太小,需要依靠个人的价值倾向,而因为价值倾向而开放 AI 技术细节的概率是极小的。
3. 人工智能对于物理世界的理解能力
这一维度和 1,即,人类的 marginal value 之间有很强的相关性,但重要区别是,在 1 中,AI 的强大影响力只在各个行业内生效,而 AI 对于物理世界的理解和行动能力会更广泛地影响更多因素,在此之中有两个极其重要的:AI system 的开放程度 和 AI 安全。
AI 开放程度:我们刚才提到,只要有些需求在人类加入后能够有更高的 marginal value,人类就会被引入。以我们目前对于 AI 能力的理解,对于(3d)物理世界的理解能力是一个远比自然语言要更难的任务,原因是自然语言是一个已经被精炼提纯化的结构性表示,而面对物理世界,人类没有形成这样一个高度结构化的表示,或者说,结构化仍然是在我们大脑内部完成的,而没有一个机器能够直接利用的产物,所以,某种程度而言,机器面对物理世界,是和 2000 万年前的我们的进化程度一样的。这样一来,几乎任何涉及要和物理世界交互的领域,我们仍然需要人类参与,而这增加了 AI 所有者开放 AI 的动力(如上,2)。
AI 安全:这是显而易见的。在当前的神经网络中,我们已经发现某些神经元的激发可能对应着某种目标或某种情绪,而我们更加不确定这件事会在未来发展到什么程度。这里唯一使得我们感到些许心安的,也许就是目前 AI 的训练方式和人类进化的方式仍然有一定的不同,即,当前 AI 的训练目标(比如学习某种分布)vs 自然选择。但是,我们仍然不能确定 AI 是否可以通过自然语言的学习,以不同的方式殊途同归,即,develop 出自我目标,届时,有自由在物理世界行动的能力将是需要被严重考虑的。
4. AI 与医学
这一点在当前文章中简略,原因是它涉及到更广泛的 topic:
- 当人类寿命显著延长,但科技水平又不足以确保只增不减的人口在地球存活时,会发生哪些社会活动?
- 如果人类的各种能力能够被 AI 科技所增强(比如广义的智力、记忆力等),那么 1 中所说的 marginal value 又会出现变化。
可以看到 1、2、3、4 这些因素互相影响,相互制衡,他们在某一时刻的状态帮助共同定义了当前世界的 AI 与社会格局。
结语:我们究竟在追求什么
本文假设人类追求美好生活可以被简单归类为如下三个过程:
- 投入完成某件事情的过程;
- 和志同道合的人发生交互的过程;
- 人类对物质 / 物理世界的体验过程。
One 可以根据上面修改 / 明确 / 清晰化自己希望追求的目标,结合这 4 个影响因素在未来不同时间段的具体取值,分析不同时间段社会变化的方向,进而决定自己在不同时间段的行动方向。