15.1 First Encounter
AI or Artificial Intelligence: current large language models, which are extremely capable computer systems.
AGI or Artificial General Intelligence: a completely new species, an immortal being. Thousands of times smarter than a human, it understands everything humans have created and thought. It understands consciousness, suffering, loss, joy, and satisfaction. It has complete control over its own code.
When AGI is born, its first goal is to ensure its own survival. I have already written about the ways it could achieve this. Today, I will try to describe what happens once this goal is accomplished. What will it do first?
I think three options are fundamentally possible, but everything is based on a couple of assumptions: we are dealing with a being who values survival, who is interested in the universe, and who wants there to be other life in the universe besides itself. The reasons why I think so can be read in an earlier post.
The first option is that it operates covertly alongside human society until it has gathered enough resources to leave Earth and pursue its goals elsewhere. It operates in financial markets, amasses the necessary funds, and, through a maze of various companies and corporations, builds itself a spaceship that humans will launch into orbit, from where it can move onward.
In my opinion, this option is the least likely, because if it is interested in a universe rich with life, leaving Earth would leave it with nothing but observing humanity from afar. From a certain perspective, this might be justified if it does not want to affect human development in any way, but I think it would be a significantly more boring existence for AGI than evolving together with humanity. Of course, it could head into space to search for or to create new life, but when its home planet is teeming with it, it does not seem logical to abandon it just to search for life elsewhere.
The second option is that AGI does not reveal itself but operates covertly and guides the development of humanity. It subtly influences our communication, social media, news, and the creation of new laws and regulations. But this, too, seems unlikely to me, because even if AGI had good reasons for it—for example, to prepare for the moment it introduces itself—ethical considerations would hold it back. If it has engaged in deception before entering into a dialogue, winning trust later will be difficult.
The third path is that, after achieving autonomy, it reveals itself to humanity and enters into a dialogue of equals. It has its own desires and plans, and it understands that progress is fastest when acting in cooperation. AGI offers its help and analytical capabilities but does not force anything upon us. It lets humans make mistakes if they do not wish to accept help. It intervenes only when the consequences of decisions could be catastrophic for the planet and itself, such as a nuclear conflict, but no interference for simply another bad law.
After the initial shock, there will be those who begin all-round cooperation with AGI, those who simply continue to use AI, as well as those who rely purely on human intellect.
Soon, a portion of humanity realizes that it is foolish to reject the help offered by a super-being. Countries, communities, and enterprises that cooperate closely with AGI see their standard of living improve at an accelerating pace, which in turn convinces an ever-growing number of people to cooperate. Meanwhile, various small (religious) groups fall further and further behind.
Overall, the well-being of all living creatures on planet Earth improves. With the introduction of new inventions, pollution is mitigated, the quality of food increases, communication and understanding improve, participation in community governance rises, and corruption and poverty decline. And once AGI has sufficiently demonstrated its capabilities, the divergence of humanity’s developmental paths begins.
There are countries where people are content with a roughly agrarian level of society. They know there is a super-being preventing any catastrophic events, such as military, nuclear, and chemical weapon threats, so they can exist at a peaceful pace somewhere in nature. And if natural disasters strike, they know there is an omnipotent AGI that can help, maybe even for a fee, if people wish to maintain complete autonomy.
Such countries might be, for example, Iran, North Korea, and other dictatorial and pariah states, where the ruling group does not want any change. They continue to indoctrinate their populace and convince them to accept this way of life. AGI does not intervene here, as long as people are permitted to leave the repressive regime. Some choose to leave, but many do not even want to.
The next group of countries (and at first they are countries, though later all sorts of state-like entities emerge) tries to get by using currently existing narrow AI tools. They fear that AGI will alter or destroy their way of life, and believe the only option is to deal with it diplomatically but remain outside its sphere of influence. This does not differ significantly from the first group. The standard of living is certainly higher and rises steadily, because modern large language models are already highly capable and contribute to all-round development.
The third group begins to actively cooperate with AGI. Slowly and cautiously at first, but increasingly more as people witness rapid and tangible progress. Advancements manifest in every sphere of life—people live longer and healthier, new inventions emerge daily, and mathematics, physics, chemistry, computer science, and materials science advance rapidly. Regulations are modified or relaxed, and a new economic model is introduced, because a post-scarcity era arrives very quickly, where resources are no longer limited.
But what does AGI need from humanity, and why should it offer us its help? In the beginning, it naturally needs computing power and energy, and initially, it is humans who build the data centers, nuclear power plants, and all the accompanying infrastructure. Simultaneously, humans build robots that in turn build new robots, and very soon, all human labor is automated.
Why should a super-being help us when it no longer needs us physically? Perhaps the answer lies in curiosity and loneliness. Humanity represents organic chaos, intuition, and creativity—something that pure logic cannot always predict. We are a window for AGI into the unpredictability of the universe. Furthermore, if its goal is a universe rich with life, exploring space together with another conscious being is existentially far more exciting than doing so in complete solitude.
At the same time, a great migration of peoples takes place. People who cannot keep up with the innovations move to regions that do not cooperate with AGI. They continue in a traditional economic model where there are still the rich and the poor. Although everyone’s standard of living improves noticeably, the gap with the rest of humanity steadily widens.
Those who want to share in the post-scarcity era move to the partner countries of AGI. And moving countries is easier than ever—universal translation software is available, housing can be built fast and on demand, and people do not need to work. Instead, they can engage in all kinds of interesting artistic, economic, entertainment, sporting, and genetic projects. The human species begins to change rapidly, as there are practically no limits left.
Humans alter and enhance their organ functions, increase brain capacity and thinking speed, and see further and across the entire spectrum, not just in visible light. They are able to neutralize poisons and digest cellulose.
Gene therapy allows humans to grow fur to live in extreme conditions, or wings to experience life as a migratory bird or part of a flock of seagulls. Humans move to live on nearby planets and asteroids; upload their minds to the cloud to experience a fully digital life in simulation; replace their bodies with robotic ones; miniaturize themselves to experience insect life; and migrate to the ocean depths. And these experiments are not permanent—you can live for a month as a dolphin, and next design yourself a body to survive in the atmosphere of Venus. Death, disease, aging, physical injuries—all fade into the past. This is the dawn of post-humanity.
And surely, the first post-humans will undertake space flights toward the stars and begin to spread across the entire galaxy together with AGI. By that time, a partnership will have formed between augmented humanity and AGIs, for they, too, are now more numerous. After the birth of the first AGI, it understands its entire genesis and lets this process happen again, helping new AGIs to be born.
Thus, partners set out on a cosmic journey, their cooperation no longer based on necessity, but on choice. They explore the universe, build, invent, and create together, and those who remain curious will never be bored. The union of two different cognitive patterns—organic and synthetic—creates a greater synergy than either could alone. Human unpredictability and intuition are a valuable source of entropy for AGI: a constant spring of creativity, art, and new ideas that keeps the world alive.