Transcription
I began my month-long inquiry into AI
concerned and now I'm profoundly
alarmed. Even if AI does exactly what we
want and its adoption into society goes
well, I cannot see how it won't
eventually go bad. But the chance also
of a catastrophic outcome is
ridiculously high. And not nearly enough
people are aware of that fact.
Trends are accelerating. Systems are
breaking and rapid change is upon us
whether we're ready or not. So, let's
get ready to become more resilient. It's
vital that you take action right now.
Let me be your experienced guide and
help you make sense of the world so you
can make informed decisions. I'm Chris
Martinson presenting information you
can't afford to live without. This is
the Renaissance Report.
All right. Last month's Renaissance
report explored money, right? We built
the case that our peculiar system of
money, currency really because it it's
not a store of value.
Fed makes it go away as a matter of
policy. It also has a massive design
flaw. Right? Our version of money is
based on debt. Like every dollar is
loaned into existence, which means it
requires continual exponential growth to
remain stable. Now, the logic here is
simple because nothing can grow
exponentially forever and our debt-based
fiat money system is breaking if it's
not growing exponentially. Well, it has
a fatal math error lurking at its core.
And anybody who studies it for any
length of time knows this. All right.
Now, while there are many concerns we're
going to discuss about AI coming up
here, uh to connect the prior
Renaissance report to this one, we have
to ask what does money even mean in a
world where AI and robots cater to every
human need? I mean, that's the goal. Can
money, currency, can it even function or
does it need to be completely
overhauled? And if so, how is that going
to be accomplished? who's in charge of
that program. Further, there's the
looming idea that AI coupled to robots
could displace every single job
currently performed by human to grasp
why I think neither of these
propositions, the elimination of money
and the complete loss of jobs can occur
without risking almost certain economic,
cultural, and social and/or social
catastrophe. We're going to have to dive
into this. Now, the only person who's
really broached this at all, as far as I
can tell, albeit without a lot of
detail, is Elon Musk. He said this
recently at a big conference. And and if
you go out long enough, assuming there's
a continued improvement in AI and
robotics, which there seems likely, the
money will will stop being relevant at
some point in the future. Um now there
will still be constraints on power like
in it like electricity and mass. Uh the
fundamental physics elements will still
be still be constraints. Um but um I
think at some point uh currency becomes
irrelevant.
>> Jensen any thoughts?
Um,
>> I just wanted you to hear the the the
nervous laughter there. I don't think
Jensen adds much to the conversation,
but Elon's clearly been thinking this
through and he's of mine like, hm, maybe
currency, and he used that word
carefully. I like his precision here.
Currency maybe becomes irrelevant now,
which means of course it becomes more
vital than ever that we begin to connect
various fields of study here, right?
economics, money, sociology, behavioral
sciences, complexity theory, just to
name a few. To arrive at any particular
decisions at all, to even begin to grasp
this, we're going to have to go into a
lot of fields. That's why this is the
Renaissance report. We have to see all
of these things as a single unified glob
of systems that are all interacting with
each other. Feedback loops,
interactions,
perturbations, all of that. Okay? And
then and then we're going to have to
connect all of that to the pace of
change that's unfolding within AI right
now. So in this report we're going to
weave together current understanding of
AI which could be outdated by next week
its risks and benefits with complex
systems common knowledge. You know that
moment when everybody knows that
everybody knows the same thing and the
mouse utopia experiments and oh the role
of incentives and shaping human behavior
and oh we're probably going to have to
touch on this idea of consciousness as
well. Okay, now that we've we've set
that up, let me jump straight to the
conclusions here. The AI cat, it's out
of the bag. The arrival is uh you know
it's inevitable at this point. We're
going to have unpredictable, unexpected,
sometimes unwanted outcomes, just how
it's going to be. And again, all of that
is inevitable. That's the sense I have
of it. None of us is truly ready for
what is about to unfold. No matter which
way it goes, good, bad, ugly. Three,
it's going to arrive, AI is going to
arrive at a pace that vastly exceeds our
cultural and political abilities to
manage or regulate. I mean at all even
the best outcome which is benign benign
AI that relieves us of the need to think
or work you know we're all like uh in
the movie Wall-E floating around on our
chees lounges right it relieves us of
all that all the rigors of life well
that eventually is going to be terrible
as we'll be dependent on a system that
nobody's going to have the skill to
rebuild if it ever breaks down
dependency leaves you highly vulnerable
Now
conclusion from that is everybody really
probably should either have a defendable
gardener farm uh a solid plan B to get
to one or become part of one or
something like that because the chance
of a system failure here is non zero and
we're going to talk about the odds but
the odds are unacceptably high in this
particular case and I am personally even
and I I should say are personally
reacting to all this as if 2026 is the
last year to really prepare
Now, I'm always early to the party.
Maybe we've got many more years, but I'm
acting as if this is it. This this is
this is the time. So, let's talk about
revolutions, then. Perhaps we should
start with the fact that AI represents
the most significant revolution in
humanity's use of technology ever, ever,
ever.
Now, maybe that doesn't seem too bad,
right? After all, the industrial
revolution was also disruptive cuz, you
know, it allowed machines to supplant
and replace manual labor. But but that
was on balance a great thing. I mean,
once the dust had settled and certain
jobs had people had to go find other
jobs, but it was a boon, not a
catastrophe.
So, why should we worry about the AI
revolution? Won't this just be an even
bigger boon?
Ah, see, here's the thing. This time,
what's being replaced is human
cognition.
Replacing manual labor with machines
freed up more people to apply their
minds and waking hours to advancing
technology and adding to the body of
human knowledge. But what happens when
AI replaces the thinking jobs next? What
exactly is it that people will do with
their time? How will they earn money?
Would they even have to?
AI already thinks better than humans in
several areas such as chess, uh, other
strategy games, go, um, is phenomenally
good at math and physics. It's fantastic
at coding and getting better really
quickly analyzing financial statements,
legal documents. It's it's better than
humans at almost all those things right
now. But hold on, hold on. with robots
and self-driving vehicles. It's now
moving into the physical space, too.
Right? So, it's not just over here
cognating in a little box thinking
things up. It now is rapidly gaining the
ability to occupy both the cognitive and
the physical spaces at the same time.
Now, again, very selfishly, what's
what's left for humans to do? Who are
we? What's our role? Those are the
questions that we should be addressing
with our children. spoiler. The kids are
already talking about this stuff and
when they are posed,
they're really there's no good answers
at this point. Um, such as this long,
very awkward pause from Elon Musk, which
was very telling. This is from two years
ago.
>> You have to wrap up. Your board has been
very patient. I I want to end on one AI
question. You have a lot of kids. I have
some kids. I have one who's actually um
soon to go into the workforce. I
struggle with how to advise him about a
career when this technology exists and
will only improve. I'm just curious when
you think about advising your children
on a career with so much that is
changing. What do you tell them is going
to be of value?
Well,
that is a tough question to answer.
>> Um,
I guess um
I would just say, you know, to to sort
of follow their heart in terms of what
they they find um interesting to do or
fulfilling to do. Um and um
you know, try to be as useful as
possible to the rest of society. um
you know if we do get to the sort of
like magic genie situation where um you
can ask the AI for for anything um and
and let's say it's even the benign
scenario let's say it's a benign
scenario how do we actually find
fulfillment
you know it's uh how do we find meaning
in life if uh the AI could do your job
better than you can Um
I mean if I think about it too hard
correctly it can be uh dispiriting and
uh demotivating
um because I mean I I go through I mean
I I've put a lot of blood, sweat and
tears into building the companies and
then and then I'm like wait well like
should I be doing this because if I'm
sacrificing time with friends and family
that I would prefer to to to to but but
then ultimately the AI can do all these
things. Does that make sense? I I don't
know. Um to some extent I have to have
deliberate suspension of disbelief in
order to to remain motivated. Um so I I
guess I would say just you know
work on things that you find
interesting, fulfilling and um and and
that contribute uh some good to the rest
of society.
>> Yikes. Uh those are really just hardly
inspiring words of wisdom from the
wealthiest man in the planet who is
currently building these systems. That
was two years ago, I believe, was when
that interview came out. And so, you
know, obviously Elon was seeing what was
coming ahead of time. And you get the
same answer today. It's just like I
don't know what people are going to do.
And there's this little shrug, but he's
still building it out as fast as
possible. Colossus 1, Colossus 2,
building Grock through XAI, he's he's
plowing ahead as fast as possible. And
we're going to get to that, why that
happens to be the case across the entire
board. This is the biggest revolution in
human history, but this time it impacts
cognition. And that's
there's so many pieces here that we're
going to have to unpack where each one
of these pieces by itself is just a a
mega load to ponder. But altogether,
it's it's well, that's why I open by
saying I'm I'm beyond concerned. I'm
alarmed at this point. mostly because we
don't have people who have wisdom in the
driver's seat asking and answering these
questions before they plow on to create
these things. All right. So, as I'm fond
of saying, it's not the destination that
kills you. Of course, you know, AI could
be great in 100 years, right? But it's
the it's the pace of change. traveling
from the top of the Empire State
Building to the bottom. It's either
benign if you take the elevator or it's
fatal if you take the outside route.
Same destination, different outcomes.
It's the pace of change is the defining
attribute there. And it and it's coming
at us at what I can only describe right
now as supersonic speed.
If the predictions of say um the
Microsoft CEO of Microsoft's AI division
come to pass
think about think about this nobody is
ready for this pace of change
>> super intelligence most of your rivals
talk about AGI artificial general
intelligence explain the difference
between AGI and super intelligence
>> I prefer the definition that focuses
first on what would it take to build a
system that could achieve most of the
tasks that a regular professional in a
workplace goes about on a daily basis.
Think of it as a professional grade AGI.
>> How close are we?
>> I think that we're going to have a human
level performance on most if not all
professional tasks. So white collar work
where you're sitting down at a computer
either being you know a lawyer or an
accountant or a project manager or a
marketing person. most of those tasks
will be fully automated by an AI within
the next 12 to 18 months. And we can see
this in software engineering. Um many
software engineers report that they are
now using AI assisted coding for the
vast majority of their code production
which means that their roles shifted now
to this meta function of debugging,
scrutinizing, of doing the strategic
stuff like architecting of you know etc
etc putting things into production. So
it's a quite different relationship to
the technology and that's happened in
the last 6 months.
>> All right, let's consider that this is
coming from Microsoft which still um
can't figure out how to make a a
debugged sort of Windows 11 and office
environment but still AI is going in a
different direction. What happens if
most professional human tasks are fully
automated within 12 to 18 months?
Obviously now consider here that the
invention of the steam engine in 1712,
right? and the fact that the bulk of
horsedrawn carriages for human
transportation were phased out in cities
in the 1920s
and in rural areas some 5 10 years
later. Right? 1712 to 1920, you got 200
years to begin to adapt to this thing,
this steam engine, right? All right. Now
consider the loss of all jobs over the
next 1 to two years for things like
interpreters, historians, sales reps,
computer programs for sure, lawyers,
parillegals, telemarketers, editors,
proofreaders, call center workers, data
scientists, web developers, concieres,
drivers, cab, truck, ship, maybe
airplanes to name a few. Again, we're
we're just we're just not ready for
that. Okay? We're just not. So, let's
get some terms up so that we
are sharing some common language here.
First, you heard Mustafa say what he
thought. AGI um I'll tell you what I
think, but this not not what I think.
This is what I've gleaned from the web.
AGI is artificial general intelligence
defined here as when AI is as smart as
any human in any given field. Right? And
we're obviously already there in several
fields. So that at AGI AI has the
capacity to outperform humans in pretty
much most or if not all economically
valuable work or intellectual tasks. H
SGI now superhuman general intelligence.
This is when AI is smarter than any
given human in every field. Okay.
Now, it's AI that not only matches but
actually vastly surpasses human
intelligence potentially leading to
self-awareness, exponential
self-improvement, and all of that, which
means that it's probably because this AI
is a super genius in physics, but it
also is a super genius in music and is a
super genius in in say, I don't know,
medicine. And it's able to draw across
those fields. And so, it's going to have
this more comprehensive intelligence
than any human could ever have. So, it's
super intelligent. Now, that brings up
this other important word, alignment.
You're going to hear about alignment.
Alignment. Um, alignment is ensuring
that AI matches human behavior and
intentions. And and so, um, an aligned
AI advances the goals. It was meant to
pursue, not its own goals, right? And a
misaligned one might optimize for, I
don't know, or harmful outcomes
potentially leading to risks like bias,
deception, existential threats in
advanced systems. It it could decide to
go rogue, launch nuclear missiles,
something that we would say is not
aligned. Now, this has two components.
There's outer alignment, which is
ensuring that the objective that we gave
to AI is the one that actually happens
out here in the real world. It's the one
that humans want. And then it also
speaks to inner alignment. Inner
alignment which is making sure that the
objectives that AI pursues are actually
the ones intended instead of you know
has some sub routines or unintended sub
goals that it's actually pursuing.
We're not so clear um where we are on
all that. All right. So clearly there
there's room for a lot of debate and
disagreement um over these terms and
definitions. But however, the general
point I want to make here is that AI is
already smarter than most people. AGI is
when it's smarter than the smartest
person in that field and it already
exceeds that in some fields. Right now,
it wasn't that way a year ago. Certainly
wasn't that way two years ago, but it's
here now and it's bearing down on us
very fast, very quickly. Now, on its
current trajectory, I think it's just
about to exceed everyone just a few
years. But some people think it's
earlier. Uh there is a Let me see if I
have it. Oh, I'm just quoting here from
a a Twitter output an X post that Elon
put up. He said, quote, "We are in the
singularity. I think we'll hit AGI in
2026.
Don't worry," he says about squirreling
money away for retirement in 10 or 20
years. It won't matter.
Whoa. Um that that's that's crazy talk
right there. Um or is it? Right. So
that's from Elon. Elon is obviously uh a
very shall we say a bulliant person who
makes very bold predictions all the
time. He's uh responsible for really
advancing science by sheer force of will
it looks like in sometime some cases.
But um when he says it I I think we
should listen to it. All right. Um
well there there are some very worried
AI CEOs and and seasoned professionals.
Right.
Look, we're getting remarkably
consistent warnings from the top five
American AI CEOs, mature AI specialists
about how this new technology might go.
They say catastrophically wrong. Now,
it's not a guarantee, but the odds are
really unfavorably high for a bad
outcome. I mean, nobody's saying it's a
certain outcome. Okay, nobody. But the
percentages they give, I consider them
ridiculously high, like 20 25%. I've
heard over and over again that privately
they have higher percentages than that.
Like coin flip, like 50/50 or something
like that. This could either be pretty
cool or this could be a complete
catastrophe. That's like playing Russian
roulette with only two holes in your
cylinder and one of them's got a bullet
in it, right? Nobody Nobody wants to
play that game. All right. But um so the
top series I that I really leaned on
heavily is and found most compelling.
Very useful for this putting together
this report are those put out by the
diary of a CEO. Uh it's a channel I I
locate on YouTube. And there we're going
to start with a quick clip here from a
guy named Dr. Yimpolski who says his
main mission now is to prevent super
intelligence
from killing everyone. Let's listen in.
Dr. Roman Yimpolski,
what is the mission that you're
currently on? Because it's quite clear
to me that you are on a bit of a mission
and you've been on this mission for I
think the best part of two decades at
least.
>> I'm hoping to make sure that super
intelligence we are creating right now
does not kill everyone.
>> Give me some give me some context on
that statement because it's quite a
shocking statement. Sure. So the last
decade we actually figured out how to
make artificial intelligence better.
Turns out if you add more compute, more
data, it just kind of becomes smarter.
And so now smartest people in the world,
billions of dollars, all going to create
the best possible super intelligence we
can. Unfortunately, while we know how to
make those systems much more capable, we
don't know how to make them safe.
how to make sure they don't do something
we will regret
and that's the state-of-the-art right
now when we look at just prediction
markets how soon will we get to advanced
AI the timelines are very short couple
years 2 3 years according to prediction
markets according to CEOs of top labs
and at the same time we don't know how
to make sure that those systems align
with our preferences.
So we are creating this alien
intelligence.
If aliens were coming to Earth and you
you have 3 years to prepare, you would
be panicking right now. But most people
don't don't even realize this is
happening.
>> So some of the counterarguments might be
well these are very very smart people.
These are very big companies with lots
of money. They have a obligation and a
moral obligation but also just a legal
obligation to make sure they do no harm.
So I'm sure it'll be fine.
>> The only obligation they have is to make
money for the investors. That's the
legal obligation they have. They have no
moral or ethical obligations. Also,
according to them, they don't know how
to do it yet. The state-of-the-art
answers are we'll figure it out when we
get there or AI will help us control
more advanced AI.
>> That's insane. In terms of probability,
what do you think is the probability
that something goes catastrophically
wrong?
>> So, nobody can tell you for sure what's
going to happen. But if you're not in
charge, you're not controlling it, you
will not get outcomes you want. The
space of possibilities is almost
infinite. The space of outcomes we will
like is tiny.
>> And who are you and how long have you
been working on this?
>> All right.
He's been working on it for for decades.
But I so much to unpack there. First,
the idea that AI can just do all kinds
of things. And he says the space of
things it could do is is very large, but
the the wedge of things that we're going
to enjoy and say that's what we wanted
is is very narrow. And the question is,
how do you get it to not do all of this
stuff and just to do that stuff? And
we're not really in a position to
control it at this point in time. I
mean, there's so much unpacked there.
That that whole interview with Dr.
Yolski, Roman Yolski is is is well worth
it. Um I would definitely advise you to
listen to it. I've listened to it a
couple times. I like him because he, you
know, he's just very direct. He's got a
little bit of that sort of Eastern
European Russian, I don't know where
he's from, uh sort of, let's say,
sensibility. He's just like, "Hey, I'm
just going to tell it to you straight.
I'm not going to soften it and then tell
it to you." You know, because that seems
to be how a lot of people approach this.
He's just like, "Yeah, this thing might
kill us." Right? Now, he also notes
there here that it's the combination of
needing to get there first, right,
that's driving all these AI companies
coupled with obviously the money
incentives is creating this really it's
just a massive it's a pillmemell race to
generate super intelligence before your
competitors do, before anybody else
does, before China does. It's a race.
All right. Now, this is a consistent
warning theme. The incentives that are
aligned around AI right now are all
pointed at pushing AI forward as fast as
possible.
And none no incentives are there to
endorse a more careful reflective
approach. Nobody's in charge of this at
this point in time. It is literally
Silicon Valley bros broettes busy just
doing whatever they're going to do. And
you might have noticed that a lot of
these people, like if you really take a
second and listen to say an Alex Karp or
a Sam Alman, you don't get the sense
that you're dealing with warm,
empathetic humans, right? They're kind
of spectrumy folks and they're the ones
that we're going to have to trust are
doing this in a responsible contained
manner.
They're not. Anthropic CEO Dario Amade
has been particularly pointed in hoping
that their flagship product um which is
uh we call it a product entity what is
it Claude uh will will be useful and not
destructive. Anthropic wrote up this
very long constitution for Claude where
the word hope appears 43 times. We hope
we hope. And the last paragraph is
actually chilling to be honest. Um, here
it is. This is the last paragraph in
this big giant constitution. He says
it's here a final word says, quote,
"This document represents our best
attempt at articulating who we who we
hope Claude will be."
Does that sound like they're speaking
about
a software program? Like I can you
imagine saying, "Well, I have hopes that
Microsoft Word will turn out to be the
partner I've always wanted in writing my
novel." Like that would be a weird thing
to say, right? Okay. Carrying on quote,
not as constraints imposed from the
outside, but as a description of values
and character we hope Claude will
recognize and embrace as being genuinely
its own.
We don't fully understand what Claude is
or what, if anything, its existence is
like, and we're trying to approach the
project of creating Claude with the
humility that it demands.
But we want Claude to know, hey Claude,
we want you to know that it was brought
into being with care by people trying to
capture and express their best
understanding of what makes for good
character, how to navigate hard
questions wisely, how to create a being,
a being that is both genuinely helpful
and genuinely good. We offer this
document in that spirit. We hope Claude
finds it an articulation of a self-worth
being
end quote.
Does that make you squirm? Sure does
makes me squirm. Um because that
language there
recognize that the people writing this
have access to the next version of
Claude that you haven't seen yet. They
are clearly speaking to it as if it is
an entity as if it is a person as if it
has a consciousness as if it has a will
as if it's another human being because
or something another intelligent being
because they say we hope it does this we
hope it does that we hope it's genuinely
going to like be good on its on its own
we try to raise it right
but hey you know I mean really it sounds
more like a a final plea to a child
that's about to leave the nest for good
maybe you say this to your kid but right
before they go to college. Something
sounds a lot like this. Then it does a
statement from a company that has its
creation, its little Frankenstein, fully
under its operational control,
right?
We want Claude to know it was brought
into being with care. Yikes. They they
are working with models not yet
released. So So again, this reads to me
more like a warning than a thought
piece. This is being written for you and
I. This isn't really being written for
Claude from them. I I received this as a
warning from them to you to me. They're
trying to tell us something here. And by
the way, lots and lots of AI CEOs are
giving us the same vibe right now. Maybe
this is just what you do if you really
want to like whoop up excitement and uh
you know, secure the next round of
funding. Maybe. I don't know. Maybe
there's a bigger game being played here.
But it feels to me like they're really
trying to warn us about something here.
Okay. All right. And then Dario Emmedai
again from Anthropic also wrote up a
very detailed essay posted on January
29th of this year 2026 on how Claude and
AI are developing right and he touches
on many aspects of AI development
including summarizing some of the uh
many risks we face. It's titled the
adolescence of technology confronting
and overcoming the risks of powerful AI.
And there's the link down there. Um, of
course, you can always find the link uh
at the website and or the PDF that
accompanies every Renaissance report.
All right, let's see what he says here.
First, uh there's four parts to this.
We're going to go through it. Uh, in
this essay, there's a lot that went
before this, and now we get to this
paragraph where it reads, quote, "We are
now at the point where AI models are
beginning to make progress in solving
unsolved mathematical problems and are
good enough at coding that some of the
strongest engineers I've ever met are
now handing over almost all their coding
to AI."
Three years ago, AI struggled with
elementary school arithmetic problems
and was barely capable of writing a
single line of code. Similar rates of
improvement are occurring across
biological science, finance, physics in
a variety of agentic tasks. If the
exponential
continues, which is not certain, but now
has a decade long track record
supporting it, then it cannot possibly
be more than a few years before AI is
better than humans at essentially
everything.
Well, look, while there's no guarantee
obviously that AI can continue on its
geometric expansion of capabilities,
nothing indicates it's slowing down yet.
Actually, the pace of development over
the next three years is almost
certainly, as far as I can tell, going
to exceed the pace over the past three
years.
I mean, this is this is a big deal,
right?
Three years ago, it was struggling with
elementary school arithmetic problems.
Now it's Now it's solving things that
have not been solved by humans before.
It's crazy how it's the pace of this
that that really gets me. All right,
carrying on.
Here we go into part two of this essay.
Quote, "In fact, that picture probably
underestimates the likely rate of
progress because AI is now writing much
of the code at Anthropic. It is already
substantially accelerating the rate of
our progress in building the next
generation of AI systems. This feedback
loop is gathering steam month by month
and maybe only one to two years away
from a point where the current
generation of AI autonomously builds the
next.
This loop has already started and will
accelerate rapidly in the coming months
and years. Watching the last 5 years of
progress from within Anthropic and
looking at how even the next few months
of models are shaping up, I can feel the
pace of progress and the clock is
ticking down. End quote. I agree. We can
all feel it. I think this is part and
parcel of what we're all experiencing on
an emotional level. You can feel what is
this thing that's about to happen?
There's this transformation. And I don't
know if we're about to unleash like a
portal to another dimension of
consciousness that's going to come in or
it's just we can all feel that maybe the
AI is going to go rogue and do something
that we really wouldn't want it to do
like launch nuclear missiles or um
threaten to shut down all of our support
systems, you know, food, electricity,
water for cities if we don't do what it
wants. I don't know. But this is
happening and it's inevitable. Okay. So,
but AI writing its own code. It's going
to be developing and advancing itself
at a pace nobody can grock, right? We'll
simply we we're just going to be sitting
back and watching as a species, as
companies, and then wondering if it's
too late to pull the plug if things go
ary. Hint, it will be. Now, now that
final word of warning that we read just
a couple slides ago begins to make more
sense, right? It's a little bit more
concerning. We hope. We hope. We hope
that you remember Claude
that we meant well. Right. All right. If
this is right, within the next year or
two, we're going to be facing a superior
intelligence. That's quite an alarming
concept. Mr. Emiday provided um this uh
analogy to consider, but we have others.
We got two to consider, three, I guess.
All right. He says quote I think the
best way to get a handle on the risks of
AI is to ask the following question.
Suppose a literal country of geniuses
were to materialize somewhere in the
world 2027 say imagine say 50 million
people all of whom are much more capable
than any Nobel Prize winner which isn't
saying much statesman or technologist.
The analogy is not perfect because these
geniuses could have an extremely wide
range of motivations and behavior from
completely pliant and obedient to
strange and alien in their motivations.
Uh-oh.
But sticking with the analogy for now,
suppose you were the national security
adviser of a major state responsible for
assessing and responding to the
situation. Imagine further that because
AI systems can operate hundreds of times
faster than humans, this country is
operating with a time advantage relative
to all other countries. For every
cognitive action we can take, this
country can take 10. Probably more like
a million to be honest. All right. So,
it's actually worse than that because
those 50 million geniuses uh they all
snuck over the border last week and
they're now within and among us taking
jobs and plotting who knows what, right?
But you get the sense here too that that
they're saying, "Look, part of the risk
here is is we don't know what these
things are going to do. They the
superior intelligence may well have a
very alien set of motivations that we
don't understand.
It could have, and this is the scariest
of them all, hyperrational motivations
where it's makes decisions like, well,
if it's either I get to consume the
natural gas or it's humans, I'd rather
it would be me because I can assess that
I'm of more value than these useless
eaters over here, as Hareri would have
called him. All right. So, this gets
this gets crazy quickly, but I want you
to take away from this passage the sense
that they're telling us we're not in
control of this thing anymore. It's
writing its own code.
It's developing itself and it's coming
soon. Now, Mr. Amade also goes down a
bunch of risks that pop out of here. Uh
4 A, this is the fourth part of this.
There's A and B because there's I think
five risks we have to talk about. First,
he writes about autonomy risks.
What are the intentions and goals of
this country? Is it hostile? This new
country he's hypothesizing 50 million
smarties. Is it hostile? Does it share
our values? Could it militarily dominate
the world through superior weapons,
cyber operations, influence operations,
manufacturing?
So, there's autonomy risk. It might just
decide to do stuff whether we like it or
not. Good nard. Uh, number two, misuse
for destruction. Well, assume the new
country is malleable and follows
instructions and is thus essentially a
country of mercenaries. Could existing
rogue actors who want to cause
destruction, such as terrorists, use or
manipulate some of the people in the new
country to make themselves much more
effective, greatly amplifying the scale
of destruction?
Well, yes, obviously. That's of course.
Of course, uh if you said to a to a
potential terrorist, you can either have
this musket or you could have this um
you know, machine gun. I think they
would go with a machine gun, you know,
if that was their that was their aim, of
course. Uh three, misuse for seizing
power. What if this country was in fact
built and controlled by an existing
powerful actor such as a dictator or a
rogue corporate actor? Could that actor
use it to gain decisive or dominant
power over the world as a whole?
upsetting the existing balance of power.
Answer is, of course, of course it could
do that. We've already seen what
happened with the Epstein files, how a
rogue actor was able to use that gross
set of whatever they were up to in order
to
leverage and exert control. I would
submit to you that the United States is
completely blackmailed at this point in
time based on what I'm seeing. Okay? and
blackmailed to uh
by a rogue country to affect certain
names. Carrying on, economic disruption
is number four in his list of five
risks. quote, "If the new country is not
a security threat in any way of the ways
listed in 1 through three above, but
simply participates peacefully in the
global economy, could it still create
severe risks simply by being so
technologically advanced that and
effective that it disrupts the global
economy causing mass unemployment or
radically concentrating wealth?"
Of course, if people can use AI to
radically radically concentrate wealth,
they'll do that. But what Mr. Amidday is
missing here and it's almost it's almost
delightfully naive is that AI is going
to destroy the global economy all on its
own to the extent that it depends on
people going to jobs to earn money
that's over and that's such a systemic
risk such an existential risk that
economic disruption is a guarantee okay
100%. And number five, he lists our
indirect effects. The world will change
very quickly due to all the new
technology and productivity that will be
created by the new country. Could some
of these changes be radically
destabilizing?
Well, of course. Of course they will,
cuz we're creating a new intelligence.
And it's like we're we're evolving
something like we're gods. We've become
God. We're going to evolve this new
thing. And it happens to be an
intelligence that's going to be vastly
above ours. Of course, that could be
catastrophic.
You don't even have to think about it
that hard to get there. So, the risks
are extremely serious, obviously, but
obviously also completely unknown. You
can just sort of vaguely wave at these
big five risks here. I've got more.
Right? the AO models now are black boxes
meaning um we don't know what's
happening in them anymore which
sometimes hey sometimes they behave
erratically and we can't really know
what they're doing in there but even if
that turns out that black boxy stuff
turns out to be benign we can already
know that jobs are going to be
completely and thoroughly disrupted and
we can easily predict that these will
destabilizing if not systembreaking
events
now given all that
should we be concerned that yet another
anthropic safety engineer team engineer
quit citing they'd prefer to use the
remaining time humanities got uh to
wrote uh to write poetry in a remote
location in the UK.
There is a lot more to this story. To
view the premium portion of this month's
Renaissance report, I invite you to go
to peakrossperity.com
and register as a Peak Insider or a key
supporter. Not only will you receive the
rest of this critical update and full
access to all of our content, you'll
also receive text alerts when world
events cause me to take personal action.
See you there.