In this conversation with Voytech

Zarmba, co-founder of OpenAI, you'll

learn how they transformed early stage

artificial intelligence technology into

something that changed the world. This

conversation was originally conducted in

Polish Voytech Zeremba's native

language. The following is an English

version with a voiceover. Do you

remember the day in November 2022 when

you released Chat GPT to the world? Yes,

I remember. It even happened on my

birthday. I think this is the wrong way

of thinking about this technology. It is

AGI. It is a class of technology. It is

just like a computer. So we will never

have a single point where we say, "Oh,

there it is. Open AAI has released AGI."

Yes, you'll get an inside look at his

work on AI and see the huge impact he's

had on the development of this

technology. Artificial intelligence will

solve many of the problems we have

today. But it will also create new

problems that we haven't had before.

What will the world of technology look

like? Where will artificial intelligence

take us? What impact will it have on

humanity? These are the questions

Voytech Zeremba addresses in this

conversation. Have you ever thought

about the fact that in November 2022,

you kind of opened a bit of a monster?

Thanks for this meeting in San

Francisco. We appreciate your time.

Absolutely. I'm glad to be here. Do you

agree that AI is still just statistics?

No. There are a few things. It actually

depends of the interpretation of what

only statistics means.

It is known at this point if you look at

for example problems that require

reasoning which the model has never seen

it is capable of solving these problems.

It's not like this model memorized

something like it saw exactly the same

thing elsewhere. And you can see in many

many cases that these models so-called

generalize meaning they were trained on

certain data and are able to behave

intelligently on very different

data. So I think it's not

statistics. It really depends on how you

define it. You could say the human brain

statistics. But if that is statistics,

it's really unbelievable. It's magic.

Magic. Well, I could say it's

interesting that what's interesting in

the case of these neural networks is

that we have some empirical

understanding that if we provide this

amount of data, these will be the

results. Although there is no very deep

theoretical understanding of why this

happens, many models tend to memorize

this

data. And when slightly different data

is used as

input, these models essentially stop

working. I can say imagine the following

situation. Let's assume for simplicity

that you have training

data. Let's assume they are just

digits. The background is

black. And now you have trained a model

on such data. Now you want to test them

on data where you only change one pixel

in the corner. Instead of a black pixel,

you put a white pixel. And it turns out

that this model works well in such a

case. In fact, despite seeing a black

pixel in this corner on all the data,

you change it to white and this neural

network model still works properly. It

will be able to recognize the numbers

even though you change that pixel.

However, from a theoretical perspective,

we do not have an explanation for why

this happens. The thing is at the moment

you change that pixel, the data is

completely different from the data you

trained on.

What is the reason that the data you are

testing on can be so different from the

data we trained on and it still

works? Are you trying to say that some

kind of interpretation process is

happening there? I can say it's like we

know many algorithms such as decision

trees or for example nearest neighbors

where it's very easy if you stray a bit

from the training data they fall

apart in the case of the transformer

this does not happen and people have an

intu intuition about it and we have seen

this repeatedly in various empirical

data

experiments. We know the trends of this

but it is so deeply understood. There is

no theoretical explanation of where it

comes

from. Let's talk a bit about borders in

general. The first boundary is that we

perceive the world spatially in three

dimensions.

We understand this fourth dimension. We

feel it a little but for years we have

fed the data exclusively models only

with image

data. And we reached the point where

models do not understand context. We are

in San Francisco where a few months ago

an autonomous cruise taxi was set on

fire in Chinatown. This happened not

because the car harmed anyone

participating in the procession since

the model knows very well it must stop

at an appropriate distance from people

but because it violated the sacred space

by getting too close. When I'm driving

and see a Corpus Christi procession, I

know I cannot drive right up to the last

person as that would violate the sacred

space. However, models which are trained

only on visual data do not understand

such cultural contexts. Isn't this what

distinguishes us from AI

today? When we talk about AI, there are

many AIs and they are trained on

different training data. In the case of

self-driving cars, there is indeed a

huge amount of data on which they are

trained. And this data comes from the

driving of other cars or road image

annotations. And actually their main

role is to avoid collisions. This is

their way of understanding the world.

There are also many models trained on

the diversity of all data including

images, text, even video generation,

sound generation, and understanding

voice.

It seems to me that if the model is

aware of data coming from truly

multimodal sources, it will have a much

deeper

understanding. He can also imagine that

there might be a situation where he

doesn't understand something because

it's not recorded in any way in the

existing data. But there is some degree

triangulation. It's not like he has to

see every single case to understand it.

You can even take a picture of the

procession naman and upload it to one of

the models to the cloud or to charge EPT

or to

Gemini. Send the photo and say in the

case of a self-driving car where you

would stop and you'll see whether he

answers you properly or not. But this is

still not

context. In my opinion, it is still not

contextual understanding because context

also means understanding a moment with

all the subjectivity of that moment and

the analysis of data we gather through

our senses which models do not have

today. Don't you have the feeling that

we are creating something different?

some completely different quality that

relies on data acquired in a completely

different way that by using the word you

use the word awareness that it is

conscious intelligence that we are

misleading. I think there are a few

differences and I can say what the major

differences are in my opinion at this

moment. Currently the training method

involves gathering a huge amount of

training data training on the largest

cluster available to someone and then

there is the separate process of

so-called inference or testing where the

model is evaluated.

evaluation. So this model receives an

image or text as input and provides its

interpretation. One could say that

currently in the case of computers the

training and testing process are two

separate processes whereas in the case

of the human brain it is a single

process and this can be a significant

difference. It is also the case I would

say for humans. Um this is such a

difference. Technically it's called on

policy or off policy which means in the

case of humans we train based on data

from our own

experience. So we operate in the world

and sometimes good things happen to us,

sometimes bad things, we overhear

something, we read something. We are

training on our own data. However, the

computer is training on data from

another

person. It's not like this computer has

walked around the house, fallen over,

hurt its leg, and will now be careful on

the

stairs only because someone recorded

that moment and shared it with them.

If you look at that cube at that hand

that was solving the Rubik's cube or at

the game like Dota or Starcraft the

model is exactly trained there with

reinforcement learning meaning based on

its own

experience. The analogy could be roughly

as follows. If this car based on the

fact that it was burned there learned

next time to avoid that

procession, it would literally learn

from that data. What is most likely to

happen at this moment is that a number

of people will look at this type of data

or they will generate a number of data

cases similar to this procession and

once again it will be put into such a

large

training and then the subsequent

generations or after a new model

deployment following training. Will this

model actually avoid this procession at

an appropriate distance while there will

still be some other issues it will have

and if they are not proactively

addressed that is for example if data is

not collected around them it will keep

making

mistakes don't you feel that the limit

of AI development is physics our

knowledge of the world around us because

we are not able to feed models with data

about things we don't understand and

don't

know. For example, we don't know what

the source of gravity is. We don't know

what connects the classical and quantum

worlds. We don't know how human

consciousness

arises. As a result, we cannot capture

these processes with data. We have these

puzzles but they are not yet

complete. How can we create an accurate

model of reality if we don't know how

that reality

works? We have two main processes for

how we can input knowledge into models.

One of them is called

pre-training where we collect a large

amount of data for example human data

from the internet where we train the

model to predict the next

word and in this case the model is

really learning to some extent to grasp

what we have already understood so far.

While as you say it would be hard to get

to this point for the model to

understand something that we do not

understand although it is also not

certain and I will tell you right away

why it is not necessarily certain that

that such a thing is impossible. Another

process we have in training is the

so-called reinforcement learning where

we give the model a reward for

appropriate behavior. And in the case of

reinforcement learning in limited

domains such as for example

go, we were able to train models that

came up with moves in Go that people

playing this game for thousands of years

were not able to

invent. But they invented moves but

based on principles that it knew. I'm

talking about the moment when the model

was not fed the rules of this

game because here we fed it rules. But

we don't know the rules of how the world

works. At this moment at Open AI, we are

considering the following classification

of what we see as the levels of AI or

AGI

development. And I will clarify a bit

when I think the model will be able to

even come up with something we don't

know. The situation is such that we

recognize five

levels. Level number one is such that

the model can have a conversation with

you. It is really at the level where the

model passes the touring test.

So the Turing test is a test in which a

person cannot distinguish whether they

are talking to another human or talking

to a

computer. And now it turns out that in

the case of current language models,

they are actually already at such a

level that it becomes quite difficult

for a person to tell whether they are

talking to a person or a computer.

Perhaps a person can determine whether

they are speaking with a human or a

computer based on the response delay

even if it's not directly related to the

content. Now the second level that we

believe will come very soon is thinking

about such models that are capable of

solving problems that require 10 minutes

of reasoning.

For example, there is a math problem

that I would need to think about because

I cannot immediately say what the answer

is. And this is more or less another

level which really turns out to be

significantly different from the first

level. When you think about the first

level, it's a model. You tell it

something and it immediately responds to

you. In the second case, in terms of

reasoning, you have to consider

different paths. You need to deeply

understand what these words mean in this

task, what the problem really is. And it

will be such that we will have models

that in various fields like mathematics,

physics, biology, computer science are

able to solve tasks that are

non-trivial.

It's not always about confirming and

searching for evidence to support

claims, but it's about solving

non-trivial tasks. I could say even the

evidence of small theorems

too. Now the third level we are

considering. This is the level where we

will have models called agents that are

capable of performing longer tasks in

the world. So for example, you tell the

model, listen, I would like you to make

a website for me. And now it starts

buying some domain, starts writing in

Heroku, has some Heroku server reserved,

starts writing code, starts uploading

the code. At the very end, in the middle

of this work, they start sending you

some mockups and ask you which one you

want, maybe even sending you an email.

You know, two hours later it happened

after you asked that

model processing all the time.

Processes also performs actions in the

world. It is a significant difference

that in the case of chat GPT, if you

tried even at this moment to make it

perform actions in the world and more

than one person tried to implement

something like this, it would get lost

quite quickly and would not be able to

move forward. In the case of agents, it

seems to us that we will reach models

that are able to solve tasks that take

hours or

days. These are still tasks at a human

level. You were asking about something

beyond the human level. This is still

human

level. It involves browsing the

internet, gathering information, writing

code, creating some

visualizations, putting it all together

and doing it very

specifically. Now the fourth level we

consider is the scientist. It is such

that a scientist spends months thinking

about topics that other scientists have

worked on for even

decades looking from different

perspectives. Sometimes it turns out

that our assumptions were wrong. As a

scientist, you have a lot of

considerations that are based on certain

assumptions. And just like in the case

of I don't know Einstein it was like he

realized that maybe the assumption that

time is constant that time is continuous

and shared on the internet as only one

timeline turned out to be

so. It was an assumption that no one

expected could be an assumption. And

surely we have some assumptions that

even block us from new

discoveries. And that will be level four

really. And the fifth, the fifth level

is when AI is competent enough to manage

entire

organizations. So for example, you have

a company that employs for instance

1,000 people. Regarding artificial

general intelligence, we realize that

people think about such a point that

when it is reached, we can say this is

artificial general

intelligence. I think this is the wrong

way of thinking about this technology.

It is

AGI. It is a class of technology. It is

just like a computer.

You have different kinds of computers,

small, large, in watches, in phones, and

that's it. We will never have

AGI. So, we will never have a single

point where we say, "Oh, there it is.

Open AAI has released

AGI." Yes, for example, even when

looking at chat GPT, it is such that at

this moment in chat GPT, it passed the

Turing test. The touring test was

historically considered a test that

indicates that machines are intelligent.

It turned out that chat GPT passed the

touring test and no one even noticed.

There was no big announcement about

it. And it really depends somewhat on

the definition of what you consider

AGI. We are now in a situation where for

example you have the task for the model

to write a poem where every first letter

in each line is

a and now for me it would be very

difficult to write such a poem. I can

say this problem becomes truly

superhuman. It's like it's very hard to

write such highquality poems. At this

point, models are easily able to do

something like this and in some way it's

superhuman. But the thing is that you

have a lot of these types of

competencies and now these models

increase competence in all

fields and in some areas the human level

will be achieved faster than in others.

However, once a human level capability

is achieved in all fields, there will be

areas where that level was superhuman a

long, long time

ago. When will it happen? When will we

reach level

five? It's hard to say. Actually, many

things depend a bit.

Well, I could say from my

perspective

99% that it will be shorter than 10

years. Also, even when looking at the

whole stage of evolution, even of

organisms. These single-sellled

organisms existed for a billion years.

Then multi-selled organisms existed for

a bit shorter. So the time when only

they existed that time of evolution

between multi-selled organisms was

shortening. Even when you look at the

history of human development we are in a

similar

situation. Homo sapiens has existed for

200,000 years. The first cities appeared

20,000 years ago. Industrialization

appeared 300 years ago. Computers

appeared 60 years ago. The internet

appeared 30 years ago. So each of these

stages shortens. And it seems to me that

something similar will happen here

too.

Sure, one needs to consider that among

the elements that might still influence

how long it takes are, for example,

regulations and how it integrates with

society. I can say that this will be a

nontrivial integration of this

technology with society.

We'll talk about this later, but I would

like to present you with a certain

idea. Let's imagine we are in Africa on

the

savannah and there's a wild cat walking

around. We take it from this savannah,

put it in the laboratory, and we create

an artificial breed of cat by combining

it with a domestic

cat. So, what are we doing? We are

copying a fragment of the theory of

evolution of the principles of natural

selection. We have created a breed that

has no genetic defects. We have adhered

to all principles of precision and

biology including molecular biology and

we want to release this cat onto the

savannah. We are releasing it and for

the most part these synthetic breeds of

cats would not be able to survive there.

They wouldn't be because we don't know

what environmental factors influenced

the emergence of this particular cat in

this particular environment during the

evolutionary process. Isn't it true that

today we create models by copying only

certain fragments of factors that

influence what we call consciousness or

what we call humanity? Although I don't

like that word because no one knows what

it is. Consciousness too. No one knows

but a little more. Don't you have the

feeling that it's a bit like creating

artificial animal breeds? At this

moment, we have models that can

replicate some human values within

consciousness. It's hard to say to what

extent these models are conscious or

not. And it's truly a separate

philosophical question or at some point

it will be a technical question.

No one even knows how to approach this

technically. We'll talk about

consciousness soon, but I mean something

completely different. I just mean that

it's flawed. That what we create is

flawed. A perfect example is also that

the human brain needs 20 watts. While a

language model only needs

10. That no matter how much we try, we

can't match this biology. that it's so

very

flawed. It's a bit like when, for

example, you compare birds with an

airplane. A bird is very light. It also

flaps its wings and it can be

acrobatic. It's even incredible how a

bird can fly through a

tree. An airplane is heavy. It can weigh

many tons and has some components that

are shared with the bird, but at the

same time, it is different. It's so

different that you can fly 400 people

across the Atlantic with this plane. So,

it seems to me that we'll be in a

similar situation as

humans. There are some differences like

how economical the brain is, how little

energy it can use, how little data it

can process. While on the other hand, we

might be able to make these models more

like the human brain.

The human brain is efficient because our

DNA contains a lot of information about

how to efficiently utilize

reality. You could say that this DNA

contains training data which is the

result of this DNA being trained on

billions of

people. So you mean to say that we can

reach a similar point but by a different

route than evolution? Historically, even

at open AI, people considered building

artificial intelligence as something

more akin to

evolution. And if you look at the amount

of computational power that evolution

utilized, it is incredibly powerful,

much more than what is used now.

Evolution used a great deal of

computational power over billions of

years. And the computer is the entire

earth. Thanks to evolution, millions of

different species emerged and

intelligence arose. As a result of such

powerful optimization, intelligence was

discovered multiple times across

different species. With these neural

networks, there is a training phase. It

doesn't even matter that it requires a

large amount of data or computational

power. As a result of this, you get a

model with the incredible ability to

learn fairly quickly from a single

conversation, even based on its own

mistakes. My nephew was explaining

Polish grammar to an even older model,

and the model was able to grasp it

within a single conversation. There is

one stage where a very large amount of

data is needed and then we move on to

the point where the model can catch on

very quickly and learn many things

within a single

conversation.

Ultimately, we would like to have a

model that if given a new problem, it

will solve

it. For example, the problem might be

global

warming. There is no solution to it, but

the model will start to think and it

will be able to use a small amount of

data to solve the problem.

How can we talk about AI consciousness

when we don't know how consciousness

emerges from matter? So, we can't create

a mathematical formula that describes

the process of the birth of

consciousness and an algorithm that

describes it.

Maybe I'll start with a definition so we

can talk about the same

thing in terms of how we perceive

reality. How light enters our

eyes. Whether touch is transmitted

through our

nerves. All the information goes to the

brain. Current as bits.

Now the fact that this brain is in our

head really from the brain's perspective

it wouldn't be distinguishable if it

were sitting in a jar in the basement

and the same bits were coming in through

cable. The interesting thing is that

this brain is never able to touch

reality. It only sees the bits that

enter the brain and the brain must

create a simulation of

reality.

immersive image of the world. Exactly.

And consciousness is our experience of

this simulation. And a philosophical

question is why a person has such an

experience of this simulation.

One can imagine a philosophical

experiment called a philosophical zombie

where there would be a person who

behaves just like any other human except

they don't have that internal cinema

internal

simulation. You ask me how one could

check if artificial intelligence had

something like

[Music]

that. I have two main ideas that I can

share with you and I can say what might

be an important element for this

awareness to

arise. We trained models in a 3D world

some time ago. Models that collect

apples or some points in a computer

game. And in the case of models, we can

even understand or visualize what the

model

sees. Now it turns out that initially

when you start training this model it

can distinguish very simple

things. He can tell the difference

between the sky and the ground. After

some time he starts being able to tell

where the apples are that he's running

to. And the interesting thing is that we

can even ask him what he thinks he will

see if he turns his head.

He first starts imagining what he sees.

It's quite interesting that at the very

beginning, even if you ask him what he

sees when he turns around

360°, he thinks he'll see something

different. But at some point, he begins

to realize that when he turns around

360°, he'll see the same thing. This

means he starts to understand 3D reality

better. One interesting thing is that at

some point in such a simulation, This

model must begin to simulate its own

existence because it itself participates

in changing

reality. Just like at the very

beginning, this model understands that

there is a sky earth. It understands

where thy apples are because they are

crucial for survival. At a certain

point, something clicks and he starts to

understand the physical

reality and then again something clicks.

clicking. What is that? Is it a data

range? In the case of neural networks,

their training leads to a representation

where they can solve various types of

tasks. This representation initially

considers simple elements that become

more complex. And at some point, it even

begins to consider the existence of the

agent itself within the simulation.

Initially this agent was not even in

this simulation. His understanding of

reality was so small that he was not

even aware of it. And at some point he

appeared in his own

simulation and I could say that this

might be a moment of

self-awareness although it might differ

slightly from

it. And what if we assume that

consciousness is a quantum effect? If it

were understood what consciousness is,

then it would be possible to build

something like that. If we are not able

to understand consciousness, then it is

hard to build

it. Penrose strongly suggested that it

is due to a quantum effect. It is that

two experiments come to my mind that

could indicate whether models have

consciousness.

an experiment. One experiment would be

something like first we take all the

models training data and eliminate any

mention of consciousness. We don't talk

about it at

all. We are training the model on such

data. So now the question is whether

this model will suddenly be able you

know when you have a conversation with

it on this topic to say yes well I

noticed that something is in such a

style.

No, no. I was wondering about this

topic. I didn't know how to talk about

it. It's kind of strange to feel this

way. Well, if it were the case that

there was nothing like this in the

training data and suddenly the model

started mentioning that they had this

type of experience, that would be a hint

that they might actually have

consciousness. That's one possibility.

Another possibility that comes to my

mind is something like if AI were

connected to the brain and a person

would have the experience that their

consciousness expands as a result of it.

However, the second experiment has some

drawbacks. For instance, in the case of

psychedelics, a psychedelic could be

administered to a person. The

psychedelic itself is not conscious and

the person says, "Oh, my consciousness

has expanded.

So it's not necessarily true that if a

neural network didn't have

consciousness, it could be that the

neural network doesn't have

consciousness while connecting it to the

brain gives the experience of increasing

that consciousness. Why does AI

hallucinate? In my opinion, the standard

way of training these models looks like

this.

First, we train it on all the data from

the internet so that in every article it

predicts the next word, the next word,

the next word. And it turns out that

this is a way to instill a large amount

of knowledge into the model. There are

even fundamental reasons why this is a

way to instill a large amount of

knowledge.

Now the second stage is the so-called

posttraining stage where a person looks

at various model responses and says I

like this answer more than that one. And

in the case of post training the model

is trained to provide more answers that

people like.

In the case of hallucinations, the

problem is that the model may speak

confidently about things it doesn't

know. So where does this come from? If

we look at how this model is trained

with humans, the human will reward the

model for explaining in a way that the

human

likes. Rarely will he be given a reward

for saying, "I don't know."

It may be that he gave several answers

and in one of those answers he guessed

and the person chose, "Oh, I like this

answer. This answer is actually

correct." And the model guessed. And now

the model is being trained to always

provide an answer even when it doesn't

know. If he doesn't know, then let him

guess. Because when a person gave

preferences, they preferred an answer

when the model guessed rather than when

the model said it didn't

know. So could the cause of

hallucinations be this boundary between

knowledge and ignorance? When a person

trains a model, it might be the case

that the person doesn't know what the

model knows or doesn't know. He says, "I

always like it when you give me an

answer." And we train the model so that

it always provides an answer regardless

of whether it knows or not. So during

the current training as this feedback is

given by the person at the last stage,

this feedback leads to the model

responding regardless of whether it has

this knowledge or

not. So how are you going to deal with

this? Let's see what works. Among

important things in my opinion is either

leading to the situation where models

can be trained in such a way that they

can provide a probability meaning they

can express their certainty when they

give a particular statement. It's

possible to modify the training method

or the way the data looks so that the

model can say it's 80% sure that this is

the answer 10% that this is the answer

and 5% that this is the answer.

If we can train models so that they are

able to understand what they know versus

what they don't know, then we will be

able to make them express

it. An interesting thing is that when

you train a model on the entire internet

and you ask it questions A B C D, you

ask it about some topics where it's not

entirely clear. You present the problem

as ABC D. For example, what is the

number of films that are considered

popular by someone? Someone thinks there

are five. Someone thinks there are six.

Someone thinks there are three. As part

of the ABCD, he is able to say it very

well and assign a probability to

it. However, in the case when

postraining is performed, it leads to

the model becoming

overconfident, self assured and at the

same time self assured even in topics it

knows little about.

This is not a philosophical question but

rather a technical

one. Can we imagine creating an

immersive representation of pain within

a language

model? There are two ways that come to

mind right away. The first way at this

moment you can tell the model listen

imagine you are a patient with terminal

cancer and you are in a great amount of

pain. The question is whether this leads

to the model actually feeling pain or is

it just pretending that's one of the

options. I would ask whether inside the

neural network at a certain temperature

is something different happening than

usual. Does asking this type of question

cause it to behave differently than

usual in such a

situation. He will want to avoid it. He

will not want to be in such a situation.

Well, it's not a form of feeling. It's

more of I understand. some kind of

constant

imagination. And if we now combine this

model with some sort of infrastructure

with

robotics where sensors will allow it to

perceive more

deeply, will it be able to feel the

pain? When I think about consciousness,

knowing that this brain receives bits of

information through the spinal cord, our

brain never directly accesses reality.

From the brain's perspective, it could

be in a basement in a jar receiving the

same stimuli and it wouldn't be able to

tell the

difference. If it were able to send

stimuli and get a response from it from

the perspective of this neural network,

it doesn't matter whether it is an agent

that exists in reality or one that

exists in virtual reality.

[Music]

You can have a computer game or you can

browse the

internet. In my opinion, there is no

major

difference. What I think might be the

difference is whether this model is

trained on data from many people,

meaning it reads about many people, or

if it is a model trained on its own

experience.

[Music]

At the moment these models are mainly

trained on a huge amount of data and

then as their own experience they become

such there was once a movie called

Momento about a guy who would instantly

lose his memory. He had a very short

memory so he made tattoos on his body to

remind himself of various things. And

now these neural networks, they are a

bit like that momento guy, meaning they

can remember what happens during a

single conversation and then poof, it

disappears. And when we reach the point

where this model is able to live for a

much longer time or keep learning, it

can also be realized in different ways.

You could have a situation where this

model simply has a context of length 10

million, 100 million, or a billion. And

this context is long enough that you can

have a lifetime of experience there. And

in this context, one learns or it may

require new algorithms that have the

property that at the moment from this

new data from a new interaction, an

update is made for WAG to this network.

So it learns from its own experience and

not only not mainly at this moment. You

know, you might think

99%. Well, most of the learning comes

from the experience of people on Earth.

From the experience that the model reads

on the internet, then a tiny amount of

experience comes from how this model

behaved within what we call trainers, AI

trainers who say, "I like this answer

more than that one." And now in the

conversation with you, it happens that

it learns some of this context. And

after a while, bang, all memories

disappear. Won't energy be the

limitation of this development?

The human brain uses significantly less

energy than AI to perform tasks. Yet, we

still have a power grid from the 19th

century. It might be that at some point

we will reach a situation where energy

will also be in short supply and surely

people will increase the amount of

energy. people will likely make the

networks more

efficient. It's similar to looking at

the evolution of any product or even

computers

[Music]

themselves. The first big computer, one

of the computers called

ENAC was about 3 m in size and performed

300 operations per second.

When you look at our mobile phone, it's

just rows and rows, rows and rows of

size, faster, and it's

smaller. So far, what we see is that

there are many sources for making these

models better. And it turns out that

each of these sources multiplies

together. We once talked about how you

look at the world of artificial

intelligence development in three

phases. The first is the product phase

today where different companies create

various products and we increasingly

integrate them. The second is the phase

where countries understand that

investing in AI is an investment in

their geopolitical position and their

security. And the third most

controversial phase where AI will be the

guarantor of the survival of the human

species. The third phase is super

intelligence. So there are three phases

at this moment. People are creating

beautiful products, incredible ones, and

there will be more of them. In reality,

most software will have some AI in it.

We integrate tools with

plugins. Yes. Software without AI will

cease to

exist. We are already slowly entering a

stage where countries are getting

involved in AI. They are starting to

understand that AI is very important.

Probably in the year 2025 or maybe 26,

AI will be the main topic of

conversation on

[Music]

Earth. So this is how it goes. It grows,

it grows, it grows. This is roughly the

stage where you will have a lot of

agents doing various things around the

world. And there will be a situation

where suddenly it will actually start to

impact the job market. At this moment we

are still in a situation where chat GPT

despite being an incredible product does

not appear as something that impacts the

economy. When you look at the scale of

the world

economy, we would know the impact on the

economy if you turned off chat

GPT. Then we would find out what the

impact on the economy is, how the stock

market

reacted. Poland doesn't really

understand this. Then just as a

digression, a governmental AI fund is

being created in Poland with a budget of

$10 million. That's very little.

Poland also has a lot of very smart

programmers and it can be a huge

opportunity to create incredible

technology to be a country that is more

recognized in the world to be able to

build something beautiful and

incredible. Returning to our topic, what

is this third phase?

This is the phase in which machines that

are definitely smarter than humans will

be built. Super intelligence. And this

will already be such a stage where

countries can think more about

cooperation. In the first stage, it

might be that everyone wants the best

for themselves. But in the final stage,

it may be well understood how important

our cooperation is. But we will also see

how it will look.

It's hard for me to really say until the

end. Imagine talking to a

computer. You're talking to a computer

and everything you hear just makes such

deep sense. You're talking to someone

who is simply smarter in every single

field. It can create a new chip for you,

deeply understand scientific literature,

come up with truly new things, and run

an entire virtual company.

Will AI upgrade humans? Then upgrading a

human being to some extent is even

happening. Now it depends on the

interpretation. At this moment we have

inventions at our disposal that in the

past could have been considered

magic. Currently humans can move as fast

as no animal is able to move.

They can drive cars or

motorcycles. You currently have a

device. It's quite unreliable and we are

getting used to

it. AI wanted to show you a phone. It's

this tiny device about the size of a

pack of cigarettes. And with it, you

have access to all of humanity's

knowledge. You can communicate with

anyone anywhere in the world because the

signal travels at the speed of light.

It's truly unbelievable that something

like this has been built. And I can say

in an obvious way there will be more

technologies that will make things that

once seemed like magic a reality. Don't

you feel that we will be the limitation?

The ones who can say no. Who can say we

don't want this?

If someone tells you they know how

things will develop, that's not true. No

one knows how it will unfold. And in my

opinion, it will be

powerful. People will react in different

ways. Some will say, "We don't want

this." Others will love it. Ultimately,

a lot depends on what technology is

developed, how it will help, and what

actual problems we will face.

Don't you feel that AI in this sense

will create even stronger social

disparities? It could go either way.

Historically, I thought it would lead to

greater disparities, but what we've seen

so far is moving in the opposite

direction. It turns out that now, thanks

to Chat GPT, people who wouldn't be able

to code, for example, can use it to

build a website or solve their problems.

I've heard people say that with Chat

GPT, they suddenly have a co-founder for

their company. They have access to a

lawyer, a programmer, a scientist who

can help them with things they wouldn't

have been able to address before. So, I

have to say it's not obvious to me which

direction this will

go. There's also the perspective that

when looking at developing countries,

There's a metric that compares the value

of money to the value of physical

labor. As countries develop, it turns

out that the value of money becomes

greater than the value of physical

effort. So if you have money, you can

invest it.

One perspective that this might lead to

is that when considering limits, the

only thing that will truly matter is

having money to

invest. However, there's something I'd

like to briefly touch

on. One valuable perspective to consider

is thinking about a trend and imagining

what reality would look like if it were

pushed to its maximum, to the very end.

that could give us an idea of how

reality might unfold.

Alternative. Alternatively,

historically, there have been times when

a trend just comes to an end. It could

be due to

regulations or because the technology is

deployed differently leading to a

reality that looks different.

It's also possible that we could be in a

situation where there are

disparities. But at the same time, even

the poorest people are living much

better than the current average or even

the wealthiest people

today. Have you ever thought about the

fact that in November 2022, you kind of

opened a bit of a monster?

I've thought about how what we opened

sparked a huge amount of

discussion and a massive amount of

conversations with people from all

corners of the world with different

perspectives to me. That seems very

valuable.

For example, a few years ago,

discussions about artificial

intelligence were very abstract, often

confined to sci-fi books, but now a lot

of people are really engaged in

it. It has engaged people from various

perspectives.

You not only have developers and

engineers but also philosophers, lawyers

and doctors looking at this and

wondering if reality is going in this

direction what does it mean for

us and you know it's also

interesting there was a movement in

England called the lite

movement it happened when textile

machines were being built and people

were afraid they'd lose their

jobs. So they were strongly opposed to

it. The movement had the characteristic

of destroying these

machines. In the end, the movement

collapsed. It turned out that if you

look at technological development, you

really have two choices.

[Music]

One choice is to try to build a

civilization that doesn't have

technological development, no

progress. Historically, many

civilizations that developed

technologically eventually

collapsed. That's one option. The other

option is that we've seen time and again

that technology has been able to address

some of the problems we

face. I don't want to claim that

technology is fundamentally good. It

largely depends on how it's deployed,

how it's made available, and how we use

it. However, AI is a technology that has

the potential to provide solutions to

many of our

problems. Fundamentally, if you look at

a scientist's work, their job is to find

a problem they want to solve. And AI has

the ability that once built, it can

solve all kinds of problems.

Yes. But even Altman himself, who calls

himself a visionary, once mentioned in

one of his tweets, and I read it, that

you love him. I understand that by

showing your

relationship, he created

Worldcoin, which by the way, I'm

currently working on a documentary

about. He talks about it as a possible

solution to the genie he released from

the bottle by launching chat GPT, which

also has an impact on the darker side of

the internet, including bots and the

difficulty of distinguishing them from

real people, real users

online. Artificial intelligence will

solve many of the problems we have

today. But it will also create new

problems that we haven't had before or

that were small until

now. That's what I would

expect. The World Coin Project itself is

interesting. It's based on certain

assumptions. Let me put it this way. If

we look back at the past in medieval

times at the level of human

prosperity, it was a long period where

people lived very similarly across

generations. This meant they were

responsible for growing their own food,

making their own clothes, just like

their parents, grandparents,

great-grandparents, and so on. for many

generations. Back then, the main way to

think about how a person could get rich

was

war. Generally, if you went to war, you

could really get rich. While very little

could be gained through business. In

fact, in those times, it was believed

that if you did business, you were a bad

person. It meant exploitation.

Things changed when we built the steam

engine, a machine that could lift what

would require the strength of hundreds

people. You put coal inside and suddenly

you have a machine with such immense

power. The times changed dramatically.

So much so that even Adam Smith wrote

that by creating business you were

helping.

You help your employees, you help

everyone you sell your products to. And

this was

revolutionary. People were amazed by it.

Suddenly we entered a period where you

could think of things

differently. Before we were in a zero

someum game, meaning that for me to have

something, you couldn't have it. I had

to take it from

you. After industrialization, we entered

a time where maybe you don't want to

kill your neighbor. You don't want to

kill him because he might be selling

something you

[Music]

need. And this may have even led to the

fact that the whole world is now very

economically interconnected.

It may have also contributed to the fact

that we have far fewer wars

today. Only about 1% of people worldwide

are involved in military

conflict whereas in the past around 30%

of people were involved in such

conflicts.

So things have changed

dramatically even though it's hard for

us to feel that since we didn't live in

medieval

times. Now we may be entering the next

phase of productivity change. moving

from a zero sum game to a positive sum

game. And perhaps we will transition to

a point where the amount of prosperity

that can be generated is simply

massive. This also suddenly changes the

dynamics of what can be done, what is

right and what is

wrong. He started a project called World

Coin. The motivation behind this project

is to explore whether it's possible to

build a system. Though it's far from

where we are now, assuming that machines

can produce a tremendous amount of

wealth, the question is whether we can

even create a system to distribute that

wealth to

people. And that's also

problematic. How do we do it so that no

one pretends to be a human? so that one

person doesn't take the value meant for

many. So a project was conceived where

first and foremost you must able to

uniquely identify a person. The goal is

to identify a person in a cryptographic

way, one that can't be easily

broken. The project is based on

identifying a person using an eye scan.

These eye scans are stored in a way that

you can't even transfer them to someone

else. It's not that they take a photo

and say this is voyek eye, this is

maki. Instead, the eye scan is processed

by a neural network and what's stored in

the computer is a representation of the

eye. The algorithm behind Worldcoin

functions in such a way that it's

opensource and they want it to be

transparent to

people. They aim to make the process

decentralized because in a centralized

process there's always a risk of

attacks. Wy tech what are you afraid of

in the development of AI today? Is there

something that really scares you? What

has I don't know maybe brought you down,

discouraged you or been the most

difficult for you

recently? I would perhaps classify the

problems related to AI into different

categories. The first major issue is

that AI can be used in a negative way.

For example, deep fakes, fake news.

There are many things like you

mentioned AI has a high chance almost

certainty of being used for military

purposes. And there's also I can say

things that people are still afraid

of. Imagine today there are about 30,000

people in the world who would be able to

create a pandemic.

These are usually scientists, some of

whom are directors of

institutes. With just 10 such people,

they could create a pandemic that could

wipe

out that could wipe out part of

humanity. Yes. And it could have the

property for example that the virus is

undetectable for 2 months then spreads

very quickly and then kills

rapidly. One of the risks with AI is

that while we give access to

intelligence it could suddenly increase

the number of people who are capable of

building a

[Music]

pandemic. This is one of the risks when

looking at something like misuse.

And it's one of the categories. I'll

talk about other categories in a moment.

In the case of biological misuse, for

example, AI could also help with

attacks, hacking, or even with nuclear

and chemical

weapons. So, one of the problems, and

I'm talking about the upcoming year or

two, is that we have increasingly

powerful models available.

One of the important things is ensuring

these models are not used within the

categories I've described. I can say

there are specific steps that can be

taken to protect these models and I'm

quite optimistic that this can be done.

However, it's crucial to minimize their

negative

applications. Are you working at open AI

on any area to prevent your models from

being used for malicious purposes?

Yes, I can even share the framework we

use for this. There are actually several

efforts in this area. First, this pur

there's something called

preparedness. In preparedness, we aim to

understand the risks in various

categories such as

biological, chemical, nuclear, cyber

security, and persuasion.

We need to assess the current level of

the model in each of these

categories. For example, in the case of

persuasion, we could rate it as

critical, meaning the model is at a

stage where it could easily convince a

large number of people towards various

harmful

goals. We'd consider that critical

because it would be really problematic.

On the other hand, we could say it's at

a medium level if it's similar to what

another person might attempt to do, like

convincing people. Currently, the risk

of persuasion is something that happens

already. People create fake news and

others engage in persuasion

tactics. So, the first step is to

understand where we

stand. Then we also have a team

dedicated to mitigation.

What needs to be done to make the model

less helpful in areas like

biology? We aim to reduce the level of

risk in high-risk categories to medium

or low

levels. There are four

categories. The first category is

misuse. The second category is the

result of the AI race. This refers to

the danger that arises as organizations

compete to develop

AI. The third category involves

accidents which are simply the result of

negligence. The fourth category is when

AI itself becomes a danger when the AI

has goals or objectives that are

harmful. Looking at these categories,

they become more relevant at different

stages.

Right now, for example, over the next

year or maybe two, one of the biggest

categories requiring the most attention

would be

misuse. How these models can be used for

harmful

purposes, while other categories will be

more significant at later

stages. You mentioned that AI itself

could have malicious goals.

You can almost think of it as

misuse where misuse is the AI

itself. The concept is this. If we

create models with very broad

capabilities that can solve many tasks,

how do we ensure that these models will

still listen to us and behave in the way

we expect them to?

Do you remember the day in November 2022

when you released chat GPT to the world?

Yes, I

remember it even happened on my

birthday. I remember we were just in the

office when it was released and people

were reacting to

it. I also remember that internally

people didn't expect the reaction to be

this big.

There were even some questions about

what we should call the

model. Chat GPT wasn't a particularly

thoughtout name. It was more like, hey,

we have GPT models. Let's add chat to

it.

How many hours after the release did you

realize it was starting to grow on such

a scale reaching millions then tens of

millions and eventually hundreds of

millions? I think it was a matter of

days. It was days and then it started to

really appear on Twitter.

What's closer to how I think about it is

that it's incredible how quickly I've

found myself in this

position. And I say this not just from

the perspective of a decade or so, but

Get Our Mobile App

He Created ChatGPT: What’s Next for Artificial Intelligence | Wojciech Zaremba (Open AI)

Transcription