Top Data Scientist Reveals AI Challenges

Every time AI enters the real world, it

meets resistance. Competition, shifting

markets, and human behavior. I'm Michael

Cricggsman, and this is CXO Talk,

episode 890.

We're discussing AI misadventures and

the adversarial economy with prominent

business leader Steven C. Daffron and

Anthony Scriffin, one of the world's top

data scientists. Let's get into it.

Gentlemen, welcome to CXO Talk. I am

delighted to see you both.

>> Thank you, Michael.

>> Thank you.

>> Steve, tell us about your work.

>> We are a private equity fund that

focuses on financial technology. AI, of

course, is a major part of that. We have

a model as being investors. Of course,

we're private equity, but also operators

since we are private equity fund made up

of people who have actually built and

run companies. And we're innovators. We

actually think hard and spend money on

trying to be at the innovative edge of

where financial technology is going. So

AI is literally in our bloodstream and

we work the order pretty hard.

>> Anthony, tell us about your work.

>> Right now I am a distinguished fellow

with the Alfred Lee Lumis Innovation

Council at the Stimson Center, which is

a Washington DC think tank which is

focused on what I would call action

research. So not just white papers but

actually doing stuff uh that that

matters quite a bit uh all in the kind

of for good kind of category and then I

also am involved in some space related

ventures

>> when we talk about the adversarial

economy what do we mean and how is it

being shaped by AI and and maybe you can

share your views on this

>> our economy is adversarial because we

choose to make it adversarial it's

rather than fostering cooperation

and mutual benefit. We teach each other

and I've been in this business. I've

been working for 50 years. I see how

these things develop. We teach each

other to emphasize to exploit

vulnerabilities and weaknesses

and systems or individuals. So we get an

advantage. Not it's not new. Everybody

here remembers Wall Street. Remember

Gordon Gecko? Greed is good. So

competition is good. Right. Right.

within limits.

Philosophically

um there are limits to how competition

how that adversarial economy works.

Compos competition to an adversarial

extreme is not good. Aristotle said

competition could actually lead people

to do something positive lead towards

honor and excellence and motivate people

to strive for the common good. But he

also said in excess it leads people to

have self-love and disregard for

faileness for fairness. Well, candidly

AI makes that adversarial

economy easier. When AI can make this

adversarial nature be a a a constant a

pervasive spirit making everything feel

like zero sum, we we can see it. No one

trusts anyone. AI reinforces that.

It makes it the the adversarial extreme

easier to accept because it puts it at

arms length. You can see it. Look at all

the places where digital markets where

AI is being used to bias data structures

where it's being used to manipulate the

algorithms. And

you want to list just Google just Google

adversarial attacks on financial

reporting. Just Google that and watch

the list scroll up. We've made it so the

adversarial economy AI makes adal

economy easier because it puts it at

arms length makes it doesn't feel as

personal because the AI algorithm is

driving it

even more in dimmic and the broader

public will see this when you have chat

bots that are being deliberately

designed algorithmically to capture user

attention and then use that attention to

extract from them their vulnerabilities

so we can manipulate responses.

to find and exploit what they want to or

should do. That's adversarial to the

extreme. Now, we choose to do it. We

could choose how to make the adversarial

economy work and not take but we operate

in this adversarial financial economic

ecosystem. There's no doubt about it.

There are misaligned incentives. There

are heightened competition.

We have to choose how to manage that.

And we who are in this space can make

those choices. Anthony, thoughts on

this? You know, one thing that's not

100% clear to me, Steve, as you're

talking.

Is

this fundamentally good or fundamentally

bad? And

>> and maybe I would add, how is it

fundamentally different now that there's

AI in the room?

>> Well, it's fundament again, I'm I'm

still like philosophy as much as I like

data science. And one of the things that

my favorite philosopher Aristotle would

say is in the moderate in at the median

it's good because you need competition

to move the world forward. It's bad if

you allow it to go to either extreme to

have no competition or to have too much

competition. And what AI does is allows

us to take that extreme competition and

move it away where we don't feel it. We

didn't manipulate the chat bots to get

grandma to share her intimate details

with us so we could exploit her bank

accounts. We didn't do that. The

algorithm did it. AI allows us to be

adversarial but not feel like we did it.

That's not good.

>> Anthony, let me ask you this. How does

AI depersonalize

the psychology

of us the participants in the economy so

that and I'm paraphrasing here so that

we can screw each other over and feel

good about it.

>> AI doesn't care. Um it's a bunch of math

and it's a bunch of math that is

designed to achieve a certain goal and

we tend to an anthropomorphize it. we

tend to talk about what it wants and

what it's doing. And part of the problem

here is that we don't have the right

language and if we did talk about it

convoluting as opposed to what it wants,

people would very quickly, you know, be

asleep. So, we have to be careful about

what's happening here. We use the term,

actually, I think you use the term in

the the title of this episode

misadventure, and I like that term

because misadventure can mean many

things. And to an AI guy, they'll say,

"Oh, you're talking about hallucination.

Let me talk to you about hallucination."

By the way, hallucination is like a

fancy word for when AI does something

you didn't expect it to do and don't

think you asked it to do. Um, sometimes

it turns out you did ask it to do it and

didn't realize it. So certainly Steve is

talking about algorithmic bias. Um, a

lot of times these algorithms, which are

the the mathematical equations and

processes that are sitting underneath

the AI, are designed, they're trained on

a certain type of data, and then you let

them out into the wild, and the real

world doesn't look like the data they

were trained on, and they go ahead and

start doing things based on the training

that they had, which is inappropriate.

Sometimes they consume their own output

as input and you get these these um

recursion loops where the data is this

is sometimes called uh overfitting when

you have the wrong training and then

when you start to to recurse when you

start to consume your own output you

become increasingly confident that what

you're saying is correct because you've

heard it before. But what you don't

realize is you've heard it because you

said it.

>> But we're going let's let's go back to

this. That's true and it's meant not

meant to happen. But I will tell you

that in many cases the adversarial

nature of this is if we can design

something that allows us to extract more

information even if it's not information

that we should be extracting. We tend to

do that because it's adversarial because

if we can do it if we don't do it the

competition will.

>> So so let's uh talk about two types of

adversary here. One is the adversarial

like uh antithetical. it's doing

something we don't want it to do or we

don't there's we don't intend for it to

do we don't want it to do and we want to

get there first we want to do it better

than it does it right those are actually

different flavors of ice cream here um I

can if I understand what your algorithms

are doing

>> or if I even have any idea of what type

of AI you're using I don't have to

invade your systems I don't have to

invade your your turf all I have to do

is poison the milk that you're using to

make those decisions and that's another

type of of adver adversary here. There's

misinformation, disinformation, there's

um you can hide the data that would

allow the AI to reach the conclusion

that it needs to reach. Just basically

prevent certain types of data from

getting into the algorithm. There are so

many ways to manipulate this, which begs

the question of how would you even know

if it were happening without using

another type of AI. So, this is

definitely a big big fine kettle of fish

you've opened up, Michael. Well, we have

a very interesting question that has

come in on Twitter from Arcelon Khan

who's a regular listener and he always

asks uh provocative questions and he

says this. In this adversarial world,

who is right or wrong depends on who are

the gatekeepers and who sets the

guidelines for the guard rails. How do

you understand what is rogue AI and

who is not, what's not?

>> I would even double down that it's not

as binary as that. Very often there are

different uh regulating authorities,

different uh whatever you want to call

them, authorities that want different

things. And so you can't actually be

quote unquote right with respect to all

of them at the same time. One of them

Steve's talking a lot about like

harvesting information. So that's a

goal, right? Another goal might be

privacy. I might want to harvest your

information so that I can customize the

app and make it a better experience for

you, but I also want privacy. Well,

those are opposite things to want. And

you can't be right with both of those at

the same time. There isn't a person who

says this is where the dividing line is.

There isn't someone who says these are

the right guardrails or these are the

wrong guard rails. Part of this becomes

a matter of choice that each each

company, each developer, each CEO makes.

That's let's balance the easy one. Not

so easy but but straightforward in terms

of understanding it. Privacy of the

individual versus knowledge of the

individual that we can use to develop

and extract rent from that individual.

If we allow AI developers to extract the

max amount of information and use it to

design exactly what we want to be able

to touch the right button and bring the

right response from that consumer that's

good for the company and probably good

for the company's bottom line not

necessarily good for that consumer and

not necessarily good when I use that

data to predict that you're about to

commit a crime that you haven't

committed yet

here we go right so there there's lots

of ways that you can take this too far

there are certainly ethical princip

principles of AI, OECD. I would

recommend definitely looking at them.

There's lots of other sources where lots

of smart people have gotten together and

thought about this and and said, you

know, what do we as a collective uh

group of experts believe are we can all

agree, well, it turns out that's not

actually accurate because depending on

where you go in the world, there are

countries that value national security

over personal privacy. There are

countries that are completely

capitalistic. There are other places

where they want to have the right to be

forgotten. Those are all completely

opposite things to want.

>> I just want to tell everybody that you

should ask your questions on Twitter X.

Use the hashtag CXO talk. If you're

watching on LinkedIn, just pop your

questions into the LinkedIn chat. When

else will you have the chance to ask

these two brilliant people pretty much

whatever you want? So, take advantage of

it. ask your questions and oh

go to the CXO talk website and subscribe

to our newsletter because we have great

shows coming up and you should be a part

of it. All right. Kent Sparks who is

provost and vice president for academic

affairs at Eastern University says this.

He says, "From a higher education

provider, he applauds your thoughtful

concerns about the ethics of AI, which

adds steroids to the adversarial dark

side of our competitive system." And

here's the question, hence a good one.

Can these issues really be tackled

effectively apart from political

solutions?

>> Apart from is nearly impossible because

there's politics in everything. But I

would say whatever you're doing, you

should do on purpose. So I spend a lot

of time with academia. I spent a lot of

time attempting to be a good counselor

there. Um some very big questions right

now of you know what do we even teach

that will be relevant by the time these

students graduate? What how do we

understand provenence and permissible

use in the context of peer-reviewed

research when the peer that's doing the

review might be an AI agent? Now there's

some really big questions that we don't

have an answer to yet. But there's also

a huge opportunity cost. The cost of

doing nothing is not nothing. You will

slide backwards farther and farther. So

we have to be good stewards of this

amazing technology. Do it on purpose. We

will not be perfect and there will be

politics. My solution to this is not a

solution, but it's a recognition that we

have to each develop our own

philosophical perspective. Philosophy

that worldview is what should be

approaching this. We individually as

well as collectively choose to make the

adversarial economy as adversarial as it

is. We can also choose to do the right

thing when there's data that can be made

available to the for the right reasons

to feed starving children to ensure that

people don't get sick to give people the

right health care when they need that

health care. When that exists and we

choose to do that, that's a good thing.

choosing at the same time to take

artificial intelligence in a way that

exploits people that hurts people. It's

it's not always that complicated.

Sometimes it's philosophically back to

Aristotle, find the middle. Too much

competition that allows any company to

go into any realm to do anything as long

as it improves the bottom line is too

much if it hurts people. Too little and

we have nothing to drive the economy.

finding that that middle ground and

being willing to say this middle ground

is the right mix of good to move us

forward and sticking to it. And how do

you teach that? Well, frankly, I think

you teach with some philosophy, not just

math. I like the math part of it, but

it's also the philosophical part of

understanding that what you do has

consequences.

>> One of the most important questions I

think you can ask whenever you use AI is

what do we have to believe in order to

do what we're going to do? and to do it

deliberately and do it on purpose.

>> Are we having a technology discussion?

Are we having a discussion of one's

viewpoint on selfinterest?

I I mean what does any of this have to

do with AI?

>> AI is embedded in everything we do right

now. we that the words that we are

speaking are being transcribed by

something and they're being synthesized

and they're being inferred upon while we

are speaking them. So to ignore what's

going on with the technology behind the

scenes is is foolhardy. However, if you

only lead with that AI, if you run

around with your AI hammer and say,

"What can I hit with this hammer?" That

is equally foolhardy. So you have to do

both. You have to do them at the same

time. And you can't ignore either side

of this.

>> Let's jump to another question again

from LinkedIn and this is from Andrew

Lamar

and he is head of fraud and a B2B fraud

expert and he says this. How can AI

systems be designed to remain resilient

and trustworthy when traditional metrics

often do not signal emerging threats?

It's a really interesting question.

>> Definitely the way we measure things

when you talk about fraud or I'll I'll

expand it. I use the term malfeasants

because a lot of times the bad behavior

is in anticipation of the fraud, but

it's not really technically fraud yet.

So, uh, you know, I lie to you and then

you go tell somebody and they give me

better terms. It wasn't wrong for me to

lie to you, but it was wrong when I took

advantage of it. That kind of thing. Um,

the the the way we've measured these

things in the past is based on canonical

understandings of things that people do

wrong. identity theft or or

misrepresentation of facts, etc. But now

with AI, you have a whole new type of

novel fraud or novel malfeasants that we

don't have names for yet, and we

certainly don't have metrics for it yet.

So there's a the good news is there's a

lot of AI out there that can detect

emerging patterns of behavior, not

necessarily adjudicate whether they're

bad or not, but adjudicate that I've

seen this behavior before and it's

starting to become more prevalent. And

now smart people like the person who

asked this question can go look at that

behavior and we can separate the the the

noise from that signal and point them at

that. This isn't looking for needles and

h stacks. This is looking for needles

and stacks of needles. All the data is

valuable. The way we measured it

yesterday is nowhere's near good enough

to measure it in this kind of context

which is highly multimodal and massive

amounts of data. But I think you can do

some practical things Andrew and this

one of the things that I would that I

when I think about this and talk to

people about it. You can't predict

everything. Resiliency though is the new

is the new value. Resiliency

understanding and being early

early to them early to the recognition

of things happening. What do you do?

Well, first you should we should be and

I are a if you are being a a

conscientious prudent AI developer, you

have key metrics and you monitor those

metrics in near real time. Your

accuracy, your precision, the recall,

the mean absolute error that you're

getting the or the mean squared error

for regression models.

>> And you do that constantly and you use

that to then when it happens, you react

to it. you watch for prediction errors.

I mean the prediction errors that are

happening, they're not they're not

randomly distributed. These things that

are new kinds of mouths will actually

cause changes in those prediction

errors, finding them and understanding

and analyzing them first and then watch

what happens after the fact. You'll also

have a growth in your residual errors.

Look at the difference between the

actual and predicted values. Watching

those over time allows you to you can't

have a complete prediction of the of the

new malfence but you can have ways of

finding it finding it early analyzing it

and reacting to it and back to the

competition point that's a good thing

that's a good competition the people who

do this best the people who are building

this best and and I won't cite them by

name but I can tell you there's the

people in the marketplace now who are

being really good at developing

resilient metrics that allow them to

know this is happening first and

therefore react to it and save

the errors that come with that from that

mouthpiece.

>> Let me just double down on something

Steve just said since you brought math

into the room. U there's a a concept

called elasticity which is normally used

in economics but I'm going to use it in

decision-m. If you think about decision

elasticity how wrong can you be and

still make the same decision that you're

making? you you will not ever have a

perfect measure of bad behavior because

the best bad guys if they think they're

being watched they will change what

they're doing. So you're now modeling

what they used to be doing rather than

what they're doing now. The good news is

that you can use math to figure out how

much of the observed error is

explainable versus not explainable. And

when the unexplained error or what we

call random cause variation starts to

overwhelm the assignable cause

variation, guess what? something news is

going going on that you don't

understand, go back and figure it out.

There's really good math and there's

really good AI that can be pointed at

problems like this, but you've got to

ask a very different question. That's

exogenous, but you also have endogenous

variables that you can be watching too.

Absolutely. The things that cause AI to

go off off the road more often than Yes,

I I'll put malfeasants at the very top.

>> Bad data.

>> Bad data. Data drift. You started with

one set of data. When AD model's

performance decreases over time because

of the changes in the data, the real

world data that's encountering, did your

governance change? Did data come in that

you didn't recognize? Did your upstream

data stream change? If you have data

coming in from multiple sources, did one

of those upstream vendors change. I I

watch this every day in financial

technology because markets change really

rapidly and the AI that's being

constructed takes that input as being

relatively a constant. But the AI has to

be built to acknowledge the change and

that happens because you'll otherwise

you'll wind up with a data model

mismatch.

>> There there are two measures that are

very easy to implement in any system.

Character and quality of the input data.

Did the character is did the nature of

the data change the metadata that the

sources and the quality of the data. Did

it the measures of central tendency all

of the other statistical measures?

Anybody can monitor those things. In

most cases, we what we see is the data

gets quote unquote onboarded and then

people are on to the next shiny object

and nobody's paying attention to

character quality of data. The other

side of this that's that's also equally

suspect is concept drift where you

actually have you have when you build

the AI you have a relationship between

the inputs and the outputs and that's

what you start with but over time those

inputs and outputs shift and that

concept can make your AI be

malperforming

without you even recognizing

>> that can also happen when your customers

start using your product for an

unintended purpose. Oh yeah.

>> And that happens all the time. That's

actually the most important that

important kind of concept.

>> Yep.

>> Steve, I have a question for you. You're

running private equity fund. How do

these sets of the kinds of issues impact

your thinking about your fund and your

investments and so forth?

>> Our model is we are is an II model.

We're investors,

but we're also operators and we're also

innovators. So this kind of thinking on

the innovation part of that model

infects how we as operators run run the

companies we invest in and I'm

our norm is to is to take a firsthand

view when we invest in a company.

We take a firstand view of how to manage

that company in a way that allows this

kind of growth to happen. That ties into

especially these days into where we're

going with AI. The we have portfolio

companies whose whose performance is

dramatically improved by the

introduction of AI. We have we buy

companies, we invest in companies where

AI hasn't been used and we can then

bring it to bear so that we can create

some of these value that the value

creation that comes with AI is a

function of how you invest and even more

importantly of how you build the

operator resilience of that company.

Being able to layer AI on top of the

existing processes is is is a recipe for

disaster. Running around with AI and

saying, "How can we use AI here? How can

we use AI there?" is the wrong approach.

AI is a tool and it's a tool in your

toolbox along with PowerPoint and along

with all the other things that you do.

And there are places where it makes

sense. So if you said we're going to

improve the the performance of a company

using AI and particularly this approach

of AI, great. Let's measure it before we

implement. Let's come up with the how

the fact and why do we think this

particular tool or this approach is

going to be right and by the way how do

we know we're compliant and all of that

stuff. How do we know that we have the

right data and after we ask a few

difficult questions then go push the AI

button. Don't there's a lot of ready

fire aim out there right now.

>> But that's not fun

doing it your way. That's not fun.

>> I'm sorry but you know this is the real

world. But I but I will also tell you I

think there there are lessons to be

learned here that goes back that goes

back to the question of of both choosing

to do the right thing but also it's it's

also good to do the right thing. The you

know I think the latest lang silicon

sands is 75% of the companies the public

companies who are actually trying to use

AI and don't actually hit the ROIs they

expect to hit. You know why? Because

they they don't think about it the right

way. They try to cheese cheaper or

faster without thinking that what what

the what the entire process is of

measuring the trust of their products,

the resiliency of their processes,

>> the cost of the tech stack, the the the

cost of compliance failures.

>> Absolutely.

>> And those all have to happen before the

fact, which is honestly one of the

reasons I like in the the private equity

space is because we can get a lot closer

to the we can sail closer to the wind to

use that metaphor. We can be closer to

seeing what it takes to make

the right kind of investments in AI to

get the right kind of returns over time.

For example, and I and I hear the noise

all the time about this from the when we

portfolio companies that there's there's

arguments, well, we're not getting the

same kind of gross gross margins we get

in AI companies as we get in in regular

SAS companies. No, you don't. gross

margins on ad companies will be will be

lower and slower because you've got

special work that you're doing to build

the data under the underlying data

structures first only that data

architecture works and then you layer on

top of that the specialization you need

to have the AI brought to bear with the

human in the loop only then can you

start receiving those benefits now if

you look at the payoffs for punish do

well I won't cite them but you can

anyone can look look up Silicon s you'll

see exactly what I'm talking about. The

companies who do this well raise money

at a lot greater greater rate and a lot

faster than the companies who don't do

it well. But it's harder unless you get

your KPIs created before the fact and

you get everyone from the CEO, the CTO,

and especially the CFO. Sorry CFOs if

I'm critical here, but the CFOs are the

ones who want to be they want to to make

these metrics match with the old SAS

businesses. And these are not the old

SAS businesses. The hard part here is

that sadly the reality is if you can't

talk to your your overlords and your

constituents and say we're doing AI, you

know, somehow you're behind the times,

right? If you don't focus on what

Steve's talking about first or at least

at the very least at the same time, if

you don't get the data right, if you

don't get the compliance right, if you

don't get the tech stack right, if you

don't get the KPIs right, you'll be able

to check that box and tell everybody,

"Look at this thing we built." And I

promise you, you'll you'll be licking

your wounds in in very short order for

one of those reasons or all of them.

>> I wanted to do some CXO talk shows with

CFOs on the subject of how they look at

AI investment and balance risk and

innovation. And I asked several CFOs uh

who I know and I can't get anybody to

want to really talk about this. Most of

them aren't very happy with their AI

investments right now unless they're an

AI company and that's their actual

product. Most of them are, you know,

they're not seeing the return that I

don't want to I'm going to you're going

to get crushed with comments uh

disagreeing with this. Of course, there

are examples where there's great

success. Um but that road is paved with

lots of you know uh whoops and the CFOs

are when it's also very hard to measure

in the enterprise because it's not

there's no AI line that they're charging

to this is you know part of this is tech

debt part of this there's a lot of

issues so you need to put in CFOs and

there needs to be a focused effort on

this and this is this is one of the

conversations that we have across the

across the industry how do we frame get

the right financial margin financial

framework for this because gross margins

for for AI companies are different over

time. They're probably 50 to 60% where

they where a a standard SAS company

would be running 80 to 90% just

beginning.

>> Exactly. You got to got to figure in the

cost of the requisite data engineering

and I can tell you the requisite data

engineering is going to cost triple what

you think it will at the beginning that

you have to hit that right first to make

this work. You have to have the

foundations for the large language

models. Those take time. You have to

have acknowledge the higher compute

costs that go with bringing AI to bear.

You need special specialized technical

oversight. This is to be candid is where

I see a lot of gaps because we think a

software engineer is a software

engineer. Sorry, there was a time when I

could have called myself a software

engineer. I cannot do this. That takes

some specialized technical oversight to

make this work effectively. And you need

to be prepared to to pay up to make that

happen first because if you don't, you

have these these ongoing R&D investments

won't actually pay off. Let me just

comment on one thing because Steve is

underestimating his ability to do this.

Anybody can open a Jupyter notebook and

include a bunch of code and do quote

unquote this. That's not the this we

should be doing. So there's a lot of

people out there that are um you know

falsely um very impressed with their

ability to do AI in a very controlled

environment with a very small amount of

data and their production environment

doesn't remotely resemble that.

>> So basically what you're saying is there

are a bunch of suffering CFOs out there.

>> Suffering is a choice if they think hard

about how to do this.

>> Suffering is a choice because and and

let's talk about let's talk about how

you how you think about this. Let's talk

about suffering being a choice.

>> Well, suffering is I bet in Buddhism if

I will tell you suffering is a choice.

It's your choice.

>> I have some deep knowledge.

>> Let's go let's don't go there for the

moment. Let's let's for the moment go

back to this. How do you get the CTO's

and the CFOs in line?

>> I have coming up as a guest on CXO talk

the CTO of Google Cloud. So, I'll have

to ask him about this and I apologize

for interrupting you. I just think that

the idea of having the CF CFOs be pillar

because of something they haven't seen

before is the wrong approach. What we

need to do is to realize that they we

can make a choice to learn how to do

this effectively. And part of that is to

start small. There's a really good book

by Danny Go.

It's called uh the AI Republic. He's

coming up with a new one called AI

Native. But in there he talks about how

to do this and to get the the entire

enterprise to work together from the CFO

and the CXO, the CFO to the CTO to work

together by starting small and learning

how to work this. So you understand why

the gross margins will be different. You

understand what happens when you put it

into the from design into testing into

production. You can all

most most software engineers do this as

a matter of reflexives. And of course

they do this AI is this requires a

different level of approach and frankly

a different level of technical oversight

to make it effective which is why I

always encourage people to start small.

You also if you most most larger

enterprises are using some form of agile

methodology and I'm not this is not an

agile methodology comment but it's a

agile with a lowercase a you know taking

small steps and understanding the impact

of those small steps rather than trying

to eat the whole whale with one bite.

Right? There's a lot to be said for that

right now in this regard. You probably

are not going to be able to measure

unless you have an actual test bed where

you can take a product with and without

AI and actually measure the marginal

return. That's not reality. The reality

is that you're going to see incremental

benefit here over a long period of time.

You're not going to see that big giant

bang unless you have a particular corner

case that you couldn't do it without AI

and now you can and it's easy to

measure. That's often not the case. And

we Wong says in AI you're they're seeing

a red queen effect. How a red queen

effect can rapidly lead to inefficient

decisionmaking

driven by influential figures such as

prioritizing hope hype rather or it

could be hope but prioritizing hype over

empirical validation.

Here's the question. How do you as

leaders identify and mitigate the

negative impact of such influential

individuals or practices within your

long-term innovation trajectory?

>> So, a red queen of problem is where um

you know the Alice is at the tea party

and um she says to the Red Queen, "This

is a a crazy place. I've been running

and running and I don't seem to be

getting anywhere." And the Red Queen

says, "That's the kind of place this is.

You have to run as fast as you can just

to stay where you are. So a red queen

problem is where you can't just do more

of what you're already doing and

necessarily make progress, but you can't

stop doing what you're doing at the same

time. So in effect, this is a red queen

problem because we don't get to just

stop doing whatever we were doing in the

enterprise and go try AI. The world is

continuing to evolve and get disrupted

and the customer expectations are

changing and the board wants what it

wants and all of that. And by the way,

here's AI. So in introducing to the

question now is the the the the the

voice that everybody's listening to.

There's that one voice that represents

that orthogonal thing which is the way

you get out of a Ray Queen problem. And

everybody wants to listen to that voice

because oh they they have an answer.

Let's go follow them, right? They're the

shaman. And you don't ask the question

of what would we have to believe to

follow them because you kind of too busy

in the quicksand. And so it's really

important to I used to play water polo

and when you're playing water polo, you

want to get to the other end of the pool

as fast as possible when there's a fast

break. But you got to pick your head up

otherwise you're either swimming in the

wrong direction or you get hit with the

ball. Either of which is a really bad

day. So it's important to pick your head

up here and it's important to watch how

the environment is changing while you're

solving this problem. While you're AIing

the problem, make sure the problem isn't

changing. And also ask a few questions

of why you should believe that shaman

and what that shaman is selling and what

what what data is is is being used to

form that conclusion. Don't just run

there because it's a solution.

Influencer problems are become serious

problems when you allow an influence to

be an influencer to be determinate. And

one of the things we try to do

as we operate

operation during innovation means that

you learn to step away from the

immediate problem. Pick your head up and

look at all the players and say, "I know

she thinks that and I know she's

powerful. I know she's really smart,

really articulate, but you have to shut

her up for just a moment so that the

other people can ask their questions and

listen to each other and not allow

this is this is this is why hierarchies

don't work so well in this space. U this

is this is this is horizontal rather

than vertical. You're talking about

psychological emotional factors that and

factors of appearance and perception

that have absolutely nothing to do

necessarily with the underlying

intelligence argument or factors of of

what's actually being discussed.

>> And I'd argue that have everything to do

with it. And again, I I'll cite a

different book, Angus Fletcher's book

called Primal Intelligence, where he

talks about the the things we see as

human beings that you wouldn't

necessarily see as a function of the

math that you would see as a hunch of

watching other people interact. the so

simple answer and it's not simple to to

towing's

question is to say

having an environment where you force

the issue of making sure that the red

queen is not the only one talking and

everyone else gets a chance to speak and

listen to each other. There's two

sociological terms that are really

important here and editor. Right? So

you're either in the problem so you're

in the red queen problem trying to solve

it or you're at it. You're outside of it

looking in. Right? A lot of times what

happens is you've brought somebody in

and they have a product or a service or

and they're going to be the that shaman

that leads you out of this and they

can't see the problem the way you see it

and they actually aren't trying to sell

you snake oil. They think they're right,

but they think they're right because

they don't know what you know. You don't

understand what they could do. So you

think they're right and you get this

echo chamber where you you walk the

plank together, right? So, it's really

important to do what Steve is saying,

even more so when you're expanding that

circle and bringing in other people that

bring in fresh ideas and products and

services to make sure that you all

understand what the what you know what

you don't know from the other side of

that ven diagram. That often doesn't

happen because either you don't want to

pay for the the time it would take or

because everybody's too quick to, you

know, book the sale or whatever. back to

the point of the the shareholders if

it's a public company or the investors

if it's a private company want a return

right now. Y

>> and part of this becomes a willingness

to say I know you want a return right

now but the best thing to do is stop and

listen to make sure you're actually this

is you can't treat these as two separate

problems. Yes, the math and and the the

building ability to develop the the

artificial intelligence and to get it to

work and to ensure that the concept

stays consistent to make sure we don't

have data drift. That's all important.

At the same time, you have to be able to

bring the people who are building this

into a room or a virtual room so they

can listen to each other so you don't

get the red queen

making the decisions

alone. Ravi Carara, co-founder of a

global air and water generation

initiative says on LinkedIn, is there a

national strategy on AI and education to

prepare the next wave of AI skilled

workforce? Thoughts on that?

>> Yes and no. So there are guidelines,

there are wishes and hopes, and there

are recent um more than wishes and

hopes, but there isn't actually one ring

to rule them all. We are nowhere's near

there yet.

>> I think the answer is no. In fact, I for

the that if you go into the academic

world, there's no coherent academic

synthesis. If you definitely in the

government world, there's no coherence

there. In the mathematical world,

there's more because we that's where it

tends to come first, but there is no

coherent strategy.

>> The most recent thing in in the US was

the AI action plan, which you know, it's

a plan to a plan, right? Um there are

other parts of the world where there

there are AI frameworks. The EU I would

point you to there's a unbelievably

complex uh framework that's been

published and and a set of regulatory

guidelines as well. Um but even there

the regulation and and policy is never

going to keep pace with innovation. I

think the way the what will lead in the

right direction is this the the the

pearl will start to form around the the

the grit of a particular problem and I

think that problem will probably be the

cyber problem that we'll start seeing AI

being used more and more by the

malfactors to to create problems and

therefore we we forced to develop a

uniform strategy to encapsulate that

>> and that's what you see happening in in

a particular country in the world right

now where there is exactly that pearl

forming around that bridge. Um I'm not

sure I want it to only go there. Um as

an example, uh if you look at uh medical

research and and innovation there, you

know, I want them to go faster, but I

also don't want to give up all my

personal details. back to that back to

that that that's the trade-off because

this is this is this is

>> in order to have the cyber security we

want the level of personal details you

have to be prepared to give up

>> what you see in a lot of parts of the

world that have more

I'll say egregious structure around this

is if you peel it back um you can do a

lot more if you stay within our four

walls but you can't do so much when you

when you leave our four walls most of

these problems are global problems and

so that is really um a very dangerous

type of thinking at times.

>> Clauddio Carino says, "What are your

thoughts on how to protect your data

from misuse without being completely

riskaverse?"

>> The place where we're not paying enough

attention to security is the fact

protecting our data. And I would spend

more time and more effort, I would be

more

obsessive about protecting data than we

are. And I I'm I realize that that's a

hindrance towards rapid growth and that

keeps people say, "Well, I want to I

want to build my I want to build my

models in the in the light of all the

data that's available." When you do

that, you're opening up Pandora's box.

I'm much more along the lines and

perhaps it's because I'm old and slow,

but I would much prefer to have the data

you can control and then use those that

data to actually re reach an endgame

while controlling that data. I just I've

seen so many places where you can you

could you can create bias in the data

that creates an an unexpected and un an

unpleasant outcome because you didn't

control the access of the data. You

can't completely do this. Um there's all

kinds of initiatives around the world

around data. The the broader concept is

what's called data rights. Um you know

who gets to benefit yeah from the the

monetization of a corpus of data. There

are three frames that you can think

about. One is a test. Uh whoever's using

your data will sign some sort of an

agreement that says they won't do this

and you can do these bad things to them

if they do that. That's great, but very

hard to you know, you can't rely on

that. The second one is audit, which is

you can you can, you know, put steps in

there that allow you to watch what

they're doing with your data and make

sure they're not doing what they're not

supposed to be doing. Again, very

difficult. They kind of move things

before you get there or they don't let

you look where you're supposed to look.

Um and you can do things to the data.

You can there's increasingly

sophisticated things you can do to put

fingerprints in data to look at the uh

there's differential privacy where you

can look mathematically at the the

changing trends in the data to

understand if it's been manipulated and

certainly there are trust solutions with

blockchain and things like that where

you can know that things are unperturbed

from the point of dissemination to the

point of use. All of these things are

necessary but not sufficient. At the end

of the day, data is going to be a little

bit squishy out there. And what we have

to do is understand that the older it

gets, the less valuable it gets. All

true data isn't true at the same time.

So if you're making the data, then then

you know you can do all of these things,

but you're never going to be completely

protected.

>> What about personal fines in cases of

serious data breaches or bias

infiltration? Go. Yeah. Yeah. Sorry. You

have that in in many parts of the world,

not here in the US,

>> in this no in this country. So if a 100

million names are released and my credit

card shows up there, how about the the

CEO goes to jail or pays, you know, $20

million fine.

>> How do you know that that data came from

that? Two different questions.

>> The qu the question first is the

question that Claudia is asking is how

you protect the data.

>> Okay, if you're trying to protect it by

saying after the fact, I'm going to I'm

going to hang the CEO if they allow

this. Sorry, that's not really out of

the barn.

>> That's that's still out of the barn. And

frankly, part of this comes from being a

little more draconian at the beginning

of the process. We're going to build

this set of AI for these purposes with

these kind of models, which means we you

guys will recognize this. There are some

places you actually build the the the

stable that you're going to use and

you're restricting yourself by the data

that you authorize to use for at the

beginning. that slows progress and

people don't want to do this. The

shareholders, the investors want to go

faster, but the people who recognize

that the danger of of allowing that

rather access that data outweighs the

advantage of doing this faster. The act

of the adversarial economy if I allow it

to go faster, especially think of

medical data. Think of the damage you

can have if you allow the medical data

for 100,000 New Yorkers to be exploited

because you wanted one particular

health insurance company to have a

better bottom line. The outline of that

kind of failure means that you should be

spending more time and more effort being

more draconian about protecting the

data. Smeari Mohan, who is a general

counsel at Awesome, which I looks like

owns uh SmugMug and Flickr, says this.

From a legal standpoint,

she's increasingly concerned about how

liability is or is not assigned when AI

systems cause harm or are exploited

adversarially.

In a world where AI agents act semi

autonomously and their failures stem

from complex supply chain complex supply

chains of data models and algorithms.

Who do you think should bear

responsibility?

And if you have any insight into this,

how do you think the law should evolve

to address these distributed risks?

There isn't a a simple answer to that

and there isn't a complete answer to

that. There are certainly AI codes of

practice that have been published, many

in fact around the world. Again, I'd

point to the EU probably has one of the

more robust ones. Um, so one of the

things that you can do is you can start

to hold people liable just like you do

when they violate the constitution or

when they violate the code of uh ethics

in medical practice or when they violate

the um the the the code of practice to

get your the your um training license.

You know, lots of different

practitioners have codes of practice and

you can hold them at least you can

attest to their liability when they act

outside of those general guidelines.

That's a good start. The problem with AI

is the problem of agency. If my if I

hire somebody to deliver dynamite for me

and they trip and fall and hurt

somebody, they're acting as my agent and

you can come after me. It doesn't work

that way with AI agents. If my AI agent

goes and does something that's biased or

you know does something malign somebody,

you have to be able to trace that back

to my valition and it's nearly

impossible to do that.

>> Okay, time I'm going to give you I'm

going to try to get an edge what you're

on a on a practical response. I won't

talk about the law because I'm not a

lawyer and don't frankly I think that's

tertiary. primary is first figuring out

what the right thing is to do at the

level of the where the action is being

taken. So every company,

every CEO,

every general counsel of that CEO, every

CTO or that who works for that that CEO

should be saying what are our policies

on AI? At at Motive Partners, we have a

very clear unambiguously written by

lawyers and practitioners for what the

policies are for how we practice AI at

Motive Partners. And we do so carefully.

And every portfolio company we invest

in, we give them guidelines for how

here's how we think your AI should be

developed and the carriage you should be

taking with these guard rails. Start

with that. The practical value is

companies who it's companies who make

the choices. companies led by people by

women and men who are in the CEO CGC

CTO seats make those decisions in line

with clear unambiguous policies and then

stick to it. Then when the law the law

will catch up and say if you have that

you're doing the right thing we'll

reward you. If you don't have that and

are not doing the right thing and are

using this to exploit grandma and her

her 401k then we'll pin you to the wall.

You have to be careful to also watch

what your AI is doing because it does a

great job of working around what you

told it not to do. There's a great

example out there, I won't name it, but

in a country where they were, you're not

supposed to use gender to make a

particular type of decision that had to

do with parole. And so they they just

redacted gender from being used in the

AI. And they later deconstructed that

there was still a gender bias because in

that particular language, female names

ended in A and E. And the AI is sort of

convoluted around the vowel at the end

of the name because it didn't have the

gender. So you can't just take your

hands off and say, "I'm good to go. I

followed the policy and now I can push

this button." You got to pay attention

to what's going on.

>> This is from Elizabeth Shaw. She says,

"Is it the confluence of all the

factors, adversarial economy, uh, etc.

that produces these AI misadventures? If

so, you can't control everything. So

what should companies do?

>> Yes, it is the confluence of all these

factors. And the worst thing you can do

is say, well, gee, it's complicated, so

there's nothing I can do. So the most

important thing here is the way a clam

eats a whale is one bite at a time,

right? You take the most significant

step you can take purposefully in a

direction on purpose and you lather,

rinse, repeat. This is blocking,

tackling. There's nothing new here. They

had this problem when light bulbs came

out. bad this problem with electricity

came out. This is not a new problem.

>> Do it at the company level. Do not try

to do this. Do not give every developer,

every every agent their own ability to

make the decisions do at the company.

And I'm I'm a capitalist. I believe this

is one of the ways that competition

actually works for us. When companies

see that this is in their best interest

to be resilient and to do the right

thing, we'll get a better outcome.

>> This is a really interesting one for

simple question.

Not such a simple answer, I suspect.

From Simone Joe Moore, she says,

"Governance versus the law are often two

different things. How do we manage chaos

situation where these two are far behind

the advance of AI use?"

>> Sorry. Governance versus the law. The

law is always a lagging indicator. The

law is a function of of the judicial

process which occurs

in in the next decade after most of the

math has been done. Governance on the

other hand can be done. It's not it's

not a oneandone. It's not a it's not a

proof. Governance is a statistical

process. Think of think of how AI has

has developed over the years from where

we had the initial machine learning

where we're now all up to AI. The

governance has to evolve at that same or

even faster pace and that can be done by

people who are actually building the AI.

So don't you don't wait for the law to

tell you what to do. You think through,

forgive me, philosophically, is this the

right thing to do and can we do this in

a way that actually moves both the right

competition forward but also does the

right thing for people and then that

governance is becomes part of your

policy and then you enforce that policy

do it with the right kind of intent the

law will catch up the only one thing I

would add to that beautifully poetic

answer is governance starts with first

principles what do we believe and how do

we know that we're being

And if you don't start there, you wonder

with the traffic code. You get so many

different policies that you can't

possibly comply with them. So you've got

to go back to your first principles,

which I think is philosophy. That's why

that that's my philosophical approach to

this is because the first principles

like the same thing. What's the right

thing to do?

>> I wanted to call it epistemology, but

then he would be

>> there. We got go around.

>> And with that, a huge thank you to

Steven C. Daffron and to Anthony

Scriffino. Gentlemen, thank you both for

being here. You were brilliant and I

can't thank you enough. I'm grateful to

you both.

>> Our pleasure.

>> And thanks to everybody who watched and

especially you folks who asked such

great questions. Now, before you go,

subscribe to the CXO Talk newsletter. We

have great great shows coming up. This

episode will be posted on the CXO Talk

website by Monday. It'll be lightly

edited and there'll be summary and all

kinds of great information that'll pop

up next week. So, check it out and

everybody, we'll see you again next

time. Take care now.

[Music]