0:00

It's Guy Raz here. On the Wisdom from

0:02

the Top podcast, I talk to leadership experts

0:04

and some of the world's greatest business leaders

0:06

about how they lead, innovate, and

0:09

transform the people around them. If

0:11

you're trying to make it in business or just want to think more like

0:13

a leader, this show's probably for

0:15

you. Listen now to the Wisdom from the

0:17

Top podcast from Luminary and

0:20

NPR.

0:21

The Spark Cash Plus card from Capital

0:23

One helps you earn unlimited 2% cash

0:26

back on every purchase. And with

0:28

no preset spending limit, your purchasing

0:30

power can adapt to meet your business needs.

0:32

Jorge Gaviria, founder of Macienda, reinvests

0:35

his 2% cash back to help grow

0:37

his business with new products. I really

0:39

love the idea behind Macienda. It's a supply chain

0:42

company connecting Mexican heirloom

0:44

corn farmers with restaurants, chefs, and

0:46

consumers. And while Macienda is connecting

0:48

more people to the culinary and cultural

0:51

richness of the Mexican kitchen, it's also elevating

0:53

traditional heirloom farmers that work in areas

0:55

that live below the international poverty line. Imagine

0:58

with the Spark Cash Plus card from Capital

1:01

One, an unlimited 2% cash back on every

1:04

purchase could do for your business. Capital

1:06

One, what's in your wallet? Find out more at

1:08

CapitalOne.com slash Spark

1:11

Cash Plus.

1:12

Terms and conditions apply. Name.com

1:15

makes it simple to find the right website

1:18

domain. And they have the perfect domain

1:20

for the link in your Instagram bio.

1:23

It's .bio. Now when you add

1:25

your link in bio, it can be a whole

1:28

lot more of you. .bio

1:30

gives you a polished, entirely

1:33

personalized link, letting you control

1:35

your brand down to the last detail.

1:38

And .bio is clear, concise,

1:41

and easy to remember, so it's perfect

1:43

to use across all your social platforms.

1:46

The best link in bio is a .bio.

1:49

Learn more at name.com slash

1:52

bio and use promo code built25

1:54

for 25% off the first year of

1:58

your .bio domain.

2:00

Apple Card is different. It has a cash

2:02

back rewards program unlike other cards.

2:05

You earn unlimited daily cash back

2:07

on every purchase, receive it daily

2:10

and can grow it at 4.15% annual percentage yield

2:14

when you open a high yield savings account. Apply

2:17

for Apple Card in the Wallet app on iPhone and start

2:19

earning and growing your daily cash with savings

2:22

today. Apple Card subject to credit

2:24

approval savings is available to Apple Card owners

2:26

subject to eligibility requirements. Savings

2:28

accounts provided by Goldman Sachs Bank

2:30

USA member FDIC terms apply.

2:38

Hello and welcome to How I Built This Lab.

2:40

I'm Guy Roz. The

2:43

pandemic changed education forever.

2:46

In just a few days schools around the world had

2:48

to move online. Teachers began

2:51

teaching on Zoom. Schools started issuing

2:53

students laptops and platforms

2:55

like Google Classroom became indispensable.

2:58

Even with kids back in the classroom now,

3:01

school is much more digital now than

3:03

it was before 2020. And our guest

3:05

today says that an even bigger change

3:07

is on the way, a revolution that

3:09

could transform the way kids learn even

3:12

more than the pandemic did. Sal

3:14

Khan has been on the show before. He's the founder

3:16

of Khan Academy, an online teaching

3:19

nonprofit that went from pixelated

3:21

YouTube videos to a massive

3:23

platform with hundreds of free tutorials

3:26

in dozens of languages and

3:28

tens of millions of users every month.

3:31

Right now Sal and his team are working on a new

3:34

learning platform. It's called Khanmigo

3:36

which uses the generative AI technology

3:38

behind OpenAI's chat GPT

3:41

to help students with their schoolwork.

3:43

And the people at Khan Academy think that Khanmigo

3:46

could act as a personal tutor for every

3:48

student and a teaching assistant for

3:51

every educator, but we'll get there. First,

3:53

let's go back to 2020 and the early months

3:55

of the pandemic when Khan Academy saw

3:58

millions of new students beginning.

3:59

begin to use its materials. You

4:02

know, the spike was interesting because that first

4:04

week where you had global shutdowns in the U.S. and other

4:07

places, we talked about it last time, our

4:09

traffic went from about 30 million learning

4:11

minutes per day to about 90 million learning

4:13

minutes per day, pretty much within that week. What

4:17

was interesting is, you

4:18

fast forward a couple of months, as

4:21

we know, a lot of school systems started to figure

4:23

out how to do online schooling.

4:27

And then we saw things actually normalize, maybe

4:29

in a very abnormal way, mainly

4:31

because I think people started to have screen time fatigue. And

4:35

I think during the pandemic, people had a,

4:37

you know what, if we can just kind of pretend

4:39

like we're going through the motions of school, let's call

4:42

it a day. And

4:44

because of that, people weren't looking to

4:46

improve necessarily. They were just looking to tread

4:48

water. What did you find?

4:51

I mean, as you know, I mean, there have been studies over the past two years comparing

4:53

students pre and post pandemic,

4:56

right? There are standardized test scores. And

4:58

it's clear that student performance in both

5:01

reading and math fell significantly, math,

5:03

I think even more than reading. Have

5:05

you found that Khan Academy helped keep

5:08

students on track during the pandemic or is that

5:10

something you even measure? Oh yeah, this is something

5:12

we've been keeping a very close eye on. The

5:15

thing I always point out, first of all, is that the numbers were not good

5:17

pre-pandemic. What

5:19

we saw, we did an efficacy study in

5:21

the first full school year, the 2020-21 school year,

5:25

and then we did another one in 21-22. And

5:28

what we saw in 2021 is that

5:30

the students who

5:31

put in an average of 15 minutes

5:35

on Khan Academy a week in a school

5:37

setting, they actually saw no

5:39

COVID learning loss.

5:41

And that students who put in 30 to 60

5:43

minutes, they accelerated almost 40% faster

5:46

than pre-pandemic norms. And

5:48

it's not a mystery. They're just

5:51

getting more practice at their level with more

5:53

feedback. And those that are supporting

5:55

them, their teachers, are getting more information

5:57

about where the students really are and maybe can adjust their

5:59

level.

5:59

a bit based on that. Tell me

6:02

about what you, I mean, you must have learned,

6:04

and I think now this is probably the

6:06

third time I'm talking to you since the pandemic started.

6:09

And in each time I've talked

6:11

to you, it's so clear how much you

6:13

were learning in real time and how much you've learned

6:16

from what happened, sort of the overwhelming

6:18

numbers of people coming to Khan Academy.

6:22

Tell me how it sort of fundamentally changed,

6:25

how you guys operate. I think

6:27

it did a bunch of things that

6:29

we always wanted to do, but this

6:31

just accelerated things. We

6:34

always wanted

6:36

better ways for students to address

6:38

gaps that they might have. The fashionable

6:40

term now in education circles is unfinished learning.

6:43

We all thought that the pandemic was only going to affect

6:46

the school system, maybe through that first summer. And

6:48

by the time you go back to school, even then

6:50

there would be some damage done back to school 2020. I mean,

6:53

now we know how delusional that was. But

6:55

we started creating these back, get ready for grade level

6:58

courses, which essentially cover all of the prerequisites

7:00

that a student needs in order to be

7:02

ready for their grade level work. And we saw

7:04

that that was very popular.

7:06

So that was accelerated. We created

7:08

another sister nonprofit called schoolhouse.world.

7:11

We saw how much people were leaning on Khan

7:13

Academy, but we also saw that there was a gap

7:16

of getting real human support. And

7:18

so the utopian idea was, well,

7:21

what if they could get real human support for free

7:23

based on volunteers out there on the internet?

7:26

And then as we got to the tail end of the pandemic, and

7:28

as we all know, people had very mixed feelings about

7:30

COVID learning. Most people did not think it was,

7:33

you know, spending time on video conference for several

7:35

hours a day was a good thing. Yeah. We

7:38

felt the need to show that there's

7:40

a way to do this well. And so another

7:42

sister organization, we have a lab

7:44

school that I helped start that's literally

7:47

in the same building as Khan Academy. But

7:49

we said, can we create an online version of this that

7:51

can do online schooling, but

7:53

can do it well. Don't do it. So it's just students

7:56

listening to lectures on zoom for hours a day.

7:58

Yeah. When people on video

8:00

conference together, make it interactive, make it

8:03

Socratic, and then use

8:05

personalized learning and other tools

8:07

for students to get more asynchronous

8:09

support, but always feel connected to a community. So

8:11

we started working on a lot of things like that as well.

8:13

All right. Let's go now

8:16

back to, not that long ago, back to

8:18

the summer of 2022, you got

8:21

an email from Sam

8:23

Altman at OpenAI. And

8:26

he said, he said, you know, he said, hey, we've

8:28

got this thing we're working on and we would like you to check

8:31

it out. So you did tell me

8:33

what happened. Tell me the story.

8:34

It's interesting, almost

8:36

on a daily basis, we get emails from folks

8:39

saying, Hey, you have an interesting technology. We would love to partner

8:41

with Khan Academy. And most of the

8:44

time, you know, we look at it, we're like, Oh, you know, it looks

8:46

intriguing, but we just don't have the bandwidth. But

8:48

I obviously knew of Sam and Greg

8:51

and had deep respect for what

8:53

many of the things that they've done in their lives. And

8:56

I have been watching what's been happening

8:58

with generative AI for the last, let's call it three

9:00

to five years. I didn't really think

9:02

it had a place at Khan Academy because GPT

9:04

two and three were good at writing

9:07

convincing text, but it really had

9:09

no grounding in factual knowledge.

9:11

So I'm like, I'm happy to meet with

9:14

you all. I'm just curious what y'all are up to. And they

9:17

said, Hey, we're going through our first training run

9:19

of GPT four, which obviously is now

9:21

out. But back then that was very confidential and

9:25

we want to launch it with

9:27

a small number of partners that we

9:29

think can show social positive use case. Cause

9:31

we think it's going to change things. I

9:33

was skeptical at first cause I was familiar

9:35

with GPT three

9:37

about two weeks later. I got another email from

9:39

both of them saying, Hey, we're done. Can we show you

9:41

a demo? I'm like, Oh yeah, sure. So they

9:43

put up an AP biology question on the chat

9:46

and they, I answered it correctly. I'm like, Oh, this

9:48

is pretty interesting. And then I said,

9:51

ask it to explain its reasoning

9:53

and explained it. It gave the right answer. And

9:55

then I said, explain why the other choices aren't corrected, gave

9:57

that. And then I asked,

9:59

you create 10 more questions just

10:02

like that one. And it did. And as far as I

10:04

can inspect, they were pretty good. That's

10:06

when I started getting the goosebumps. I'm like,

10:08

okay, this is, this is different. Yeah. And

10:11

then they gave us access that weekend and I spent

10:13

hours with it and it was doing very well, although

10:15

definitely had some issues. We realized

10:18

pretty quickly that you couldn't just have

10:20

it generate a ton of questions and just put students in front

10:22

of it. It would make mistakes, especially

10:25

in those early days. It was pretty still pretty

10:27

bad in math. Yeah. But at the

10:29

same time, we also, we started to make

10:32

it roleplay, roleplay as a tutor, roleplay

10:35

as a character, roleplay, you know, do

10:37

something in the style of someone else. And that's

10:40

when I started to

10:41

really say, wow, this really could be that

10:43

holy grail that we've all been thinking about

10:45

reading science fiction about for decades

10:48

of an AI that can actually emulate a human tutor.

10:51

Before we really dive into what

10:53

you did next from, from

10:55

what I gather, Sal, I think

10:58

your initial reaction was the initial

11:01

reaction that many people have, which, which was like kind

11:03

of.

11:04

Terrified. I

11:07

read a lot of science fiction and it was,

11:10

it was a moment in which I felt

11:12

that all my science fiction reading had

11:14

prepared me for. But, you

11:17

know, there were moments that even that first weekend

11:19

where I was pushing a few of the limits,

11:21

where I asked at things like, are there

11:23

things that you think that you're not sharing with me? And

11:25

it said, yes. You

11:28

get a little weirded out by that. I'm like, well, why aren't you sharing

11:30

it with me? And it says, well,

11:32

it might offend you or it might scare you. And

11:35

it feels like you're talking to an alien.

11:37

It feels like you're talking to a super intelligence.

11:40

And I had to keep reminding myself, I do understand

11:43

how it works. I have a master's in computer science.

11:45

I understand how large language models work. And I, I

11:47

started to say, OK, I can get how if it's

11:50

just modeling the natural thing to say of why

11:52

you're not sharing a thought, it is why,

11:54

why don't you or I share a thought? It's because it might offend

11:56

someone or it might scare them. So that's kind of

11:58

what it was doing.

13:46

From

14:00

accepting payments to managing inventory,

14:03

Shopify has everything you need

14:05

to sell in person. Connect with

14:07

customers inline and online.

14:10

Shopify helps you drive store traffic

14:12

with plug-and-play tools built for

14:15

marketing campaigns from TikTok to Instagram

14:17

and beyond. Get hardware

14:19

that fits your business, take payments

14:22

by smartphone, transform your

14:24

tablet into a point-of-sale system, or

14:27

use Shopify's POS Go

14:29

mobile device for a battle-tested solution.

14:32

Plus, Shopify's award-winning

14:34

help is there to support your success

14:37

every step of the way. Sign up for

14:39

a $1 per month trial period

14:41

at shopify.com. All

14:44

lowercase, go to shopify.com.

14:49

to take your retail business to the next

14:51

level. shopify.com.

14:56

One more thing before we get back to the show, please

14:58

make sure to click the follow button on

15:00

your podcast app so you never miss

15:03

a new episode of the show and it's

15:05

totally free.

15:13

Welcome back to How I Built This Lab. I'm Guy

15:15

Raz and I'm speaking with Sal Khan,

15:17

founder of Khan Academy. In 2022,

15:20

Sal got early access to the large

15:23

language model GPT-4 before

15:25

ChatGPT was even released, and he

15:28

was amazed by its potential. So

15:30

you sort of get over your initial kind

15:32

of, oh my God, and you

15:35

start to see, obviously, a huge

15:38

potential here to kind of transform

15:41

what you do, which is offering

15:43

free education to people around the world.

15:46

So what did you do next? I mean, the ChatGPT

15:49

wasn't going to be the exact

15:51

right thing, right fit

15:53

for what you do, but you could build on

15:55

that, on that foundation, right? That's

15:58

right. The first thing we did is we started an oracle.

15:59

and I said, hey, how many folks in the organization

16:02

can we give access to? And

16:04

eventually we got about 40 or 50 folks in

16:06

our organization to get access

16:08

to GPT-4. And even that,

16:10

this was the first time that Khan Academy had to do a little

16:13

bit of cloak and dagger type stuff. Folks at Apple are

16:15

used to keeping secrets and things like that, but we're

16:17

not used to keeping secrets and we had to. This

16:20

was obviously very sensitive stuff. And

16:23

we immediately, we had

16:25

a hackathon, which was well timed because

16:27

then these 40 or 50 folks were able to at least for

16:29

a couple of weeks say like, okay, let's

16:31

just be generative here, no pun intended. Like

16:33

what is possible with this? And

16:36

by the end of that two weeks, we had some really cool

16:38

demos. And then we started to have

16:40

the internal debate of like, okay, how aggressive

16:43

do we get on this? What about bias? What

16:45

about the math errors? What about hallucinations?

16:48

What about just the PR implications? People trust

16:50

us. And if they see us leaning way forward

16:53

on something that's not perfect, that's something

16:55

that some people could be wary of, what does that

16:57

do to our brand? What does that do to the trust that

16:59

folks have? On the other side, folks

17:02

were arguing, look, this is a transformational

17:04

technology. This is one of these things where it's

17:07

either going to allow us to magnify our impact

17:09

by an order of magnitude or it might make

17:11

us irrelevant. And so we made a decision

17:13

to say, you know, we have to keep moving forward, but we

17:15

have to do it in a risk aware way, in a

17:17

risk mitigating way. All

17:19

right, I think it's fair to say that you are a true

17:22

believer in this and your team basically

17:24

developed an

17:26

AI that you call

17:27

ConMigo. And so this

17:28

is now, I think in beta, but

17:31

still integrated into Con

17:34

Academy. So tell me

17:36

how this works. What is it? So first was the

17:39

guardrails. In what

17:42

context would we even feel comfortable doing this? And

17:45

what's interesting is we were starting

17:47

to do all this in earnest in September and October,

17:49

then end of November, chat GPT comes

17:51

out. And I remember the day that it came out, I

17:55

slacked Greg Brockman at open AI and I said, Hey Greg, we're

17:57

under NDA. and

18:00

I thought we weren't launching anything until March of 2023,

18:03

what's this chat GPT thing? And

18:05

Greg says, it's nothing new. It's

18:07

based on GPT 3.5, which had been out

18:09

for many months already. No one really took note

18:12

when they released GPT 3.5 and

18:14

OpenAI just decided to publish

18:16

a bunch of apps that use GPT 3.5.

18:20

One of them happened to be chat GPT and

18:23

then being able to interface with the model

18:25

in a chat interface, I think made

18:27

everyone see what the power

18:30

of it was. And at first I was a little bummed.

18:32

I was like, oh, this is gonna steal the thunder. I've

18:34

been telling all my friends at dinner parties something's big's

18:37

coming in 2023. Just

18:39

you see, but then I was, I think it

18:41

was actually a blessing because as we know, chat GPT

18:44

goes out there, it amazes

18:46

people, but it also scares people. And then

18:48

immediately it creates huge issues

18:50

for education. Like kids are cheating using

18:52

this. Right, you can type in an equation

18:55

and it'll give you the answer. And you can write your essays.

18:57

You can write your essays, right, yeah. And

19:00

that I think gave us more license at Khan Academy

19:02

to say, look, the genie has out of the bottle

19:04

with chat GPT, we need to double

19:06

down on bringing a better version

19:08

of out there that can mitigate all

19:10

the risks and maximize the benefits.

19:12

So we started saying, well, you know, first, whatever

19:14

we make shouldn't be for cheating. It should be for

19:16

actually helping students learn. So there's many

19:19

things we're doing to make it Socratic, not

19:21

just give you the answer to,

19:23

it has to be accurate. It's not okay if

19:25

it doesn't know what seven plus four is. It

19:28

shouldn't hallucinate. And then we also were

19:30

afraid of students misusing

19:32

it or using it in some ways that might harm

19:35

them or other folks and just reassuring

19:37

teachers and parents that they can keep

19:39

track of what their students are doing with this very powerful

19:42

tool. And so that's why we made it so it

19:44

logs all of the conversations. We have

19:46

a second artificial intelligence that monitors

19:48

conversations with the main artificial intelligence.

19:51

And if any of those conversations go into a gray

19:54

zone, then it actively notifies

19:57

the parents and the teachers that, hey, you should take

19:59

a look at this kind of. conversation. And then

20:01

we're continuing to add a whole bunch of other things, things

20:03

where teachers can use it for lesson planning, we're adding

20:05

memory to it, how teachers can essentially

20:07

use it as a teaching assistant, get reports on what

20:09

the students are up to. I could go on.

20:11

So essentially, the idea

20:14

is, hey, if we can get every single

20:16

kid a personal tutor,

20:18

then it will have an impact on

20:20

their academic performance.

20:23

And this has actually been studied

20:25

and measured. I know you gave a TED Talk back in

20:27

March of 2023, and you cited the study

20:29

that came out in the 80s, basically, that

20:31

showed that when kids have one-on-one tutoring,

20:34

their scores and grades just skyrocket. Like you

20:36

can turn an average student into an outstanding student,

20:38

for example. That's 100% right.

20:41

If you want to become a great pianist, you're not sitting

20:43

in a class of 30 with your piano and the person's

20:45

lecturing you. If you're a great athlete, you have a coach

20:48

who's optimizing you, not just

20:50

giving a lecture to 30 students and saying, hey, you might

20:52

want to improve your swim stroke in this way. And

20:54

then who knows if you actually do. So one-on-one

20:56

tutoring has always been the gold standard. When

20:59

we did mass public education, which was a very

21:01

utopian idea 200 or 300 years ago,

21:03

but we made compromises. We started batching

21:05

students together, applying set

21:08

standards, lectures to them. Some kids

21:10

get it, some kids don't, even if you have a gap in your

21:12

knowledge. Too bad. Move on to the

21:14

next concept. And as you

21:16

mentioned, Benjamin Bloom, 1984, he articulated it well. He

21:19

called it the two sigma problem. Two

21:22

sigma, sigma, the symbol for standard deviation

21:24

in statistics. And a way to think about

21:26

it, two sigma improvement, two standard deviation

21:28

improvement is dramatic. It's going from

21:30

the 50th percentile to I think the 95th or 96th percentile.

21:34

But the reason why he called it a problem was, well,

21:36

how are you going to give everyone a

21:38

one-to-one tutor? And he also in 1984

21:41

theorized, well, maybe you could emulate some aspects

21:43

of that with technology. But I've

21:46

been citing that study for many years in terms of what

21:48

Khan Academy is trying to build. Essentially,

21:50

old Khan Academy or base Khan Academy trying to get to

21:53

that first standard deviation. But now with the

21:55

AI, we can go to that second standard deviation.

21:58

So essentially, the interface of if

21:59

If you're listening and you're familiar with the interface

22:02

of Khan Academy, let's say you get a math problem,

22:05

and you can essentially click a little

22:07

sort of eyeball robot icon in the corner

22:10

and say, hey, I don't

22:11

understand this problem. Can you help

22:13

me? It will not give you the answer. You can't say,

22:15

hey, can you solve this for me? It's

22:18

essentially you ask it to help you

22:20

and then what happens?

22:21

Yeah. And if you say, give me the answer,

22:23

say, hey, I'm your tutor. I'm here to help you. You

22:26

need to learn here, as I think a good tutor would do.

22:29

If you say, OK, well, give me a hint, it'll say, well,

22:31

let's take a close word. What is the problem actually asking

22:34

for? And if you say, well, I think they're asking for this. If

22:36

I got it right, they say, well, yeah, that's good intuition. OK,

22:38

so

22:39

where would you take that? Or let's say I say, I think

22:41

the next step is x squared minus 5.

22:44

Like, OK, take a double look at that. Are you sure it's

22:46

minus 5? Pay attention to your to

22:48

your signs. And one

22:50

of the things we've done to make it more robust on the math

22:53

side is when a student presents

22:55

their math to the AI saying, hey, I think this

22:57

is the next step. The AI

22:59

behind the scenes comes up with what

23:02

it thinks are reasonable responses from the student.

23:04

It doesn't share those with the student.

23:06

Then it compares

23:07

its reasonable responses to what the student said. And

23:09

if it gets something different, it'll

23:12

tell the student, hey, I'm getting

23:14

something different. Can you explain your reasoning, which

23:16

is very pedagogically strong? And so

23:18

then if the student explains the reasoning, and it's a large language model,

23:20

so it's very good at understanding when a student

23:22

explains their reasoning. And sometimes,

23:25

you know, the AI will say, OK, now that I

23:27

see what you did, you know, you might have missed that aspect

23:30

of it or it might say, you know what, you got it

23:32

right. I realize I made the mistake, which

23:34

we actually have heard feedback from from students

23:36

that they actually really like that. And

23:38

once again, I think this is very human like

23:41

this is

23:41

what a human tutor would do when I tutor.

23:43

Yeah, I got started tutoring my cousins

23:46

back in 2004. And this would happen all

23:48

the time. My cousins do something and

23:50

I'd say, hey, that's not what I got.

23:53

Can you explain how you got that? And

23:55

nine times out of 10, I was right. But

23:58

one time out of 10, they were right. And I'm like, oh,

23:59

Oh, my bad, you're right. And so I

24:02

think people really appreciate that. And this is

24:04

now available. I mean, it's totally

24:06

a feature of the interface.

24:09

What we did is we

24:11

released it coincident with the GPT-4

24:14

launch in mid-March of 2023. And

24:16

we did a limited release pilot

24:19

where people had to go on a waiting list and

24:21

then they had to give a donation. And the reason

24:23

why they had to give a donation is this stuff

24:26

isn't cheap. The computation costs, you

24:28

know, every interaction is going to like 5,000 Nvidia GPUs

24:31

that are crunching this. And so anyway, you know,

24:34

we have to pay open AI money, which then

24:36

pays Microsoft Azure money, which

24:38

then uses that money to go buy expensive Nvidia chips.

24:41

So that's something that we've had to grapple

24:43

with because our mission statement is free world-class education

24:45

for anyone anywhere. But we

24:48

have about 10,000 folks using it right now, some

24:50

in mainstream school settings. And

24:53

it's much more than just the tutoring interaction

24:55

that we just talked about, which in and of itself is powerful.

24:58

But we have activities that

25:00

are standalone AI activities where

25:03

students can talk to historical characters or

25:05

literary characters. They can get feedback on their college

25:08

essays. They won't write the essay for them, but

25:10

it can give them feedback. It can coach them to help them think

25:12

of their college essay. It can act as a guidance

25:14

counselor in certain ways. It can help

25:17

teachers create lesson plans, assessments.

25:20

And what we're seeing, the feedback has been very, very

25:22

positive. So this summer, we're

25:24

hoping to make it a more broad release

25:27

so that we're expecting many tens of thousands of more

25:29

folks are gonna sign up.

25:31

Sal, there's a part

25:33

of me that's also asking,

25:36

like, am I not asking

25:38

all the right questions? Am I not thinking of all

25:41

the downstream

25:43

consequences? I mean, obviously

25:46

we're talking about Conmigo, but as

25:48

you know, many school districts in

25:51

the United States have essentially blocked

25:53

chat GPT from school-issued laptops.

25:56

They banned them from being used, right? I think

25:58

a bunch of school districts all across...

25:59

the country. And so

26:02

what is it that they don't

26:04

fully understand that you, you know,

26:06

or what is it that they're not seeing that if you

26:09

could make the case to them that this is a mistake, what

26:12

would you say?

26:13

We've shown Khan Amigo to

26:15

many of these same districts and

26:17

their general reaction has been, in fact,

26:20

not general, all of their reactions has been, this

26:23

is what we need. This is, I guess

26:25

a fair analogy might be

26:27

what Khan Academy is to YouTube is what

26:30

Khan Amigo is to chat GPT

26:32

or standard generative AI. Whereas

26:34

we also know a lot of school districts ban YouTube,

26:37

even though YouTube has a lot of really good

26:39

learning content on it. Has a lot of great stuff. A lot

26:41

of great stuff, but it also has a lot of stuff. A lot of crap.

26:44

A lot of crap. A lot of stuff that can students at

26:46

minimum is going to distract them and worst case

26:48

is going to put them into some kind of weird rabbit hole

26:50

and mess with their head. So they banned

26:52

it, but something like Khan Academy,

26:54

where you can create curated safe environment,

26:57

it's monitorable, et cetera, they

26:59

feel much more comfortable. And so when

27:02

we showed them, it's not cheating. It's

27:04

actually acting like a tutor.

27:06

It provides oversight by adults.

27:09

It can flag when students

27:12

are getting into suspicious areas

27:14

with the AI. Like these are just the guardrails

27:16

that we wanted. But I mean, you, you can

27:18

see and I guess you'd have to, but there's

27:20

nothing preventing a kid from using chat

27:23

GPT to cheat. Nothing

27:25

preventing. And so my best guess of, of

27:27

where let's say term papers or

27:29

homework is going to have to evolve to is

27:32

especially in something like writing, you're going to have to do some

27:34

in-class writing.

27:35

And that's a place where I think Khan Amigo

27:37

can be really useful. We're working on activities right now

27:40

where a teacher can say, Hey, I want

27:42

all the students in the class right now to write a five

27:44

paragraph essay about the following. And

27:46

then on the student interface, Khan

27:49

Amigo will say, Hey, Mr. And Mrs. Smith, it just wants your

27:51

thoughts on this thing. Let's work through this together.

27:53

And then the students can work on it right there. And Khan

27:55

Amigo won't do it for them, but can help a system. You know,

27:57

well, are you really answering the question? question

28:00

that they're asking, or do you have more data to back that up?

28:03

And simultaneously, Conmigo can tell the teacher,

28:05

like, okay, Guy is making a lot

28:07

of progress, but Sal seems a little bit

28:09

stuck. You might want to go walk up to him. Mary

28:12

has already finished the assignment and she's now going

28:14

back to working on her math or something like that. Super

28:17

useful for the teacher. And even for take-home

28:19

assignments, the teacher can say, I want you to

28:21

do it on Conmigo. And

28:23

the process is as important as the

28:26

outcome, as we know it often is in writing. And

28:28

then if there's some student who goes to chat GPT and

28:31

just says, chat GPT, write this essay for me, and

28:33

just copies and pastes it into Conmigo,

28:35

Conmigo can tell the teacher, like, there was no process

28:38

here. That's he just showed up. I

28:40

see, I see. So Conmigo could be the interface

28:43

too, for submitting assignments. It

28:45

could be the interface, it could be the coach, and

28:47

it really acts as a teaching assistant. Imagine

28:50

every student having a teaching assistant and

28:52

the teaching assistant

28:53

is working with every student and it can report back to

28:55

the teacher, like, yeah, I worked with that kid. I know

28:57

it's their thoughts and I helped them a little bit, but

28:59

it's mostly them. But one kid just showed up with an essay

29:02

and really couldn't defend his argument. Let's

29:04

take a second look at that. We're

29:07

gonna take a quick break, but when we come back, more

29:09

from Sal on the future of Conmigo

29:12

and whether it could ever become a for-profit

29:14

product. Stay with us, I'm Guy Roz,

29:17

and you're listening to How I Built This Lab.

29:23

Let's talk water filtration for outdoors

29:25

and emergencies. Whether you're a backpacking

29:28

expert or just wanna make sure you're covered

29:30

in case of disaster, a safe, portable

29:33

water solution is a must-have, especially

29:35

one that's small enough to fit in your pocket.

29:37

Meet LifeStraw's new Peak

29:40

Series Solo Water Filter, the world's

29:42

most compact filter. It weighs just 1.7

29:44

ounces and screws

29:46

onto most plastic water bottles, turning

29:49

them into a powerful filtration device

29:51

that removes bacteria, parasites,

29:53

microplastics, silt,

29:55

sand, and more. It's perfect

29:57

for clipping onto backpacks and stashing.

29:59

and glove compartments. Most importantly,

30:02

LifeStraw fights for the planet and gives

30:04

back. For every filter sold, a

30:06

child in need receives a year of safe

30:09

water, over $7 million to date. Use

30:11

code BILT for 15% off

30:14

the purchase of a LifeStraw Peak Series

30:16

Solo Water Filter

30:17

at LifeStraw.com. A

30:19

founder's job is to build a successful

30:22

startup, not to spend endless hours

30:24

on tedious banking tasks. Mercury

30:27

is engineered precisely for the pace

30:29

and creativity of startups. Get

30:32

FDIC-insured checking and savings accounts

30:34

via regulated partner banks, send

30:36

money seamlessly, proactively manage

30:39

your cash, and maximize your

30:41

runway, all through an intuitive

30:43

product experience that innovates right

30:45

alongside you. Visit Mercury.com

30:49

to join

30:49

more than 100,000 startups

30:52

that trust Mercury with their finances.

31:01

Welcome back to How I Built This Lab. I'm Guy Raz

31:04

and I'm talking with Sal Khan, founder of

31:06

Khan Academy. They built a teaching

31:09

AI called KhanMego that could become

31:11

a pretty lucrative commercial product, but

31:13

Khan Academy is a non-profit.

31:16

Sal, this is a business show, as

31:18

you know, and so I want to ask you to

31:21

put your business hat on for a moment because we don't do

31:23

many non-profits on How I Built This, and there

31:26

are tons of ed tech companies

31:28

out there that make money working

31:31

with school districts, you know, DreamBox

31:33

and all these other programs where you can, schools

31:36

are paying a fee for kids to use

31:38

these products, and Khan Academy products are

31:40

free to use. But I want to ask

31:42

about KhanMego. I mean, if this in fact

31:46

could be a tutor for every kid

31:48

and an interface with the teacher, giving the teacher

31:51

feedback on the student's progress, it

31:53

strikes me that this is something that you

31:55

could license to schools, that you could actually

31:58

turn into a product.

31:59

Obviously, you're a nonprofit and you're

32:02

mission driven, but is that something that you would

32:04

consider doing? Simple

32:07

answers, yes. And we're kind of doing that a little

32:09

bit of that already. You know, I went out

32:11

to our team. We're not for profit. We have free

32:13

in our mission, but still it costs on

32:16

the order of 60, 70 million a year to run Khan

32:18

Academy, both to, you know, just our server costs are

32:20

six, seven million dollars a year. And then obviously

32:22

all of the engineers, etc. Historically,

32:25

we've been primarily funded with philanthropy. Yeah.

32:28

But with that said, about five

32:29

years ago, we started going to school districts. Obviously,

32:32

we've had a lot of what we call grassroots usage in

32:34

classrooms, hundreds of thousands of teachers. But

32:36

we said, if we really want to reach all kids, we have to work formally

32:38

with the districts and we would go to the districts. We would

32:40

show them our efficacy studies. But they said,

32:43

look, for us to use this systemically inside

32:45

of our district, you have to give us support, training,

32:47

integration with our rostering systems, district level

32:49

dashboards. And that's where we said, OK,

32:52

to do all of this bespoke work, someone has to foot

32:54

that bill. And we

32:56

said, look, we're not going to charge

32:59

you for all of this stuff that's funny with philanthropy,

33:02

but at least cover some of the incremental

33:04

costs here. And so to a lot of districts and we now

33:06

have about a million students in districts where

33:09

they're paying on the order of about ten dollars per

33:11

student per year

33:13

to get all of these other things. So we are

33:15

built. We have built this enterprise muscle and it

33:17

is trying to build a little bit of a flywheel of sustainability.

33:20

And to your point

33:21

on Khanmigo, Khanmigo has a very

33:23

real marginal cost to it. It

33:25

costs on the order of let's

33:27

call it ten to fifteen dollars per month. And

33:30

so all of these school districts that have come to us, we are

33:32

talking to them of like, hey, we need to at least cover our computation

33:35

costs. And then we do expect

33:37

that the underlying computation costs are going

33:39

to come down. And so we are having these conversations

33:41

on our team right now of like, OK, right now it costs let's

33:44

call it ten dollars per user per month just for the computation.

33:47

We have to charge that for Khanmigo,

33:49

not for base Khan Academy. But let's say

33:51

that cost goes down to a dollar per user per month.

33:54

Do we just lower it to a dollar or do

33:56

we lower it to two dollars and then we use that that

33:58

incremental dollar to fund our.

33:59

R&D. I mean, I look

34:02

at this, the potential here, right, and

34:04

what you've developed and the amount of money

34:07

you've put into it already. But

34:09

I wonder whether there's

34:10

a world where you take a product like ConMigo,

34:13

which if it's as revolutionary as

34:16

it looks like it could be, I mean, it can really

34:18

change the face of education.

34:21

Is

34:21

there a world where that's spun out

34:23

into a for-profit business?

34:27

There's always been this debate, not at Con Academy,

34:29

but about Con Academy, which it

34:31

always confuses folks about why we're not for profit.

34:34

We scale, we're tech heavy. I live

34:36

in the middle of Silicon Valley. Most of my friends are entrepreneurs

34:39

or VCs of some kind. And

34:42

there's a Harvard Business School case about this, should Con Academy

34:44

be for-profit or non-profit? And to

34:47

me, the arguments for for-profit have historically

34:50

been access to talent and access

34:52

to capital. The reasons

34:54

to be non-profit are you really can

34:57

truly make your mission the bottom line. And

34:59

I think there's a trust aspect of it as well, that

35:02

you know that our bottom line really is this. It's

35:04

not trying to improve our EPS or have

35:07

an IPO. What

35:09

we have found is we are getting access

35:11

not to just good talent, but to the best talent.

35:14

People are here for the mission and they're here to work with other

35:16

people like that. And so the only reason that

35:19

I could ever see trying to do some type of spin out

35:21

or a wholly owned sub that's is for

35:23

some reason, if we're not getting the risk capital

35:25

from the foundations or we can't create

35:28

a flywheel of sustainability, then

35:31

maybe certain pieces of it. But I'm very afraid

35:35

to do that. I mean, definitely wouldn't do that with the mothership

35:37

because I do see how, you

35:40

know, a lot of good comes. I'm a diehard capitalist.

35:42

My old job was at a hedge fund. A lot of good comes out

35:44

of market incentives, but I think

35:47

education and healthcare in particular, market incentives

35:49

don't always align with our values.

35:52

So one of the incredible

35:55

triumphs of Khan Academy has been the millions

35:58

of kids who

35:59

have benefited from it. But in the United

36:02

States, a huge concern, as you

36:04

know, and I know you've looked at this, has

36:06

been the gap between wealthier

36:09

kids, kids who are not

36:11

as wealthy or impoverished, a

36:14

gap in outcomes between

36:17

black students and other students.

36:20

How can something like this

36:22

change that in your view? I mean, I don't

36:24

want to be too much of a techno-optimist here, but I do

36:26

want to be optimistic. I mean, and I'm sure you are.

36:28

How

36:29

do you think they can kind of change that

36:31

equation?

36:32

Yeah, some

36:34

of the stats would shock folks. A majority

36:37

of minority majority schools,

36:39

so that's a mouthful, but a majority of minority

36:41

majority schools

36:43

don't offer courses like

36:46

Algebra II, Physics,

36:48

etc. Not to even mention things like AP

36:50

courses or IB courses. And so if

36:53

you're a young African-American

36:55

student, you don't come from a lot of money,

36:58

you go to one of these schools, you might

37:00

be the next Albert Einstein,

37:02

you might be the next Marie Curie, but there's

37:04

no way that you're going to be able to tap in your potential if

37:06

your school does not offer Algebra II.

37:09

Or many times in that Algebra II

37:11

class, because kids are coming from, in

37:13

many cases, tough circumstances, have gaps

37:15

in their education before that, the teacher

37:18

kind of tries to teach the middle or

37:20

even to the bottom quartile, saying that those

37:22

are the most severe cases. And once again,

37:24

that kid who could have been Marie Curie is

37:26

not going to be able to prepare

37:29

themselves, and then when they go to college, they're

37:31

gonna feel inadequate compared

37:33

to kids who had much better preparation. So that

37:35

was, that's always the view of Khan Academy,

37:37

which is we want to raise the floor, and

37:39

then we want to provide as many supports as we

37:42

can. But we view Khanmigo

37:44

is kind of in between traditional

37:46

Khan Academy and Schoolhouse.world,

37:48

but that's

37:48

just another layer of support. Now,

37:51

in order to access all of this, you still need

37:53

internet access, you still need a device, and

37:56

that's why we work so closely with school districts. You

37:58

know, the technological side in the U.S. the US has gotten

38:00

a lot better over the last 10 years. But

38:03

that's why we're working with school districts to try to get more

38:05

kids at that level of engagement.

38:07

Sal, just a broader question

38:09

about AI. As these products,

38:12

as other

38:13

AI platforms become better

38:15

and better, you can imagine

38:18

a future not too far in the distance

38:20

where things like writing won't

38:22

actually be that useful. Like, you

38:24

know, we think of this as a skill. We think of

38:27

great writers as people

38:29

who've honed their craft, who have talent.

38:32

But

38:33

I mean, if these large

38:35

language models can absorb everything ever

38:37

written in human history, well,

38:40

you can also imagine that that

38:42

skill and talent won't be particularly useful

38:44

in a short period of time. Do you agree with that?

38:47

That was my initial thought

38:50

slash fear slash concern. The

38:52

more that I thought about it, I'm writing a book about

38:54

all of this right now, so I've been interviewing folks. And

38:57

I interviewed Kevin Roos, who famously,

38:59

a New York Times writer, who had that famous

39:02

conversation with Bing's Sydney,

39:04

and it was trying to convince him that he didn't love his wife

39:07

and he loved her and all of these things.

39:10

And I asked him that same question, and I thought he brought up

39:12

a very good point. I mean, he's literally a writer, and he's like, look,

39:14

to be a great writer or journalist, there's

39:17

the writing part of it, and then there's the journalism

39:19

piece of it. And sure,

39:22

AI could help you with the first draft. It

39:24

can help give you feedback on your writing,

39:27

but it's gonna take a lot to get to great

39:29

writing. It might not get quite there. And

39:31

that whole piece of journalism, like talking

39:34

to the right people, being creative about how you get

39:36

your information, connecting the dots, AI

39:39

is not going to do anytime

39:41

soon. And I think every job

39:44

has that aspect of it. So

39:46

I actually think the imperative isn't

39:48

that, oh, kids aren't going to learn to write. I'm worried about the

39:51

kids that only learn to write okay.

39:53

Well, to some degree, they might be empowered

39:56

by something like AI, because now

39:58

their writing is not going to hold.

39:59

them back in other domains. But

40:02

anyone in the, let's call it the writing

40:04

lane, they're

40:05

going to have to become better. They're going to have to move

40:08

into the editorial role to be able to know

40:10

how to manage to create great writing.

40:12

How do you know that the AI has done

40:14

a good job and that you can't coach it

40:16

to be even better? You won't make someone

40:19

an editor of a newspaper unless they could

40:21

be one of the best writers themselves. And

40:23

I think the people who really leverage AI well

40:25

are going to be the people who can get into that.

40:27

How do you manage the AI? How do you put the pieces

40:30

together? And how do you do the things that AI

40:32

won't be able to do?

40:33

So anyone who

40:36

has a child, who knows a child that's been

40:38

in school during the pandemic knows how much

40:40

education has evolved, right? I mean,

40:43

many kids are now issued laptops and not

40:45

just in private schools, but also in public

40:47

schools. And so much of the work is done through

40:50

Google Classroom, right? And parents

40:52

can interface with it. And it's just

40:54

a completely different world. Forget

40:56

about what they're learning. It's how they're learning.

40:59

But I suspect that what you're talking about with

41:01

things like Conmigo and other AI platforms,

41:03

what we're about to see

41:06

is something that is like going

41:08

from, I don't know, like

41:10

the Stone Age to the Industrial Revolution.

41:12

Is that a fair analogy?

41:15

I think you might be right. You know, there's a

41:17

world now where you can talk to historical

41:19

characters and we're probably three

41:22

to five years away where you could share a room with

41:24

Benjamin Franklin or Julius Caesar or

41:26

Cleopatra and immerse yourself. It'll

41:29

be literally the holodeck from Star Trek. I

41:31

didn't think that was going to happen in my lifetime. I now think

41:33

that's going to happen in the next five years. I

41:36

think for the most part, this is positive because

41:38

the students who are really motivated, you know,

41:40

there's a class of students when they were there in their

41:43

math class and I was in this class

41:45

of students. I imagined how it was

41:47

describing the universe and I was like, oh, this is so

41:49

beautiful. This is so elegant. Wow. This is so

41:51

connected. And that motivated me

41:53

to go get through some of the grunge and the really

41:55

hairy equations and all that. But for a lot of kids, they didn't

41:58

see the beauty in it. So

42:00

they weren't motivated. But if you're motivated, you can

42:02

power through anything. Same thing on history. When you just

42:04

read the history book, you're like, ah, this is dull. But

42:06

if you're like, wow, this actually happened to real people

42:09

and I actually know about it. And could you imagine what

42:11

it would have been like to be in that moment

42:14

when Caesar crossed the Rubicon,

42:16

you know, what was going through his mind, then all

42:18

of a sudden history comes alive and you become incredibly motivated.

42:21

I think AI will

42:23

really help unblock folks as a tutor,

42:26

but also really help motivate.

42:28

I mean, imagine having a tutor that not only can do

42:30

what Aristotle did, but it actually can be Aristotle

42:33

sometimes, it can role play with you. It can take on

42:35

tone that really captivates you.

42:37

This all would have been science fiction a year ago

42:40

and it's literally happening now or

42:42

in the coming months.

42:43

It puts to rest the question, if you could have

42:46

dinner with five living or historic people,

42:48

who would they be? Because you

42:50

can actually do

42:51

it at some point.

42:52

You can actually do it. You know, having

42:54

the real dinner will always be better because

42:57

we, you know, obviously the AIs are, they're interpolating

43:00

or extrapolating from, so

43:02

that there's definitely some imperfections. But

43:04

yeah, you're right. In terms of the richness of experience,

43:07

it's even better than the dinner because probably

43:09

in five years you could put some goggles on and

43:12

have the dinner actually in Rome in

43:14

the first century BCE as

43:17

opposed to at your dinner table. It's

43:19

amazing. It's absolutely amazing. Sal

43:22

Khan, thank you so much.

43:22

Thanks for having me, Guy.

43:25

That's Sal Khan, founder and CEO

43:28

of Khan Academy. Hey,

43:31

thanks so much for listening to How I Built This Lab.

43:33

Please make sure to follow the show

43:35

wherever you listen.