0:00

This is the everyday

0:02

AI show the everyday

0:04

podcast where we simplify

0:06

AI and bring its

0:08

power to your fingertips

0:10

Listen daily for practical

0:12

advice to boost your

0:14

career business and everyday

0:16

life When chat GPT first

0:18

came out no one was

0:21

talking about compute, right? But

0:23

over the last few years, as

0:25

generative AI and large language models have

0:28

become more prevalent, the

0:30

concept of GPUs and compute has

0:32

become almost like, you know, dinner

0:34

time conversation, at least if you're,

0:36

you know, crowding around the dinner

0:38

table with a bunch of dorks

0:40

like myself, right? But I think

0:42

even more so the last few

0:44

months, you know, as

0:46

we've seen closed or sorry,

0:48

as we've seen open

0:50

source models really close the

0:52

gap proprietary enclose models,

0:55

I think this concept of compute

0:57

is even more important because now

0:59

all of a sudden you have

1:01

a lot of, you know, probably

1:03

millions of companies throughout the world,

1:05

medium -sized companies that maybe weren't concerned

1:08

or, you know, weren't really paying

1:10

attention to having their own compute

1:12

maybe two years ago. Now, all

1:14

of a sudden, it might be

1:16

a big priority because of the

1:18

new possibilities that very capable large

1:20

language models and even smaller in

1:22

open source models. all these

1:24

capabilities they're giving to so many people.

1:27

So that's what one of the things

1:29

we're going to be talking about today

1:31

and also how distributed computing is unlocking

1:33

affordable AI at scale. All right, I'm

1:35

excited for this conversation. Hope you are

1:37

too. What's going on? Y 'all, my name

1:39

is Jordan Wilson and this is Everyday

1:41

AI. So this is your daily

1:44

live stream podcast and free daily newsletter, helping us

1:46

all not just keep up. with what's happening

1:48

in the world of AI, but how we can

1:50

use it to get ahead to grow our

1:52

companies and our careers. If that's exactly

1:54

what you're doing, you're exactly in

1:56

the right place. It starts here. This

1:58

is where we learn from industry experts.

2:00

We catch up with trends, but then

2:02

the way you leverage this all is by

2:04

going on our website. So go to

2:06

your everyday AI.com. So there you'll sign up

2:08

for our free daily newsletter. We will

2:10

be recapping the main points of today's conversation,

2:13

as well as keeping you up to

2:15

date with all of the other important AI

2:17

news that. for you to be the

2:19

smartest person in AI at your company. All

2:21

right, so enough. Chit chat y 'all

2:23

i'm excited uh for today's conversation if

2:26

you came in here to hear the

2:28

the ai news technically we got a

2:30

pre -recorded one uh debuting it live so

2:32

we are going to have that ai

2:34

news uh in the newsletter so make

2:36

sure you go check that out all

2:38

right cool i'm excited to chat a

2:40

little bit about uh computing and how

2:42

it's changing and making ai affordable at

2:44

scale so uh please help me welcome

2:46

to the show uh we have uh

2:49

tom curry the ceo and co -founder

2:51

of distribute ai tom thank you so

2:53

much for joining the Everyday AI Show.

2:55

Thanks for having me. Appreciate it. Yeah,

2:57

cool. So before we get into this

2:59

conversation, which, hey, for you compute dorks,

3:01

this is right up your alley. But

3:03

for everyone else, Tom, tell us, what

3:05

does Distribute AI do? Yeah,

3:07

so we're a Distribute AI app player. What

3:10

that really means is we're basically going around and

3:12

capturing spare compute. It could be your computer. It could

3:14

be anyone's computer around the world. And

3:16

we're basically leveraging that to create

3:18

more affordable options for consumers,

3:20

you know, businesses, things like that,

3:22

mid -level businesses. And we're

3:24

really, the goal is actually to create kind

3:26

of a more open and accessible AI ecosystem. We

3:29

want a lot more people to be able to

3:31

contribute, be able to leverage kind of the resources that

3:33

we advocate. a pretty cool

3:35

product. Cool. Give us

3:37

an example. Even in

3:39

my hypothetical, I just talked about,

3:41

let's say there's a medium -sized

3:43

business, and maybe they haven't been

3:45

big in the data game, maybe

3:48

they don't have their own servers,

3:50

and they're trying to figure it

3:52

out. What is that problem that

3:54

you all solve? Yes. It's a

3:56

two -sided solution. It's a great example. You

3:58

go to a business and they have, say,

4:00

a bunch of computers sitting around in their offices.

4:02

At night, they can connect into our network

4:04

very quickly. We have a very

4:06

quick one -flick program to install. They can run

4:08

that at night and provide compute to the network.

4:11

And then when they wake up the

4:13

next day and they want to leverage

4:15

some of the AI models that we

4:17

run, they can quickly tap into our

4:19

APIs and basically get access to all

4:21

those models that we run on the

4:24

network. So kind of two -sided, right? You

4:26

can provide on one side and you

4:28

can also use it on the other

4:30

side. Very cool. All right, so let's

4:32

let's get caught up a little bit

4:34

with you know current day because like

4:36

I talked about right I don't think

4:38

you know Compute in GPUs. We're at

4:40

the top of, you know, most people's

4:42

mind, you know, especially when, you know,

4:44

the GPT technology came out in 2020,

4:46

let alone in, you know, late 2022

4:48

when chat GPT was released. So why

4:50

is compute now just like one of

4:52

the leading, I mean, we're talking about

4:54

national security. We're talking about $100 billion

4:57

infrastructure projects. Like why is compute now

4:59

this huge term when it comes to

5:01

just the US economy? March yeah totally

5:03

so I mean five years ago if

5:05

you go back right gaming was the

5:07

biggest use case for GPUs nowadays it's

5:09

all yeah right that's why there's huge

5:11

demand for it these models are getting

5:13

bigger in some cases they're also getting

5:15

smaller chain of thought uses a ton

5:17

of different tokens so although the models

5:19

are smaller they still use a ton

5:21

of resources the reality is is that

5:23

silicon as it stands today one of

5:25

our team members actually works on ships

5:27

a little bit we're basically reaching the

5:30

peak capacity of what we can do

5:32

with chips, right? We're definitely stretching then

5:34

the current technology that we have for

5:36

chips. So although the

5:38

models keep getting better, bigger, larger,

5:40

more compute demand, the reality is

5:42

that the technology is just not able to keep

5:44

up. We're about 10 years out, give

5:46

or take from actually having a new, basically

5:48

a new technology for chips. Sure.

5:51

And, you know, as

5:53

we talk about current

5:55

demand today, right? You

5:58

know, you always see all these, you

6:00

know, jokes online, you know, people are

6:02

like, you know, we'll work for computes,

6:04

right? And the big tech companies, you

6:06

know, open AI, right? Whenever they roll

6:08

out a new feature, you know,

6:10

a lot of times they're like, hey, our

6:12

GPUs are melting. We're going to have to pause

6:14

new user signups. You know,

6:16

why isn't that even the biggest tech

6:18

companies can't keep up with this

6:20

demand? Yeah, I mean it's a

6:22

crazy system where anthropic has the same

6:25

issue right where claw tokens are still

6:27

kind of limited to this degree We're

6:29

running to the point where you're basically

6:31

running you're stretching the power grid then

6:33

you're stretching every Resource that we have

6:35

in the world to run in different

6:37

models at the end of the day,

6:40

you know open open AI I think

6:42

they use primarily NVIDIA for their data

6:44

centers, but once again NVIDIA has demand

6:46

all over the world for these chips,

6:48

so they can't allocate all their resources

6:50

only to OpenAI. So OpenAI has certain

6:52

threshold that they rent from and use,

6:54

but the reality is there's just too

6:57

much demand. You're talking about millions and

6:59

millions of requests, and the

7:01

requests, for example, like image

7:03

generation, these aren't like one -second

7:05

returns, right? You're talking about 10,

7:07

20 -second second returns, and

7:09

video models are even worse. You've

7:11

talked about minutes potentially, even on each

7:13

100 to each 200, so... is,

7:15

like I said, our compute or however

7:17

it cannot possibly keep up the

7:19

man. And we don't have the latest

7:21

genship for not enough. So,

7:24

you know, one thing, and

7:26

you know, you kind of mentioned

7:28

it, I think at the

7:30

same time, we're seeing models become

7:32

exponentially smaller and more powerful,

7:34

right? Like as an example, OpenAI's

7:37

GPT -40 mini.

7:40

yet, then you have

7:42

these monster models like

7:44

GPT -45, right, which is

7:46

reportedly like five to 10 times larger

7:48

than GPT -4, which was I think

7:50

like a two trillion parameter model. So

7:52

what goes through like this, like the

7:54

whole concept of models both getting You

7:57

know technically smaller and more efficient yet

7:59

models also at the same time getting

8:01

bigger And then how does that impact

8:03

right the industry as a whole because

8:05

it seems like it's hard to keep

8:07

up with Yeah on one end it

8:09

kind of reminds me of like cell

8:12

phones back in the day right where

8:14

we would progressively get them smaller and

8:16

then eventually we had a new feature

8:18

to get bigger and then kind of

8:20

get smaller again the reality is is

8:22

that A year ago, larger models, we

8:24

were basically just throwing a million different

8:26

data points into these models, which made

8:29

the models much larger, and they were

8:31

relatively good. But the reality is, is

8:33

that no one wants to run a

8:35

7 billion, you know, a 70 billion,

8:37

700 billion parameter model, right? So we've

8:39

gotten them smaller. They're still now they're

8:41

kind of working with the intricacies of

8:43

how we're actually running these models. So

8:46

chain of thought basically enables you to

8:48

give a better prompt, right? It basically

8:50

takes a human prompt. turns into what

8:52

the system can read better, and

8:54

then gives you a better output. And it also might run through

8:56

a bunch of tokens to give you a better output. So

8:59

change of thoughts are a really

9:01

cool way to basically reduce the model

9:03

size. But the reality is, is

9:05

that, although we're cutting the model size

9:07

so we can put it on

9:09

a smaller chip, the reality is, is

9:11

we're still using a million tokens,

9:13

which doesn't really actually help our compute

9:15

issues. It's kind of bad without

9:17

words, but it's funny. Yeah, it is

9:19

interesting, right? So yeah, even now

9:21

we have these newer hybrid models in

9:23

Claude 3 .7 Sonnet in Gemini 2

9:25

.5 Pro, and you use them, and

9:27

they seem relatively fast. And if

9:29

you don't know any better, you might

9:31

say, OK, this seems sufficient. But

9:34

then if you look at the chain

9:36

of fodder, if you click Show

9:38

Thinking, you're like, my gosh, it just

9:40

spit out. 10 ,000 words to tell

9:42

me, you know, what's the capital

9:44

of Illinois or something like that, right?

9:46

So, you know, as models get

9:48

smaller, you know, this is something I'm

9:50

always interested in, you

9:52

know, might we see a future

9:54

where, you know, that more,

9:56

you know, hybrid models or the, you know,

9:58

reasoning models, will they eventually become less

10:00

efficient? Or is that always going to be

10:02

something, you know, kind of like on

10:04

one side models get smaller, but they're getting

10:07

smarter. And so they're going to have

10:09

to just think more regardless. Yeah,

10:11

that's a good question. I think that

10:13

we'll get to the point where they're

10:15

highly efficient. I mean, the realistically, the

10:17

gains we've made with even deep seek,

10:19

it's just incredible, right? Even their seven

10:21

billion parameter model, which is relatively small.

10:23

You can run on most consumer -grade

10:25

chips. It's extremely

10:27

good. The prompting is great. It

10:30

obviously has a pretty good knowledge base. And

10:32

once you really combine that with the ability to

10:34

surf the internet and actually get more answers

10:36

and use more data, that's where I think we'll

10:39

get to. I wouldn't call it AGI, but

10:41

we're very close to that, where basically you're adding

10:43

in real -time data with the ability to kind

10:45

of reason a lot more. So I do

10:47

think we'll get there. I think the progress that

10:49

we made, although it seems like it's been

10:51

forever since kind of the first models came out,

10:53

The progress was insane and extremely quick. Yeah,

10:57

I'm confident. Yeah. And

10:59

you know, speaking of DeepSeq, I

11:01

know it's been, you know, all the

11:03

rage to talk about DeepSeq over

11:05

the last, you know, the last couple

11:08

of months. But I mean, I

11:10

think you also have to call out

11:12

Google, right? With their Gemma 3

11:14

model, which I believe is a 27

11:16

billion parameter, you know, greatly outperformed

11:18

DeepSeq V3, which is I think 600

11:20

plus billion parameter, at least when

11:22

it comes to Elo scores. And it's

11:24

not even close, right? So what

11:26

does this say about the future? I

11:28

know I kind of named two

11:30

open models there. They're getting

11:32

even the open, right? Everyone's like,

11:34

oh, deep seek is changing the

11:36

industry. Well, I'm like, yo, look

11:38

at Gemma 3 from Google. It

11:40

is 5 % the size and way

11:43

more powerful when it comes to human

11:45

preference, right? So what does this even

11:47

mean for the future of edge computing?

11:49

And how does edge computing impact compute

11:51

need or GPU demand? Yeah, well we

11:53

we started this business the reality was

11:55

is that although we wanted to convince

11:57

ourselves that open source models were good

12:00

We were based in violence open source

12:02

models were relatively bad You know open

12:04

AI was extremely dominant at that time

12:06

It was it was like you couldn't

12:08

even believe that anyone would ever catch

12:10

up to open AI nowadays We're probably

12:12

running at like a one to two

12:15

month lag between parody of private source,

12:17

you know, and open source model, which

12:19

is really interesting. And when you tie

12:21

that in with the idea of kind

12:23

of data privacy and things like that,

12:25

I think there is a huge argument

12:27

for basically edge compute taking over a

12:29

lot of the smaller daily tasks and

12:32

then reserving some of the more private

12:34

models and things like that and the

12:36

larger models for things that might be

12:38

a little bit more deeper like research

12:40

and things like that. But a lot

12:42

of things that you do on a

12:44

daily basis that AI can actually improve, I

12:47

think you can run purely on edge compute.

12:49

and basically have your house and your couple computers

12:51

and things like that, maybe your laptop or

12:53

iPad, basically turning through this little tiny data center

12:55

that allows you to run whatever model you

12:58

want to run at that time. We're just really

13:00

far away. The reality is you can do

13:02

that today, right? We could probably be able to

13:04

that in a week. The

13:06

one problem is that getting it from

13:08

teaching people to basically use that and set

13:10

it up, right? It takes time for

13:12

people to learn how to, oh, install your

13:14

own model and start running things. So

13:16

it's more of like the UX more than

13:18

anything. Yeah, you know, and

13:20

that, you know, I always think

13:23

right, I always think with these

13:25

models. becoming smaller,

13:27

more capable, you

13:29

know, is, will most things

13:31

be edge in the future,

13:33

right? Like, you know, I

13:36

even saw the, you know,

13:38

NVIDIA GTX, right? Formerly called

13:40

digits, you know, I did

13:42

the math on that. I'm like,

13:44

that would have cost five years ago.

13:46

I think like $70 ,000. It

13:48

wasn't even capable to do it

13:51

in any ways, right? Like, are we

13:53

going to have the average, you

13:55

know, smartphone? in five years will they

13:57

be able to run state -of -the -art,

13:59

large language model? And if so,

14:01

like, how does that change the whole

14:03

cloud computing conversation? It will

14:05

be really interesting. I think you're 100 % right.

14:07

And I think five years might even be

14:09

a stretch. I think what will come down to,

14:11

like I said, is privacy. If people are

14:13

really worried about their privacy, then I think that

14:16

people will push for edge compute to be

14:18

running and you'll be able to run your own

14:20

model that only uses access to your own

14:22

data on your phone, device, whatever it is, right?

14:24

If people don't care about that as much, it

14:27

might take a little bit longer just because

14:29

people won't build that. But I really do think

14:31

there are some teams that are building in

14:33

that angle where essentially you're going to have

14:35

your little database of information about yourself and

14:37

your life and your wife and whatever else. And

14:39

essentially you'll be able to run all that

14:41

stuff without ever touching any centralized model. For

14:44

obvious reasons, privacy reasons, things like that.

14:46

We already give so much data to

14:48

the big tech, right? I

14:50

think we're good on giving you any more and

14:52

sharing any more intimate details about our lives. It'll

14:55

be a good thing if we can

14:57

do that. Yeah. And, you know, even

14:59

as we start looking, you know, at

15:01

this race, which, you know, if you

15:03

looked at it two years ago, you

15:05

know. I don't know if anyone, even

15:07

the staunchest, you know, open source believers

15:09

would believe that we're at the point

15:11

that we are now, but, you know,

15:13

between whatever we're going to see from

15:15

Metta in their next Lama model, I've

15:18

already talked, you know, we've already talked

15:20

about DeepSeq and, you know, Gemma as

15:22

well and, you know, OpenAI also has

15:24

recently said that they're going to be

15:26

releasing an open model. Yeah,

15:29

yeah, yeah. We'll see what

15:31

happens. We don't buy any of

15:33

that. Yeah, I remember the

15:35

GPT -2 opened in Fiasco, right? But

15:39

regardless, I mean, what happens

15:41

when and if... models are more

15:43

powerful than closed in proprietary

15:45

models. So number one, what happens

15:47

from, you know, kind of

15:49

a, you know, GPU and compute

15:51

perspective, but then how does

15:53

that change, you know, the business

15:55

leader's mindset as well? Yeah.

15:57

So at that point, once things

15:59

become commoditized, right, and the

16:01

models are essentially all on the

16:03

same level, give or take

16:05

a little bit of change between

16:08

their variation. The reality is,

16:10

is that compute becomes the last denominator.

16:12

Oh, basically being able to offer those models

16:14

at the cheapest cost, right? So at

16:16

that point, it basically comes down a race

16:18

to the bottom in terms of who

16:20

can get the cheapest compute and offer to

16:22

people with the best selection of bottles

16:24

and UX and UI all kinds of internet,

16:26

right marketing, things like that. Assuming

16:28

that that does happen, the

16:30

question then comes down to what

16:32

happens only to private source companies, right?

16:35

Which my personal view on it is,

16:37

is that there is probably a

16:39

world where essentially open AI and anthropic

16:41

eventually Burn so much money, which

16:43

they lose money every day already that

16:45

they don't get to the point

16:47

where they're looking to get to. And

16:49

essentially, they have to just either

16:51

change business models or run out of

16:54

money, right? I think that's probably

16:56

a little bit of a point, a

16:58

contentious point. But the reality is

17:00

that right now, we're running models that

17:02

are very close to as good

17:04

as what they have. And it's like,

17:06

at what point does the marginal

17:08

gain isn't worth it, right? When

17:11

H100s become a lot cheaper, we'll be able

17:13

to run some of the biggest models very

17:15

quickly and easy, and the access will just

17:17

be so good that it might not matter.

17:20

The problem is, is that I mean,

17:22

I personally do, I've always believed in private

17:24

source. I do believe that there's

17:27

great use cases for it. And the reality

17:29

is, it's like, whether you love Sam Holtman

17:31

or hate Sam Holtman, he's pushed things forward

17:33

a lot, Greg. He's been really productive for

17:35

the entire environment. So you don't want them

17:37

to go bankrupt, I don't think. They might

17:39

just have to figure out a way to

17:41

appeal to consumers or businesses in a different

17:43

way as opposed to just general models, which

17:45

is what they do right now. I think

17:47

they're in a great way. They talk about

17:49

a series and things like that. They'll probably

17:51

figure out ways to tie into the rule. Yeah,

17:54

so so speaking of you

17:56

know, affordable AI and you just

17:58

brought up as well, you

18:00

know companies like an open

18:02

AI and anthropic right there burning

18:04

of cash is well documented

18:07

You know, but I mean does

18:09

this at a certain point

18:11

if large language models become

18:13

commoditized because of open source models

18:15

Is it just more of

18:17

the kind of the application layer

18:20

that becomes the thing? You

18:22

know these companies real differentiate right?

18:24

Because aside from, you know,

18:26

OpenAI is $200 a month,

18:28

you know, pro subscription, it's like,

18:30

okay, which they also said they're

18:32

losing money on. Like aside

18:34

from that, you know, how else

18:36

are these big companies that so

18:39

many people rely on going to

18:41

continue to exist five, 10 years

18:43

after their, you know, $40 billion

18:45

of funding, you know, might run

18:47

out if they're not at some

18:49

point? We've been saying about this

18:51

about Uber for how many years now though,

18:53

to be fair, these companies can exist a

18:55

long time without being profit. But

18:57

reality, I think the reality is,

18:59

is that the one thing that the

19:01

centralized type of providers offer, like

19:03

OpenAI, is that they're able to

19:05

work with a lot of data that

19:07

would be very sensitive, primarily like health data

19:09

and things like that. So I'm sure

19:11

there's a lot of very good business use

19:14

cases that they can provide to very

19:16

large enterprise consumers, or not

19:18

consumers, businesses. And I

19:20

don't really know what those are outside of

19:22

like health and things like that, that

19:24

data that's very private, you know, government contracts

19:26

and things like that. Those models are

19:28

super useful for that. But

19:30

it will be tough. I mean, it would really, I

19:33

mean, I, I feel like we're almost there already,

19:35

to be honest with you. Like I said, I

19:37

don't think we're that far away from the point

19:40

where people are like, why don't let me just

19:42

cancel open AI and you don't use long, like

19:44

let me go cancel and use jump, you know,

19:46

with all these different models that. are out there.

19:48

There's so many good ones at this point. But

19:51

it might be more integrations. It might be more,

19:53

like I said, UI, UX. It

19:56

might be the fact that at the end

19:58

the day, we use iPhones every day and

20:00

Android. And maybe they just put a true

20:02

monopoly on being able to use them. So

20:05

we'll see. Yeah, it's interesting.

20:09

We've covered a lot in

20:11

today's conversation, Tom, when this

20:13

concept of distributed computing and

20:16

how the race between open

20:18

source AI and closed AI

20:20

is really changing the compute

20:22

landscape and just the AI

20:24

landscape as a whole. But

20:26

as we wrap up today's

20:28

show, what's the one most

20:30

important or the best piece

20:32

of advice that you have

20:34

for business leaders when it

20:36

comes to making decisions? about

20:39

how they are using

20:41

AI at scale. Yeah,

20:43

that's a great question. I think the

20:45

best advice, the thing that we've learned

20:47

the most from our personal business that

20:49

I can provide is that the landscape

20:51

changes so fast. The last thing

20:53

you can do is lock yourself into

20:55

one specific provider or model. Don't allocate

20:57

too many resources and sell the house

20:59

on one specific setup because the next

21:01

week something comes out and totally breaks

21:04

everything before it, right? So make sure

21:06

you're open. Make sure you're flexible on

21:08

what you're using and how you're using

21:10

it and be ready for someone to

21:12

come out and completely break the mold

21:14

and change the direction of everything. It's

21:16

such a fast -paced environment. It's really

21:18

hard to keep up. And

21:21

you know, I think we're just kind of

21:23

still scratching the surface on where AI will

21:25

actually integrate. All

21:27

right. Exciting conversation that I think

21:29

a lot of people are going to find

21:31

valuable. So, Tom, thank you so much for

21:33

sharing your time and coming on the Everyday

21:35

AI Show. We appreciate it. Thank you so

21:37

much for having us. We really appreciate it.

21:39

All right. And hey, as a reminder, y 'all,

21:41

if you missed something in there, you know,

21:43

a lot of big terms where we're tossing

21:45

around and getting a little geeky on the

21:47

GPU side, don't worry. We're going to be

21:49

recapping it all in our free daily newsletter.

21:51

So if you want to know more about

21:54

what we just talked about, make sure you

21:56

go to your everydayai.com, sign up for the

21:58

free daily newsletter. And you for tuning in.

22:00

We hope to see you back tomorrow and

22:02

day for more Everyday AI. Thanks y 'all. Thank

22:04

you. And that's

22:06

a wrap for today's edition of Everyday

22:09

AI. Thanks for joining us. If

22:11

you enjoyed this episode, please subscribe and

22:13

leave us a rating. It helps

22:15

keep us going. For a little

22:17

more AI visit your everydayai.com and

22:19

sign up to our daily newsletter

22:21

so you don't get left behind.

22:24

Go break some barriers and we'll see you next time.