0:01

Reuniti. Here's to the world's favorite Lambrusco

0:03

and one of the most beloved Italian

0:05

wines worldwide. Here's to its chill, delightful

0:08

fizz that springs to life as thousands

0:10

of tiny bubbles dance in your mouth.

0:12

Here's to the refreshing taste of Reuniti.

0:15

After a long hard day it's the perfect

0:17

pour you deserve with a company you enjoy.

0:19

Reuniti Lambrusco. The refreshing, lively and easy to

0:21

drink Italian wine. Reuniti on Ice Still Nice

0:23

imported by Frederick Waldman and Sons Limited, New

0:25

York and New York campaign finance according to

0:28

EU Rec 2021-2115. It's

0:32

time to get the world talking about black

0:34

lead brands. We all have our favorites but

0:36

we can't keep them all to ourselves. So

0:38

if you're feeling a little black opal beauty,

0:40

tell somebody. If the lip bar gives you

0:42

a lip for every drip, let them know.

0:44

And if your hair is doing the

0:46

do, shout that out too. Join

0:48

Walmart in shouting out your favorite

0:50

black lead products creating a new

0:53

world of choices at walmart.com/black and

0:55

unlimited. Grammar

1:03

Girl here. I'm Mignon Fogarty and today

1:05

we're going to talk with Christopher Penn

1:07

about some of the common misconceptions around

1:09

AI tools like chat GPT that I've

1:12

seen among writers and editors and just

1:14

people in general. Whether you

1:16

love it or despise it, I

1:19

think it's really important to understand

1:21

AI and Chris is

1:23

a guy who can really help us do that. Christopher

1:25

Penn, welcome to the Grammar Girl podcast. Thank

1:27

you so much. It is nice to

1:30

be on one of the OG podcasts.

1:33

You and I have been podcasting I think since what,

1:35

05? You were 05, I think I was 06. But

1:37

yeah, it's been a while.

1:39

Yeah, and

1:42

so actually, you know, because I've been

1:44

subscribed to your newsletter for so long,

1:47

one of the things I've noticed about you

1:49

is that you are always on the cutting

1:51

edge of trends and data. I mean, you

1:53

called the housing crisis in 2007. I remember,

1:57

you know, I was looking at buying a house and you. actually

2:00

said to me, Mignon, I think maybe you should

2:02

wait. And I

2:04

did not take your advice as one

2:06

of the worst financial situations I've ever

2:08

made. But we loved the house. It

2:11

turned out okay. But you were definitely

2:13

right. And then in

2:15

2019, 2020, again, you

2:17

predicted how bad the pandemic was going to

2:19

be, but way before a lot of other

2:21

people realized it. And

2:23

so about, I guess it was about a year,

2:25

a year and a half ago, I noticed your

2:29

almost timely newsletter became

2:31

entirely about AI. And

2:34

I would love to hear more about

2:36

how that came about, like whether it

2:38

was a gradual thing or did you

2:40

just have an aha moment where you

2:42

were like, okay, this needs to be

2:44

everything. So AI has existed in

2:46

some form since the 1950s. I

2:49

started to have an interest in in 2013. And

2:53

there's three branches of AI. There's what's

2:55

called regressive classification and generation. One

2:58

is needle in a haystack. Hey, here's a bunch of data.

3:00

And here's an outcome. What in the data

3:03

is like this outcome, right? We call this

3:05

finding AI. There's classification,

3:07

here's a bunch of data, let's organize it. Like

3:09

what do we got in here? You

3:11

see this with particularly people like marketers trying to

3:14

organize things like social media posts, like there's so

3:16

many, how do we organize and classify them. So

3:18

that's the central mind. And then

3:20

there's this new branch started in 2017, called

3:23

generative, based on an architecture and

3:25

a paper created by Google called

3:28

attention is all you need. And the architecture

3:30

is called transformers. That

3:32

became a product family

3:35

really around 2020. Just

3:38

as the pandemic got rolling, when companies

3:40

started to say, maybe we can get

3:42

this architecture do something interesting, like predict

3:44

the next word in a sequence in

3:46

a way that's never been done before.

3:48

By 2021, they were usable models that

3:50

would write coherent text, not actually correct,

3:53

but at least coherent is readable. Prior

3:55

to that, it looked like you're rolling

3:57

your face on a keyword. when

4:00

I started paying attention to this technology,

4:02

I started using OpenAI's GPT-3, which had

4:05

come out 2021. And then November 2022, OpenAI

4:07

releases a tool called Chat GPT. And suddenly,

4:13

everyone's an AI expert. And

4:16

having that background of more than

4:18

a decade working in the core

4:21

technologies, I could see where this is

4:23

going because Chat GPT changed people's relationship

4:26

with AI. Prior to that, you

4:28

needed coding skill, right? You needed the

4:30

ability to work in these models

4:32

and stuff like that and build your own models.

4:34

And then suddenly, there's a web interface you can

4:37

chat with just like you were drunk texting with

4:39

your Uncle Fred. And suddenly, okay,

4:41

wow, people can make use of this. And

4:43

it was that point where I started pivoting

4:45

a lot of my content to say, okay, here's

4:48

what the stuff does. And here's

4:50

how we should be thinking about it.

4:52

And of course, the rest of the world pivoted as well,

4:55

as people start to understand the implications of these

4:57

tools. But now today, these

4:59

tools have accelerated faster than

5:01

I've ever seen any technology

5:03

evolve. It is absolutely

5:06

breathtaking how quickly they've evolved.

5:08

And I'll give you an example. In

5:11

the first release of Chat GPT,

5:13

that was the backend model is

5:15

this obscure, poorly named bottle called

5:17

GPT 3.5 turbo that

5:19

required an entire server room of

5:22

machines to run and serve up

5:24

and send your text. About

5:27

a month ago, Meta, the parent company of

5:29

Facebook, released a model called Llama 3.1. That

5:33

version, there's a version you can run that

5:35

is more capable,

5:38

more capable than GPT 3.5, about as capable

5:41

as the successor GPT 4, and

5:43

runs on your laptop with no

5:45

internet connection. Hey,

5:51

I'm just jumping in here to say that

5:53

we recorded this episode about a month ago.

5:55

And since then, Meta has released an even

5:57

newer model called Llama 3.2. So,

6:03

we've gone from an immature

6:05

technology that is extremely expensive to

6:07

use to a mature technology that

6:10

you can run, I've used

6:12

it on a plane with no internet

6:14

and to have generative AI capabilities all

6:17

the time in less than two

6:19

years. That is crazy fast evolution.

6:22

Yeah, I agree. It's amazing. I

6:24

haven't seen anything like it since the introduction of the

6:26

internet. Yeah, and if people looked into

6:28

it six months ago or eight months ago and

6:30

gave it a try, it's

6:33

different today. It's so much

6:35

better. There are so many misconceptions. One

6:39

of the things someone asked me a

6:41

few weeks ago was, well, does chat

6:43

GPT use AP style or Chicago style?

6:47

I remember having that question when I first started

6:49

playing around with it too. It's

6:51

a reasonable thing for an editor or writer to ask,

6:53

but it's not how it works.

6:57

Can you maybe start by explaining why that's

6:59

not the right question to ask and how

7:01

it works at a basic

7:03

level for people who haven't looked into

7:05

it as deeply as you have or

7:07

maybe I have? Sure. The

7:10

way these tools work is

7:12

they understand pieces of words

7:14

and how those pieces of words relate to

7:16

every other word around them in the sentence,

7:19

in the paragraph, in the document itself. They

7:22

are trained on such huge quantities of text.

7:24

To give you an idea of what the

7:26

average model is trained on, it would be

7:29

a bookshelf that wraps around the equator twice.

7:32

That's how much raw text is needed

7:34

to train one of these models. What

7:36

they get used to seeing is understanding

7:38

the statistical distribution of language. Implicit

7:41

in that is how language

7:44

works. If I

7:46

say I pledge allegiance to the,

7:48

the next word probably is flag.

7:51

It probably is not rutabaga. As

7:55

a result, the models understand the

7:57

likelihood of everything that is in the

7:59

sentence. we say. If I say

8:01

God save the depending on your

8:03

nationality and your knowledge

8:05

base, you might say that the queen or king,

8:08

but again, probably not Rudabaga. When

8:10

you ask about something like Chicago style

8:12

or AP style or any writing style,

8:14

you're asking about a specific format. And

8:17

these models have seen all that and then some

8:19

what they spit out, by definition

8:21

is probability based text. So

8:24

if you give it a

8:26

very short, naive, terrible prompt,

8:28

like write me a blog

8:31

post about B2B marketing, it's

8:33

going to give you the most probable

8:36

words, phrases, concepts to that in

8:38

a written format that is

8:40

probably going to be technically correct, probably

8:43

going to be boring, probably going to

8:45

be completely unoriginal, because it

8:48

is invoking what

8:50

its best guesses are probabilities for those words.

8:52

If you think of when we talk to

8:54

these tools, we are prompting them, we're writing

8:56

to them like chatting, etc. Every

8:59

word that we type in our prompt

9:01

kind of has an invisible word cloud

9:03

around it. And where

9:06

those word clouds intersect is

9:09

how it knows what to respond with, right?

9:11

So these word clouds are huge, and they

9:13

they they narrow down. So if you write

9:16

a prompt like write me a short fiction

9:18

story about two women in love,

9:22

you're going to get direct, right?

9:24

Because it's so generic. But if you

9:26

say write me a short story about

9:28

two women in love set during the

9:31

Victorian period, but set in Wales instead

9:33

of London, and the one

9:35

woman comes from originally from Scotland, the

9:37

other woman comes from France, originally, you

9:40

see all these extra words and all

9:42

these word clouds start overlapping, and they

9:44

get more and more precise, and

9:46

you get more and more text that is

9:48

unique and different, because you're getting more specific.

9:51

The one thing I say in all the

9:53

keynote talks that I do is if you

9:55

remember nothing else, the more

9:57

relevant specific words you use we

10:00

prompt these tools the better the output will be

10:02

if you give it i say

10:04

think of these things like the world's

10:07

smartest interns brand new intern comes in

10:09

the office this internet 255 phds got

10:12

a phd and everything but it's still day

10:14

one they still don't know who you are they

10:17

don't know where the restroom is they don't know the coffee machine

10:19

is and so you gotta teach you

10:21

to say to enter hey intern go write me a

10:24

book. You're gonna get crap out

10:26

of human intern just like you're gonna get crap

10:28

out of a machine if you on the other

10:30

hand give the engine here's our style guide your

10:32

brand safety guidelines here's the background information here's the

10:34

outline i want you to use i

10:37

want you to use a style. Can

10:39

you give the internet a big ol' pile of directions

10:41

and that is going to come up with a much

10:43

better result. Right yeah i

10:45

feel like we shouldn't go past the

10:48

idea of training unlike the entire world's

10:50

libraries without mentioning the concerns about copyright

10:52

infringement in the lawsuits that are currently

10:54

happening so that's like a whole other

10:56

podcast that's a big issue in the

10:58

field so if you weren't aware of

11:01

that i want you to know we're

11:03

not gonna focus on it very much

11:05

today. We're not

11:07

going to focus on it but there's

11:09

an interesting distinction and this is what

11:11

the law will hinge on and it

11:14

depends on where you are so in

11:16

the EU the EU has ruled that

11:18

the use of someone else's intellectual property

11:20

for creating models in fringes on their

11:22

their rights. In Asia

11:25

China and Japan have rolled that what

11:27

is inside of all of the pull

11:29

it open is a big pile of

11:31

numbers in no way resembles the original

11:33

works and therefore. Model

11:35

does not infringe on someone's rights because

11:38

it can no way could be mistaken

11:40

for the original and so

11:42

the question is gonna

11:44

be resolved in every jurisdiction separately

11:46

as to whether a what

11:49

i does infringing on the originals or

11:51

not. And it's not clear how that's

11:53

gonna play out. Oh thank you that's

11:55

fascinating i wasn't aware that there were

11:57

country differences at this point that's super

11:59

interesting. Does that mean people in

12:01

the EU can't use these models right now? There

12:03

are some tools they are not permitted to use

12:05

that are not available in the EU. Wow.

12:07

Okay. Okay. I didn't know that. So

12:11

another concern that I see... Oh, and I

12:13

just want to ask, so would it be

12:15

fair to say that, you know, back to

12:17

the AP Chicago issue and thinking about the

12:19

way it works, would it be fair to

12:21

say that if you asked it to write

12:23

a news article for you, that it would

12:25

be very likely to follow AP style because

12:27

it is looking for words

12:29

from that style of writing that originally

12:31

would have been written in that style

12:33

or is that not? It's probable that

12:35

it will be invoked something that looks

12:37

more like AP style than Chicago style,

12:39

as opposed to say writing a nonfiction

12:42

book, which is probably going to be

12:44

more Chicago style than AP style. Again,

12:47

with these tools, if you tell it, I want this

12:49

in AP style, it will write it in AP style.

12:51

If you let it do its own devices, it will

12:53

do the highest probability for the data set that you're

12:55

working with. One of the things

12:57

people knock these tools all about is saying, oh,

12:59

they use the exact same words. In a world

13:02

of whatever, and then, you know, the word delve

13:04

and all this stuff. Well, that's within that context.

13:06

If you say, write some sapphic

13:09

fiction, the word delve is not

13:11

going to appear because in the training data it's

13:13

seen, sapphic fiction

13:15

writers don't use that word, right? So

13:17

it's not going to be within that

13:19

set. So part of prompting is understanding

13:21

what words we want to invoke and

13:24

what range of addiction

13:26

we want to invoke that is

13:29

very often not something that human

13:31

writers think about. Great.

13:34

So another misconception that I hear, I

13:36

hear a lot of people who are

13:39

concerned about privacy and confidentiality, that they

13:41

don't want to upload their information because

13:43

they don't want it to be used

13:45

to train data or they have clients

13:47

that don't want their proprietary information

13:50

getting into these models. And I think that

13:52

sometimes it's the case that you wouldn't want

13:55

to do that, but I don't think it's

13:57

always the case. And how can people carefully

14:00

can deal with these issues and still carefully

14:02

use AI if they want to.

14:04

Here is the golden rule. If you ain't paying,

14:06

you're the product, right? It's been that way for

14:08

social media. It's been that way for everything. And

14:10

the same is true with AI. If you ain't

14:13

paying, your data is the product and that is

14:15

being used to train other people's models and things.

14:18

Now, there's two different branches of

14:20

thought on this. And here's

14:22

one that's, you know, the obvious one is pay

14:25

for tools, and then make sure that the privacy

14:27

settings and those tools reflects what you expect. If

14:29

you're using, for example, the enterprise edition of any

14:31

tool in the contract, it says we will not

14:33

use your data to train on and stuff. And

14:35

so if you are working with sensitive information, you

14:38

absolutely should be doing that. If there is some

14:40

data, you absolutely positively just cannot give to a

14:42

third party. And there are certain versions

14:44

of AI, like metas, llama, 3.1, four or five

14:47

the model, you can run that

14:49

on your own servers inside your own

14:52

company. And that is totally under your

14:54

IT department's control. It is reassuringly expensive

14:56

to do that. You spend tens of thousands

14:59

of dollars on hardware. But if you are

15:01

say a three letter agency based in Langley,

15:03

Virginia, then that's a

15:05

pretty safe investment, you will have all

15:08

the benefits of generative AI, but you

15:10

will know your data will never ever

15:12

leave the protective compounds of your compound.

15:15

So that's one aspect. The second

15:17

aspect, when it comes to what

15:19

goes on with your data

15:21

is knowing the level of

15:23

sensitivity of it, right? So if

15:26

your company or your work, whatever is

15:28

bound by guidelines, like ISO 27001, or

15:30

SOC 2 compliance stuff, you

15:34

know what you're supposed to be doing with your data

15:36

of any kind, you know what's allowed and not allowed.

15:38

So you can just look at here's

15:40

the here's the requirement in general that my system's

15:42

first have like HIPAA compliance for healthcare. Look

15:45

at chat GPT, not HIPAA but okay,

15:47

clearly then I can't use

15:49

HIPAA data in a non HIPAA system.

15:51

I mean, that's, that's kind of a

15:53

no brainer. So think about the regulations

15:55

you already have to adhere to and

15:57

how do and what systems are qualified.

16:00

for those regulations. Right. Well, what if you're just

16:02

a lowly person like me who maybe has a

16:04

$20 a month chat GPT or

16:06

Claude subscription or something like that? Do you

16:08

feel like that is, and I think they

16:10

say that they won't use your data for

16:12

training, but also it's pretty cheap. So am

16:15

I putting my stuff out there at risk?

16:17

If you are putting, if you're

16:20

setting, if you're confirmed to the setting say

16:22

they will not train on your data, then

16:26

the basis of contract law is that they will not train

16:28

any data. If it turns out that they do that, then

16:30

you get to join the class action lawsuit and sue them

16:32

for a whole bunch of money if

16:34

it comes out that they were not adhering to those

16:36

terms. And again, this comes down to sensitivity. So if

16:38

you are writing a book and it

16:40

turns out that they in fact were training on your data

16:42

but not, yeah, you should

16:45

join lawsuit. If you are processing

16:47

case notes from a field agent and that

16:49

leaks and that gets that agent killed, you

16:52

should not be using chat GPT period.

16:55

No license, no matter that you should be running

16:57

that internally, you know, in a protected

16:59

system because that is literally life or death. And

17:02

I would say the same is true for all of these

17:05

different use cases is how, what

17:08

are the consequences of something goes wrong? That's a

17:10

question people don't ask often enough

17:12

is what could go wrong asking it

17:14

unironically? Yeah.

17:18

So let's talk a little bit

17:20

about hallucinations and fact checking. So,

17:23

you know, one of the things

17:25

I see the most people doing

17:27

actually is using chat GPT, Claude,

17:29

perplexity as a search engine, as

17:31

a replacement for Google essentially. And

17:34

yet we know that they make

17:36

mistakes. So how do you approach

17:38

thinking about using these tools for

17:40

search and what advice do you

17:42

have for people who are doing

17:44

that? So a hallucination is

17:47

a probability error. If

17:49

you think of these tools like

17:52

a library, like a model is like a library

17:54

and there's a library and you ask the library

17:56

and questions, depending on the obscurity.

17:58

of the question, the librarian may or may

18:00

not be able to find a book that's

18:02

appropriate to your question. So if you walked

18:04

into a library and it was pretend it's

18:06

obscure, you walk and say, hey, I'd like

18:08

the joy of cooking. And the

18:10

librarian wanders around a bit and comes back with the joy of sex.

18:13

And he's like, this is close, right? Like,

18:16

no, that's not close. Not at all. But

18:19

semantically, it thinks it's close

18:21

enough. That's what's happening in

18:23

a hallucination is the models

18:26

have three basic directives, be

18:28

helpful, be harmless, be truthful.

18:31

And it's a delicate balancing act, but they

18:33

try to be as helpful as possible at

18:36

sometimes at the exclusion of being truthful,

18:38

right? So they will say, hey, you want this thing? I'm going

18:40

to try my harsh to give you this thing. And it gives

18:43

you this thing is that that's the thing you're like, yeah, but

18:45

that's not right. It

18:47

was right. And

18:50

so using these tools as

18:52

search engines is probably not the best use

18:54

case for a language model self. If

18:57

you dig into the raw technology stuff, you

18:59

got to get super nerdy. All

19:03

models hallucinate 100% of the time when they're

19:05

first built. It requires a

19:07

lot of tuning to even just stop get

19:09

them from lying and period, because they're always

19:11

assembling just probabilities. So there's a lot of

19:13

extra steps to go into refining a model

19:16

to make it more truthful. And

19:18

then there's additional safeguards built on top

19:20

of it. Now what's happening in the

19:22

field is that we're diverging into people,

19:24

these model makers recognize the validity of

19:26

the use case of most

19:29

internet experiences suck, right? Someone's website like I

19:31

want to go check out a recipe. I

19:33

got a way through 42 pages of your

19:35

grandmother's second cousins, roommates, dogs, you know, why

19:38

they like this recipe. I don't care. Tell

19:40

me how many cups of sugar to use.

19:43

And so perplexity and Google and chat GPT

19:45

and all these companies said, you know what?

19:48

That experience sucks. So instead, we're

19:50

going to generate the answer for you that

19:52

says you need two cups of sugar for

19:54

this recipe. The consumer sees

19:56

that as a much better experience. And

19:58

so we now have tools like complexity,

20:01

we have Google's AI overviews, we

20:03

are going to shortly have

20:05

open AI search GPT that

20:07

use search data as the

20:09

basis of their responses. And

20:11

then they craft responses from that data,

20:14

and they still do get it wrong.

20:16

Perplexity, in particular, will ingest

20:18

the web page content and

20:21

summarize it down. So it's more accurate than

20:23

just making it up from the model's long

20:26

term knowledge, but it's still sometimes wrong. Google

20:28

famously, when search standard experiments became AI overviews,

20:30

had the case of well, hey, how do

20:33

you make pizza, you know, cheese, you know,

20:35

stay on pizza, add a quarter cup of

20:37

glue to the pizza. That

20:39

is, that was from a Reddit forum post,

20:42

that someone said that ingest and Google just

20:44

does not have a sense of humor. And

20:47

therefore, I was unable to detect that. So for

20:49

consumers, there are some tools

20:52

which are going to likely lead to a higher

20:54

probability of success, like search GPT,

20:57

like perplexity, like Google AI overviews,

20:59

because they are rooted in search

21:01

results. Other tools like

21:03

chat GPT, or straight you know, regular

21:05

consumer Gemini, or regular Claude, you should

21:07

not use those as search engines, because

21:09

they're not pulling from at least

21:12

known sources on the web. However,

21:15

they are still subject to the

21:17

vagaries and the flaws of search

21:19

itself, right, you can find in

21:22

a search engine, absolute garbage, completely

21:24

incorrect information. And whether using

21:26

generative AI or traditional search, you will

21:28

still get the answer to what you were

21:30

looking for, even the answer is completely wrong.

21:33

Right. So if especially if you're searching for

21:35

something important, it's critical that you actually check

21:37

somewhere else that the facts so if they

21:39

say you're using perplexity, it shows

21:42

you the sources where you got the information,

21:44

click through on those sources and

21:46

double check what you've got is right and

21:48

not somehow taken out of context on that

21:50

page or or something like that. Yeah, and

21:53

and know the difference between sources. For example,

21:55

we had to we had to postpone this

21:57

interview, this interview was supposed to happen last

21:59

week. I was in the hospital getting

22:01

emergency surgery. As

22:04

part of recovery, I wanted to figure out what

22:06

I could do to accelerate recovery. So I went

22:08

to perplexity. I said, you know, find me peer-reviewed

22:10

research on post-surgical recovery

22:13

methods. And it points

22:15

out these different papers. I went and downloaded

22:17

the papers themselves after checking which journals they

22:19

were in. Nature, Cell, Science,

22:22

NIH, PubMed. I know these to

22:24

be credible journals, as opposed to

22:26

Bob's Random Journal of whatever.

22:30

And then stuck it into a system like Notebook

22:32

LM, which is a product by Google specifically made

22:34

for academic research, where you give it the data

22:37

and ask questions of the data. And

22:39

if you didn't provide the answer, it will say, I don't

22:41

have the answer. It will not try to make it up.

22:44

And I was able to design a post-surgical

22:46

recovery regimen that's working about 40% faster than

22:49

the normal because I was

22:51

able to go get the data. But I had to

22:53

be the informed consumer to do that. The tools don't

22:55

do it for you. Yeah, that's great.

22:57

I'm glad you're getting better faster. And yeah,

22:59

I do the same thing. I go to

23:01

Perplexity when I have medical questions like that.

23:03

But I always download the journal article and

23:05

read it. I used to be a scientist

23:07

and a tech writer, so I'm comfortable reading

23:09

journals. And 90% of the time, it's great.

23:13

But I have found errors occasionally,

23:16

things that Perplexity didn't quite present

23:18

properly in the summary. Reuniti.

23:26

Here's to the world's favorite Lambrusco and

23:28

one of the most beloved Italian wines

23:30

worldwide. Here's to its chilled, delightful

23:32

fizz that springs to life as thousands of

23:34

tiny bubbles dance in your mouth. Here's

23:37

to the refreshing taste of Reuniti. After

23:40

a long, hard day, it's the perfect pour you deserve

23:42

with the company you enjoy. Reuniti Lambrusco,

23:44

the refreshing, lively, and easy to drink Italian

23:46

wine. Reuniti on Ice, still nice. Imported by

23:48

Frederick Waldman and Sons Limited, New York, New

23:50

York. Campaign finance according to EU Rec 2021-2115.

24:00

family, durable and ready to carry

24:02

all your stock and you quit

24:04

for adventure. Honda's here with the

24:06

2025 Honda CR-V. CR-V outperforms the

24:08

pack with more style, more space

24:11

and more power. Complete with state

24:13

of the art safety features and

24:15

a five star safety rating. Honda's

24:17

here for you with the 2025

24:21

Honda CR-V. Check it out at

24:23

your local Honda dealer today. I've

24:27

been working on learning Spanish for

24:29

a long time and the thing

24:31

I struggle with most is speaking

24:33

and pronunciation. It feels awkward

24:35

and I find it hard to even tell

24:38

if I'm getting it right. But

24:40

Rosetta Stone is really helpful with

24:42

pronunciation. It has lessons where

24:44

you actually sound at words syllable

24:46

by syllable and it generates what's

24:49

almost like a score bar so

24:51

you can quickly see how close

24:53

you're getting. It's instant feedback and

24:56

it's not surprising that it's great because

24:58

Rosetta Stone has been an expert in

25:00

language learning for 30 years

25:02

with millions of users and

25:04

it's all about fast language

25:06

acquisition. Better yet the

25:08

program is designed for long-term retention and

25:11

you can learn on your desktop or on

25:13

the go with the Rosetta Stone mobile app.

25:16

Check out the lifetime membership

25:18

option which gives you access

25:20

to 25 languages including French,

25:22

Korean and Dutch. Don't

25:24

put off learning that language. There is no

25:26

better time than right now to get started.

25:29

Our listeners can get Rosetta Stone's lifetime

25:31

membership for 50% off.

25:34

Visit rosettastone.com/grammar.

25:37

That's 50% off unlimited access

25:39

to 25 language courses for the

25:41

rest of your life. Redeem

25:43

your 50% off

25:45

today at rosettastone.com/grammar.

25:53

So you know some of the people I encounter

25:56

who are most opposed to

25:58

generative AI are... are fiction writers, which

26:01

I think is kind of funny because not

26:03

that I'm laughing at their concern, but that fiction

26:06

writing is one of the things I think that

26:08

AI is really bad at. What

26:11

do you find are the things that

26:13

for writers, AI is like the best

26:15

for? So there's

26:17

six broad use cases of generative

26:19

AI. Generation summarization extraction rewriting classification

26:21

question answering, those are the big

26:23

six. Of those six,

26:26

two of them it's less good

26:28

at. Generation, AKA writing and

26:31

question answering because of the factuality issues.

26:33

Summarization, take big data and make it

26:35

smaller data, they are almost flawless at,

26:37

they are so good at it. Extraction,

26:39

take data out of data. Super

26:42

powerful, super helpful when, you know, we're in

26:44

the midst of a presidential election here in

26:46

the USA and this one group

26:48

released this one big document and I said,

26:51

extract out just these parts because I care

26:53

about this and I wanted to read just

26:55

that part of it and it did a

26:57

fantastic job. Rewriting, take one form of data,

27:00

turn it to another form of data. So

27:02

for example, one of the best things tools

27:04

do incredibly well is to take complex things

27:06

and explain them into something you understand. So

27:08

if you are a fiction writer, you can

27:11

have it explain something in a story format

27:13

that's maybe an arcane concept like how the

27:15

spike protein on the SARS-CoV-2 virus works. You

27:17

know, explain this to me as though it

27:20

were a fantasy epic and

27:22

it will do an incredible job that's

27:24

still factually correct. And classification, take the

27:26

data and organize it. With fiction writing,

27:28

here's why generative AI gets a bad

27:30

rap because people can't prompt to help

27:33

with fiction writing. That's really what it

27:35

boils down to. And

27:37

because they're trying to do too much. I

27:39

use generative AI for fiction writing all the

27:41

time. I write fan fiction with it and

27:44

my fan fiction is good enough that people

27:46

can't tell unless I tell them this was

27:48

written with generative AI. And

27:50

the process and procedure goes like this.

27:53

Number one, I preload the models conversation

27:55

window, the context window, which is a

27:57

short term memory with the topic, whatever.

28:00

the general ideas, then I brainstorm

28:02

with the machine what the overall

28:04

plot should be. And

28:06

then I have it build out character cards. And we

28:08

go back and forth about who the characters should be.

28:11

We look at what

28:13

are their fatal flaws, we do things

28:15

like Christopher Booker's seven major

28:17

story types, etc. And then I say,

28:19

let's start writing the outline for

28:22

the story. And I

28:24

say, first, let's do 15 chapters, right?

28:26

Write out the outline for each 15 chapters, re-arc

28:29

format, and so on and so forth. And say,

28:31

great, now let's divide each chapter into five sections.

28:34

And we'll then break down each chapter into

28:36

five sections in the same period. But each

28:38

section, now, I want you to using all

28:40

the stuff that we've got, you're

28:43

going to write chapter one, section one, with

28:45

a minimum word count of 1200 words,

28:47

do not repeat yourself

28:49

and so on and so forth, give it a bunch

28:51

of instructions. And then I will provide it a writing

28:53

style. So I'll take one of my previous human

28:55

written stories that I've written and say, I

28:58

want you to copy my writing style exactly,

29:00

because I have a very specific way that

29:02

I like to write. And

29:04

so what it does instead, that

29:06

sort of generic kind of wishy

29:08

washy machine probability of writing, it

29:11

replicates me. And I have it

29:13

assembled these pieces, you know, one

29:15

section at a time and create,

29:17

you know, 50 60,000 word piece

29:21

of fiction. That's pretty

29:23

decent. You know, it's certainly good enough for fanfiction, because

29:25

you can't make any money on it anyway. It's illegal to.

29:28

But it for me, is

29:31

great for when I want to

29:33

express a story that I don't

29:35

want to sit down and hand write

29:37

out. Because I'll have an idea in

29:39

the middle about like, you know, let this be a cool

29:41

story. But I don't know that I really want to spend

29:43

six months writing it. Could I get something that's 90%

29:46

as good as me in six hours instead? The

29:48

answer is yes. And now I have my story

29:50

and whether or not even publish it, at least

29:52

it exists. I was doing a piece the other

29:54

day, I had this wild idea of

29:57

it was from a Reddit post talking about how

30:00

In the Terminator franchise, Skynet

30:02

should have won almost immediately. The

30:04

fact that Skynet didn't, and they

30:06

keep having more and more Terminator

30:08

movies, indicates that Skynet wasn't trying

30:10

to win. Why? Because

30:12

it was built for the purpose of war. If

30:14

it wins the war, it has no purpose anymore.

30:16

So its goal then is to prolong the war

30:18

as long as possible so that it continues to

30:21

have purpose. So I wrote

30:23

a fan fiction story where the Resistance,

30:25

instead of trying to save

30:27

John Connor over and over again, sent to Terminator

30:29

back to try and reprogram Skynet from the beginning and

30:31

it turned into a machine love story. Hmm,

30:34

that's fun. You reminded me

30:36

of something that came up. So the length

30:38

of things you're trying to get out of

30:40

AI, you know, one of the primary things

30:42

I use it for is transcription. You

30:45

know, we've been doing these interview podcasts, but I

30:47

feel like it's really

30:49

important for accessibility to have good

30:52

transcripts. So

30:54

we use AI to make the transcripts

30:56

and it enables me really to

30:58

do these podcasts. You know, I can do

31:00

two shows a week now because we can

31:02

get those AI transcripts. That's one of the

31:04

reasons. And like

31:06

the other day, I put in

31:08

the audio and it came

31:11

out perfectly formatted, really great, the first

31:13

half, and then halfway through it just

31:15

started putting the text on there with

31:17

no punctuation, no capitalization. It was this

31:19

block of vomited text. And it had

31:21

been so good for the first half.

31:23

And like, what's the deal? Why did

31:25

it just, I mean, I know it

31:27

doesn't have intention, but why did it

31:29

just get tired and give up halfway

31:31

through? It depends on

31:33

so which, which model or engine was using.

31:36

I was using Mac Whisper. Okay. So

31:38

using the Whisper model, the Whisper model

31:40

loses coherence after a while, depending on

31:42

its setting. This is one of the

31:44

reasons why we don't recommend people use

31:46

it unless they have some technical knowledge

31:48

there because there's a way to start

31:51

that particular engine that has a very

31:53

specific parameters for voice,

31:56

especially if there are pauses longer

31:58

than three seconds. It

32:00

blows up. It just loses its

32:02

crap and just goes crazy. Generally

32:05

speaking, for transcription,

32:08

I will typically recommend people use, if

32:10

they have the budget, use a vendor

32:12

like Fireflies. Fireflies charges you by the

32:14

minute of uploaded audio, and then obviously

32:16

once you get the transcript, just delete

32:18

the audio and you can stay on

32:20

the lowest level of subscription plan. And

32:22

then from there, you would put it

32:24

into a tool like Google Gemini to

32:26

essentially clean up the transcript and remove

32:29

stop words and all these other things.

32:31

Whisper is a fantastic model. I use

32:33

it all the time. I use it

32:35

to transcribe YouTube videos and things, but

32:37

its transcripts always require cleanup as well.

32:39

So with all of these tools, you

32:41

need to have almost like a workflow

32:43

that allows you to take, here's the

32:45

raw audio, turn it into raw transcript,

32:47

take the raw transcript, refine it into

32:50

grammatically correct grammar, punctuation, spelling, white spaces,

32:52

all of the stuff. And then you

32:54

can then take that and do other

32:56

stuff with it. Yeah. Another thing I've

32:58

used it for really successfully is

33:01

having it teach me how to use

33:03

Google spreadsheets. So I had

33:06

the spreadsheet with the URL for

33:08

every page on our website. And I post

33:10

them to social media over the years and

33:12

I always keep the teaser. And so

33:14

I have the teaser and the URL so I

33:16

can use them again in the future. And when

33:18

we redesigned our website, every URL changed and

33:21

we didn't get it into my spreadsheet. So it broke

33:23

my spreadsheet and it was that way for years and

33:26

I didn't post as much to social media because my

33:28

spreadsheet was broken. And then I

33:30

realized I could use chat GPT to

33:32

show me how to like run a

33:34

script to replace all those URLs and

33:36

match them to what they needed to

33:38

go to. And it's something I

33:40

don't know. I don't know that I ever could

33:42

have done a Google search to figure out the

33:44

answer to that particular problem. I mean,

33:46

what do you call that? I mean, that

33:48

I didn't hear that under your, you know,

33:50

five or six things that AI can do.

33:54

That's question answering, but we're really talking about coding.

33:56

You you are having these tools

33:58

help you code because you know, you're writing a script.

34:00

script, or you're writing some macro to do some, that's

34:02

code. And these tools are

34:04

phenomenal at coding. In fact, these tools

34:06

are so good at coding that is

34:09

having a negative effect on

34:11

hiring of coders. Because if your current code

34:13

is you have on staff, pick up, generate

34:15

and start using it, they can become 2x,

34:18

3x, 4x more productive overnight. Now,

34:21

that means that you don't

34:23

have to hire new coders, you don't have to

34:26

add headcount, you can get a lot more efficiency

34:28

out of people that you already have. And

34:31

the reason for this is that

34:33

code is much less ambiguous than

34:35

the written word, right? And then

34:38

and how we read and write language, a

34:41

statement in Python is either correct or not,

34:43

it runs or it doesn't, as opposed to

34:45

say, you know, the expression

34:47

like that's what she said, right? There's

34:49

so many ways to say that if

34:51

you change the emphasis, like that, that's

34:53

what she said, or that's what she

34:55

said, just the inflection changes the meaning

34:57

of the sentence. And on a page,

34:59

you can't see that. So language models

35:02

struggle with human language, but machine language

35:04

like a Python or an R or

35:06

C++, there's very little room

35:08

for ambiguity. And so they are much

35:10

better at machine languages than human languages.

35:12

So when you're doing stuff like that,

35:14

even as writing a macro or script

35:16

or whatever, yeah, you are writing code,

35:18

you have any tools help you write

35:20

code, answer questions with

35:22

code, and they're really good at

35:24

it. Yeah, it was amazing. The

35:27

other thing that is that I

35:29

keep forgetting that I always forget about AI

35:31

is that it can it can help you

35:33

solve the problem you're having with it. So

35:36

the first time it told

35:38

me how to write the script, it didn't work. But

35:41

I put in I said, Okay, this is that this

35:43

is the error message I'm getting. And I put it

35:45

back in and said, Okay, this is the error message.

35:48

What do I do now? And and then

35:50

it told me what to do and I got it

35:52

right. I find that sort of confusing. Like how how

35:54

can it get it wrong the first time but then

35:56

know the second time what to do like it's

35:59

weird like this This technology is weird. It's

36:02

not like anything I've used before,

36:04

and you have to remember that.

36:06

That it's like an interaction. I

36:09

don't know, can you talk a little bit about that? The

36:11

nature of the underlying technology is such

36:13

that the more you talk about something,

36:15

the smarter it gets on that thing,

36:18

because it has this memory. Remember,

36:20

we were talking about how every piece of

36:22

a word associates with another word, the

36:25

whole idea of word clouds. Well, if you

36:27

ask it something and you don't provide a

36:29

lot of information up front, it's gonna do

36:31

its best to infer what it is you're

36:33

talking about, and it's gonna get it

36:36

wrong more often than it should, because it's trying

36:38

to even just guess what your intent is. The

36:41

single biggest problem that people do, the

36:43

single biggest thing that people do wrong

36:45

with these tools is their prompts are

36:48

too short. My average prompt is three

36:50

to five pages of text before we

36:52

ever get started with anything. That is

36:54

how much information. And to give you

36:57

a sense of how much they can

36:59

handle, this short-term

37:01

conversation window of a chat and GPT,

37:03

it can hold 90,000 words. So

37:06

that's like this. This is about

37:08

75. This

37:11

can be a prompt. This whole thing

37:13

can be a prompt. Google's Gemini, the

37:15

new version of Gemini, can hold two

37:18

of these as a

37:20

prompt. 1.4

37:23

million words is how much information can

37:25

go into a prompt. The

37:27

more data you provide, the better.

37:29

So when I prompt these tools,

37:31

we have a format that we

37:33

call race, role, action, context, execute.

37:35

You tell the model who it

37:37

is. You are a Pulitzer Prize-winning

37:39

author who specializes in writing sapphic

37:41

fiction, right? You give it the

37:43

action. Your first task today is

37:45

to write the outline for a

37:48

sapphic short story that's gonna be between

37:50

1,500 and 2,500 words, and

37:53

it will encapsulate a slice of

37:55

life scenario in the character's art.

37:57

Your context, here's the background. Here's who the character

37:59

is. is, here's their goals and motivations, everything

38:01

that you would expect in a character card in a

38:04

piece of fiction. Execute, write

38:06

the outline for the story

38:08

in whatever format and

38:10

things. And so if

38:13

you prompt with enough information upfront and

38:15

provide enough information, the likelihood that it

38:17

gets it wrong the first time goes

38:20

down. The more data you provide, the

38:22

less it screws up. That's so critical

38:24

for people to understand. I

38:27

can imagine some people are wondering, okay,

38:29

you're writing a six page prompt. Like,

38:31

isn't it easier just to write it

38:33

on your own? Like, how do you

38:35

decide when, you know? That's a very

38:37

good question. The answer is it depends

38:39

on how reusable you want it to

38:41

be. Google just came out with gems,

38:43

which is their version of chat GPTs,

38:45

GPTs, you might have something that you

38:47

do frequently that you would want to

38:50

reuse over and over again. And yeah, the first time

38:52

you build that prompt, it's going to be a long

38:55

test. It's going to take you 15 minutes, 20 minutes

38:57

to build out that six page prompt of

38:59

stuff. But then every subsequent time you use

39:01

it, it's seconds. We

39:06

should talk about, you know, I know a lot

39:08

of people are really concerned about the biases in

39:11

social media, not social media, in AI.

39:13

You know, it's trained on

39:15

biased material. So it perpetuates biases.

39:18

Like, how do you get around that? How do

39:20

you approach that as a problem? Well, for a

39:22

couple of, you know, a couple of ways. Number

39:24

one, you have to be aware of your own

39:26

biases. When you are creating

39:29

content, if something reads okay to you,

39:31

one of the questions should be is am I

39:33

biased? The answer is yes, everyone's biased. And

39:36

if so, in what way does

39:38

that show up in my approval of what

39:41

this content is? In

39:43

general, anything that goes out to

39:45

the public should never be go out without some

39:47

form of human review, especially if it is money,

39:50

if it is legal, if it

39:52

is health, right, your money or your

39:54

life, anything high risk where if you

39:56

you could get sued, you don't want

39:58

machines talking to the general public to

40:01

you have understand the biases in the

40:03

machine. These are mirrors of us. So

40:05

everything that you do or don't love

40:08

about the human species is

40:10

in these tools. And many of

40:12

them have some level of safety

40:14

mechanisms, but the reality is they're

40:16

about as good as the safety

40:18

mechanisms in your HR process, which

40:21

is to say you can still hire a jerk. And

40:26

three, using safeguards like having

40:29

a gem, for example, or

40:31

GPT, that is a well

40:33

trained sensitivity reader, where you

40:35

can say, yeah, check my

40:37

work, it is impossible to

40:39

build an AI that is unbiased. It

40:41

just flat out impossible. It is impossible

40:43

to raise a human being that is

40:45

unbiased, but you absolutely can raise human

40:47

beings who are aware of biases and

40:49

can check themselves. And you absolutely can

40:51

do the same thing with AI. Great. Well,

40:53

thank you so much, Chris. I'm going

40:56

to wrap up the main section here

40:58

for our regular listeners. We'll continue on

41:00

the discussion for our wonderful Apple

41:03

podcast supporters and grammar pollusians. We're

41:05

going to talk about, I want to talk about job

41:07

loss, the concerns about job loss, and

41:09

a great blog post you had about

41:11

how to make AI writing sound like

41:14

you. But to say goodbye

41:16

to the big audience, where can people find

41:18

you? I imagine that after hearing this conversation,

41:22

many of the people will want to follow

41:24

you now. So where can they find you?

41:27

You can find me for work

41:29

at TrustInsights.AI. That is my company.

41:32

And if you want help with your

41:34

AI efforts, we do that. We are

41:37

management consultants. If you want to

41:39

read my stuff, you can find

41:41

me at christopherspen.com. Super. Thanks

41:43

so much, Chris. And I

41:45

have one more announcement before we go. On Saturday,

41:49

yes, Saturday, grammar

41:51

polluza subscribers are going to get another

41:53

bonus episode. This time, it's an interview

41:55

with Drew Ackerman, better known as Scooter

41:57

from the Sleep With Me podcast. If

42:01

you don't already know about it, Drew came

42:03

up with the trippiest idea that became a

42:05

huge hit. He bores people

42:07

to sleep. He tells

42:10

rambling, boring, and yes, sometimes trippy

42:12

stories that people put on to

42:14

help them sleep or to get

42:16

through the deep dark night, as

42:18

Drew says. And Saturday,

42:20

as is timely following today's show, you

42:22

can hear Drew and I talk about

42:25

the ways he is and isn't using

42:27

AI to make his show. How

42:30

the hallucinations that are usually such

42:32

a problem for people have actually helped

42:34

him. But we also talk about

42:36

our worries and the ways it hasn't been helpful. So

42:39

if you're a Grammar Palooza subscriber, look

42:41

for the Sleep With Me interview in

42:43

your feed Saturday. And if you're not

42:45

a subscriber yet, you can always check

42:47

it out with a free trial on

42:49

Apple podcasts or subtext. All the

42:51

details are in the show notes. That's

42:54

all. Thanks for listening. When

43:03

you need an SUV that's roomy for

43:05

your friends and family, durable and ready

43:07

to carry all your stuff and you

43:09

quit for adventure, Honda is here with

43:12

the 2025 Honda CR-V. CR-V

43:15

outperforms the pack with more style,

43:17

more space and more power. Complete

43:20

with state of the art safety features and

43:23

a five star safety rating. Honda

43:25

is here for you with the 2025 Honda CR-V. Check

43:29

it out at your local Honda dealer

43:31

today.