0:00

This podcast is sponsored by TalkSpace. You

0:02

know when you're really stressed or not

0:04

feeling so great about your life or

0:06

about yourself? Talking to someone who understands

0:08

can really help. But who is that

0:10

person? How do you find them? Where

0:12

do you even start? TalkSpace. TalkSpace makes

0:14

it easy to get the support you

0:16

need. With TalkSpace, you can go online,

0:18

answer a few questions about your preferences,

0:20

and be matched with a therapist. And

0:22

because you'll meet your therapist online, you'll

0:24

have to take time off work or

0:26

arrange childcare. You'll meet on your schedule,

0:28

wherever you feel most at ease.

0:30

If you're depressed, stressed, struggling

0:33

with a relationship, or if

0:35

you want some counseling for

0:37

you and your partner, or

0:39

just need a little extra

0:41

one-on-one support, TalkSpace is here

0:43

for you. Plus, TalkSpace works

0:46

with most major insurers, and

0:48

most insured members have a

0:50

$0 copay. No insurance, no

0:52

problem. Now get $80 off

0:54

of your first month with

0:56

promo code space 80 when

0:58

you go to talkspace.com. Match

1:01

with a licensed

1:04

therapist today at

1:06

talkspace.com. Save $80 with code

1:08

space 80 at talkspace.com. Hello,

1:10

everybody. And today we have

1:12

a special guest, Jesse Speedback.

1:14

Great to be here. So

1:16

Jesse, would you mind explaining

1:19

just a bit about who

1:21

you are, some of the

1:23

things that you're doing, who

1:25

you work for, and why

1:27

you're famous, and all that good

1:29

stuff? Yeah, absolutely. So my name's

1:32

Jesse Speedack. I'm a senior engineer

1:34

at a company called Hybata, which

1:36

is a cashback for shopping app

1:38

based in Denver, Colorado. I've been

1:40

working there for about three and

1:42

a year years page. I guess

1:44

I'm famous as it were because

1:47

I gave a talk at the

1:49

first remote rails conference past May,

1:51

and I talked about kind of

1:53

how crummy of a developer I

1:55

am. Yeah, I think we can

1:57

all relate to that on the...

1:59

daily basis sometimes. So would you

2:02

mind giving a bit of a,

2:04

you know, highlight talk about what

2:06

you covered at the conference and stuff?

2:08

So we can just kind of pick

2:10

it up from there. We'll link to

2:13

the in the show notes, a link

2:15

to the conference. But just for

2:17

those who maybe didn't see

2:19

it. Sure. And there's no

2:21

substitute for actually watching

2:23

this fantastic talk. But

2:25

more seriously. The talk is really

2:28

about. my experience as a tech

2:30

lead at I bought a working

2:32

on a pretty critical project

2:34

over the course of about

2:36

six months or so and over

2:38

that time I made four very

2:41

big mistakes that put the project

2:43

in jeopardy and hopefully

2:45

there are mistakes that I will

2:48

learn from and not make again

2:50

as I continue to lead projects

2:53

that I bought in future and my

2:55

hope is that I sort of

2:57

articulating these mistakes and what

2:59

I learned from them, other

3:01

folks can benefit. And so the

3:03

four mistakes that I made are

3:06

first, we picked the wrong technology,

3:08

we can get more into that. We

3:10

also as a team, we siloed

3:12

work. So work was divided up

3:15

in not the best way. We

3:17

fell into the premature optimization

3:19

track and then maybe worst of

3:21

all, we made. way too many changes

3:23

at one time. So I can go

3:26

into detail on any of those. Yeah,

3:28

and I think it's fair to say

3:30

that I, our listeners, other members on

3:32

the panel, have been there before. Yeah,

3:35

I've worked with loads of people who've

3:37

made those mistakes. Obviously, I've never made

3:39

them myself, but oh my words. Oh

3:41

my words. Luke is the perfect developer,

3:44

let's be fair. No, I was actually

3:46

going to jump in and say the like...

3:48

I feel like I don't feel like

3:50

I'm alone but usually when you

3:52

make a mistake it cascades into

3:55

more for whatever reason I generally

3:57

feel like you either you either kind of

4:00

ever rolling along and you feel pretty

4:02

good about stuff, or you're just

4:04

like in a bottomless pit. And

4:06

there's very little in between time.

4:08

You just suddenly notice that you're

4:10

at the bottom. So yeah, just

4:13

saying. And absolutely. And I just

4:15

want to kind of call out that there's

4:17

like a certain amount of privilege

4:19

that comes with being able to

4:21

talk about our mistakes, right? I'm not

4:24

worried that my boss is going to fire

4:26

me and I'm also not worried that. folks

4:28

won't take me seriously for giving

4:30

this talk. If anything, it probably

4:33

improves my reputation as evidence by

4:35

getting to talk to three of you

4:37

gentlemen. So I just want to call that

4:39

out because I think it's important.

4:41

I think that, okay, so speaking of that, I

4:43

think that's a give and take there too. So

4:46

I think this is one of, this will get

4:48

into a thing that I care a

4:50

lot about this particular subject because for

4:52

most of my, shoot, I still feel

4:54

it today or whatever, it doesn't really

4:56

matter. I still always feel this pressure

4:58

to not let people know about the

5:00

mistakes that I made, right? Because letting

5:03

people know about the mistakes that I

5:05

made makes me more vulnerable to them

5:07

not wanting me to work for them,

5:10

right? Like fire me, whatever, whatever the

5:12

bad thing is that it's a boogie

5:14

man. It's not real. It's just a

5:16

pressure, right? And so I guess what I'm

5:18

trying to get at is like, I kind of

5:21

feel like one of the things that

5:23

I always care a lot about with

5:25

mistakes is telling people about your

5:27

mistakes I actually have discovered

5:30

in reality actually makes people

5:32

respect you more but it doesn't

5:34

change the fact that it's like really

5:36

hard and I still feel that pressure

5:38

to this day yeah I agree with

5:40

what you're saying I think that for

5:42

a lot of us talking about mistakes

5:45

openly and honestly and what

5:47

we learn from them actually

5:49

builds our credibility but that's

5:51

not always the case depending

5:53

on sort of how you present what

5:55

you look like. I think that you

5:57

guys recently did an episode. I

6:00

think you talked about issues of

6:02

hiring and getting equality in a

6:04

workplace and I think that plays

6:06

in here for sure. And I

6:09

would go as far to say

6:11

that it depends on what the

6:13

mistake is, what kind of mistake

6:15

it is. If it is just

6:18

utter incomplete negligence, then those aren't

6:20

really the kind of mistakes I

6:22

would really want to be forthcoming and

6:25

outright about, you know, but like

6:27

if I went in. and always

6:29

had a VP and tunnel into

6:31

my production environment. And then we

6:33

got malware in our local work

6:35

environment and that transferred over to

6:37

production and encrypted all of our

6:39

production data. I don't know if

6:41

I would really say like, oh

6:43

yeah, you know, that was a

6:46

silly mistake in mine. No, it's

6:48

like, okay, not only are our

6:50

customers affected, but now, you know,

6:52

my job's in jeopardy because I

6:54

decided to always take shortcuts. But

6:57

something like, and what I'm really

6:59

interested in is your technology framework,

7:01

or the, you use the wrong

7:04

technology. Can you explain a bit

7:06

more about the scenario? Sure, absolutely.

7:08

So this, the high level

7:11

we, at Ibono, we have

7:13

just a wonderful majestic monolith

7:15

rails application. Actually, originally

7:17

the application we've written in

7:19

Scala. And then after about a

7:21

month of that, they switched it over

7:24

and rebuilt the whole thing in

7:26

rails. And it's been rails ever since.

7:28

So it's going on close, almost

7:30

10 years at this point. So it's

7:32

a large application. It serves millions

7:34

of users. Very cool. And about three

7:37

years ago when I joined, we

7:39

started growing the team and thinking about

7:41

how we could, like many people have,

7:43

pull pieces out of the monolith

7:46

into. microservices. So this

7:48

project in particular was about taking a

7:50

piece of billing logic from the system

7:52

from the Monmouth and pulling it out

7:54

into a microservice. The hope was

7:57

to make it better encapsulated, easier

7:59

to iterate on. I don't think it's

8:01

bad in and of itself. I

8:03

just think it was not the

8:05

right technology. Actually, before I say

8:07

the technology, before I get trolled

8:10

by all the lovers of this

8:12

technology, I'm going to preface this

8:14

by saying, I don't think that

8:16

this technology is wrong, and I

8:18

don't think it's bad in and of

8:20

itself. I just think it was

8:22

not the right technology for the

8:24

problem and the team. Hold on

8:27

a second. Hold on a second, Jesse,

8:29

because the slide I'm looking at says

8:31

big mistakes. It doesn't say small mistakes.

8:34

It says big mistakes. So let's

8:36

just keep this in mind for

8:38

we reveal what this technology is,

8:40

what a big mistake was. Absolutely.

8:42

This, whoever, whoever, who ever uses

8:45

this technology is, is definitely making

8:47

a big mistake. So, I know, spoiler,

8:49

we weren't building like us, a

8:51

side rails app micro service, which

8:53

would probably would not have been

8:55

as big of a mistake. The

8:57

issue really was that our team,

8:59

the small team of developers and

9:01

then a larger team of engineers

9:03

in the company, really did not

9:05

have a ton of experience with

9:07

the framework that we chose. And

9:09

as a result, we ended up

9:12

having to do a lot of

9:14

plumbing and reinventing the wheel and

9:16

just not benefiting from

9:18

the institutional experience that

9:21

exists at IBATA. Unfortunately,

9:23

right, this this could work if

9:25

you're doing kind of like a proof

9:27

of concept, like, let's show what

9:29

this technology can do, let's pick a

9:31

pretty isolated use case. But

9:33

the billing logic that we were pulling

9:36

out about Monolith was basically

9:38

like, do or die, right? If it did not

9:40

work, it costs millions of dollars

9:42

to fix or, you know, it ends

9:44

up costing the company millions of dollars.

9:46

So it really... We were walking on

9:49

a tie road and there was no

9:51

net underneath us. And unfortunately we decided

9:53

to, I guess, like walk on our

9:55

hands instead of go across normally.

9:57

So the framework that we use is called

10:00

ACCA and I think for a

10:02

team that knows ACCA this probably

10:04

would have been really a perfect

10:06

tool for the job. But our

10:08

team and our company really did

10:10

not have a ton of experience

10:12

with ACCA. And so unfortunately we

10:14

weren't able to sort of take

10:16

advantage of it and use it

10:18

in a way that sort of

10:20

professional ACCA developers likely can. So

10:22

this is kind of like a

10:24

functional programming thing? Right. It deals

10:26

with data streams and passing data

10:28

along. in a functional paradigm and

10:31

it's meant to accommodate high volume

10:33

data across highly parallelized system. So,

10:35

you know, at the time, we

10:37

went there, well, I'll talk about

10:39

what we went there in a

10:41

second, but in retrospect, it was

10:43

something that could handle basically 10,000

10:45

X really what we needed in

10:47

terms of what it was designed

10:49

to handle. So just sort of

10:51

on paper, it probably wasn't the

10:53

best, the best move. in that

10:55

respect. But I could also talk

10:57

about the team as well and

10:59

why it wasn't a great, a

11:02

great fit. Yeah, can you, can

11:04

you talk for just a minute

11:06

about like what was the problem

11:08

that you were trying to solve?

11:10

Why did you choose, like what

11:12

did you need ACCA for? Sure,

11:14

yeah, perfect. So the problem that

11:16

we're trying to solve and you

11:18

have to know a little bit

11:20

about the IVADA app. So I

11:22

assume all of you, all of

11:24

you guys have downloaded it and

11:26

are actively using, like. Nope. That's

11:28

funny. We did assert, well, I'll

11:30

get to that later, but basically,

11:33

I bought it as a way

11:35

for you to get digital coupons.

11:37

You can get brands, put offers

11:39

in the app, you click on

11:41

the offer, you show us evidence

11:43

that you bought the thing that

11:45

is on offer, and then I

11:47

bought it, we'll pay you cash

11:49

back, they'll send it to your

11:51

PayPal account, give you a gift

11:53

card to Amazon, whatever you want.

11:55

So the problem that we're trying

11:57

to solve is how we make.

11:59

Don't exceed the budget. that is

12:01

allocated to them by the brands

12:04

that put those offers in the

12:06

app. And that sounds maybe like

12:08

an easy problem, like there's an

12:10

easy way to just say, okay,

12:12

there's $500,000 in budget for Oreo

12:14

coupons, just divide $500,000 by how

12:16

much money we're giving out per

12:18

coupon, and then you know, but

12:20

it's actually much, like, it's obviously

12:22

much harder than that. And in

12:24

order to preserve a good user

12:26

experience, we need to make sure.

12:28

that we're not yanking content and

12:30

surprising our users. Like you would

12:32

be really upset if you went

12:35

to the store specifically to buy

12:37

Oreos to get coupon and then

12:39

by the time you checked out

12:41

the coupon is no longer in

12:43

your application. So we have to

12:45

run some predictive algorithms to basically

12:47

guess when we're going to run

12:49

out of money and kind of

12:51

slow the coupon, slow the velocity

12:53

down as we approach that point.

12:55

Oh dear. That's a dangerous recipe.

12:57

I must add that I don't

12:59

think about it is available in

13:01

the UK at the moment. Is

13:03

it available in Canada? We are

13:06

only in the United States right

13:08

now. Sorry, Luke. So... In my

13:10

defence. This looks like a perfect

13:12

storm for me because you've both

13:14

got the kind of strong financial

13:16

components, so if you get it

13:18

wrong you lose money. And if

13:20

you get it long, then people

13:22

will waste their time going to

13:24

the shop, you know. in Britain

13:26

it's so small I can walk

13:28

to the shop. You know sometimes

13:30

I just open the window and

13:32

shout at them and tell them

13:35

what I want. But you know

13:37

I know when I was living

13:39

in the States and people maybe

13:41

drive for two hours to go

13:43

to Walmart. So this is this

13:45

is quite a quite a problem

13:47

you've got there. Yeah it's interesting

13:49

that you say that because right

13:51

when I joined the company I

13:53

was kind of put on. The

13:55

team, I was actually given, you

13:57

know, my tech body, my mentor

13:59

when I joined the company. This

14:01

was his project and as someone

14:03

knew in the company, it was

14:06

very, it was really overwhelming problem

14:08

space because basically these campaigns and

14:10

our application almost got like a

14:12

physical momentum to them. So if

14:14

you imagine trying to stop a

14:16

moving train, you can't just hit

14:18

the brakes and expect it to

14:20

stop on a dime. You have

14:22

to apply the brakes over some

14:24

distance to slow the train down

14:26

and that's really how the content

14:28

in the application is modeled. And

14:30

I'm not a physics person and

14:32

so it's really confusing. I mean,

14:34

it's a it's a twist right

14:37

on your classical inventory problem, right,

14:39

is what it kind of sounds

14:41

like. So the way that ACCA

14:43

came into this is is what

14:45

you were taking like what all

14:47

these requests from users and trying

14:49

to decide whether or not you

14:51

were going to, I guess, give

14:53

them the coupon or not or

14:55

something. Yeah, that's, that's, that's sort

14:57

of right. So we have an

14:59

event-based architecture. where our system, for

15:01

the folks who are familiar with

15:03

that, that means that basically your

15:05

system publishes events, which is data,

15:08

that signify that something just happens

15:10

of interest in the system. So

15:12

maybe like, you know, shopping cart

15:14

loaded is an event that you

15:16

might have in like a typical

15:18

inventory space or something like that.

15:20

So we have events that we're

15:22

interested in, like, content awarded, which

15:24

means that John went to the

15:26

store submitted or submitted or received

15:28

through the app. and got cash

15:30

back, so the content has been

15:32

awarded. So we listen for those

15:34

events in order to keep track

15:36

in real time of how much

15:39

budget is being used, and we

15:41

basically track that over time to

15:43

make a rough prediction about how

15:45

fast things are moving. And so

15:47

ACCA, sorry, John, to get back

15:49

to your original question, ACCA is

15:51

really good at streaming data, at

15:53

streaming large amounts, high volume data,

15:55

high volume data. And I bought

15:57

a, we'll get on the order

15:59

of 700,000. these content awarded events

16:01

per day, which seems like a

16:03

lot, but it's actually much lower

16:05

than I think what ACCA can

16:07

kind of deal with or is

16:10

designed to deal with out of

16:12

the box. Yeah, did you, so

16:14

did you consider alternatives? Did you

16:16

reject them for various reasons? And

16:18

I guess, so obviously ACCA didn't

16:20

work out for you, so you

16:22

must have picked something else that

16:24

you like better. And how did

16:26

you arrive at that new thing?

16:28

And what was that, if that

16:30

makes some sense? Yeah, perfect. So

16:32

yeah, we basically, this comes down

16:34

to some team issues again and

16:36

not an issue with ACCA. So

16:38

the team issue was basically that

16:41

at the start of this project,

16:43

we scaled up our team. Like

16:45

this is a lot of work.

16:47

We need to bring in some

16:49

artillery and we brought in a

16:51

new developer from outside the company

16:53

who is awesome. She's a rock

16:55

star. and had it was coming

16:57

from the ad product space and

16:59

with dealing with volume much volume

17:01

of streaming data at a scale

17:03

much higher than what we needed

17:05

or we're going to be dealing

17:07

with it in any near near

17:09

future and you know she was

17:12

coming from I believe in Akasha

17:14

and so she joins the team

17:16

we're excited to have her we

17:18

think she's a rock star I

17:20

mean she is a rock star

17:22

and She's like, this is a

17:24

perfect use case for ACCA. We're

17:26

like, okay, never heard of that,

17:28

but you know, we trust you.

17:30

You crushed our interview and you

17:32

think you're amazing. So yeah, that

17:34

sounds pretty good. And again, this

17:36

isn't a knock against ACCA. I

17:38

think this is just that in

17:41

our company, we have a ton

17:43

of infrastructure set up to support

17:45

the tools that our company has

17:47

sort of blessed that that that

17:49

are kind of frequently in use.

17:51

In fact, we have like a

17:53

name for that we call it

17:55

the paved road which is like

17:57

if you're an engineer, you have

17:59

a lot of autonomy about about

18:01

what you use, but if you

18:03

stay on the paved road, it

18:05

should be an easy path. And

18:07

ACCA, which is a, you know,

18:09

which use with Java or SCALA,

18:12

typically, is not, I-bought as paved

18:14

road. So we had to kind

18:16

of have a contentious meeting, a

18:18

contentious conversation with the architects at

18:20

Arbata and say, no, we really,

18:22

we believe in in our developer,

18:24

and we think that she knows

18:26

what she's doing, and we're ready

18:28

to, and we're ready to. sort

18:30

of make our bed and lay

18:32

in it. And they were like,

18:34

well, you know, as long as

18:36

you know that. So they let

18:38

us kind of, they gave us

18:40

just enough rope to, I forgot

18:43

how the rest of that goes,

18:45

to hang ourselves with. Okay, so,

18:47

so it sounds like you had

18:49

an advocate and that's kind of

18:51

how you landed on it. Where

18:53

did you go after you decided

18:55

that ACCA was the poor choice?

18:57

Like, what did you end up

18:59

with? Yeah, so this, and this

19:01

kind of gets into the next

19:03

problem, which was the silo, the

19:05

silo, the silo, the silo, the

19:07

silo, We're starting to work on

19:09

building this ACCA-powered race car of

19:11

a microservice to pull billing logic

19:14

out of our rails monolith. And

19:16

there's sort of two streams of

19:18

work. There's the development of the

19:20

ACCA microservice, which is, we were

19:22

writing, the reason Cotton, which is

19:24

the JVM, it's kind of a

19:26

nicer job. It's what Android apps

19:28

are written in. It's actually pretty

19:30

nice. So we have that work

19:32

going on. And then we have

19:34

to integrate. that microservice, sorry, into

19:36

the rails model. So I ended

19:38

up taking on a lot of

19:40

the integration work and the other

19:42

developer took on most of the

19:45

ACCA and Scotland work. So we

19:47

have these sort of two very

19:49

isolated pieces of work that are

19:51

siloed and then something terrible happened.

19:53

And I don't blame this person

19:55

at all because I wouldn't want

19:57

to be on my team anyway.

19:59

She decided that she was much

20:01

more interested in data engineering and

20:03

she moved to... different team in

20:05

the company. So, and that's actually,

20:07

you know, a great thing about

20:09

working at IBata. Like, we've got

20:11

tons of really cool problems, different

20:13

spaces, and people are almost encouraged

20:16

to move around to find things

20:18

that they're, that they like and

20:20

are good at. So, she made

20:22

this move, and now, now the,

20:24

the problem of using a framework

20:26

that none of us had experience

20:28

with and that the general company

20:30

didn't have a ton of experience

20:32

with really became self-evident, because now

20:34

there was, there was still work

20:36

left to work left to do.

20:38

on the microservice. And I'm just

20:40

a, I'm just a rails developer.

20:42

Like I had to go into

20:44

there and start ready Kotlin, start

20:47

reading the ACCA documentation and try

20:49

to wrap my head around what

20:51

this whole, you know, actor system

20:53

meant. And it was, it was

20:55

tough. And that's why I call

20:57

it a big mistake. Sorry, Dave.

20:59

You know, from my experience too

21:01

is that, yeah, Ruby is slow.

21:03

No, there's no getting around that.

21:05

when you compare to a compiled

21:07

language. But holy crap is it

21:09

fast too. You know, I have

21:11

a production application which processes over

21:13

500,000 active jobs every single day,

21:16

and it does it extremely quick.

21:18

I don't need it to be

21:20

faster. I mean, that's plenty fast.

21:22

On the current setup that it's

21:24

on, which is... two cores and

21:26

four gigs of RAM and we

21:28

have two servers dedicated to the

21:30

background job, so two VMs, it's

21:32

able to handle that load and

21:34

we don't have to worry about

21:36

it crashing or anything like that.

21:38

So I mean, that's good enough

21:40

for us. Now, imagine that it

21:42

would be able to double that

21:44

workload before we ever ran into

21:47

any kind of performance issues where

21:49

we needed to start scaling up.

21:51

Absolutely, and we were originally in

21:53

the monolith. The way the system

21:55

worked was it ran on a

21:57

background scheduled job using rescue and

21:59

the scheduled job. basically took roughly

22:01

45 minutes to run. So it was

22:03

running like kind of like a waterfall like

22:05

every 10 minutes and started a

22:07

new job that would take 45

22:09

minutes to run. And so going from

22:12

that to basically completely real time

22:14

is a huge improvement. And if

22:16

real time for 500,000 events per

22:18

day versus 5 million events per

22:20

day are different things. But if

22:22

you're at real time, it's

22:24

already this just enormous improvement.

22:26

over what we were working with, which is

22:29

like the 45 minute loop. So at

22:31

that point, and I guess I'll say another

22:33

thing, before I get into like

22:35

how we fixed my mistake, this

22:37

is actually getting to what Dave said.

22:39

This is a case of premature

22:41

optimization. We sort of didn't

22:44

do the back of the envelope math

22:46

well enough or didn't have a

22:48

clear picture of like, okay, this

22:50

is really like designed to handle

22:52

like literally a thousand times more

22:55

traffic than our best day. So, you

22:57

know, we didn't, that was, that was definitely

22:59

a mistake. We should have asked

23:01

that question and realized, okay, maybe,

23:03

maybe this is a little too much

23:06

horsepower, we don't need this, it's too much

23:08

trouble for what is, for what is buying

23:10

us. And on my end, also, I was

23:12

imagining, you know, getting all these

23:14

events, we call it, so the

23:16

microservice was producing like prediction events,

23:19

okay, so predicting every time a

23:21

piece of content should come down

23:23

and making an when it thinks that

23:25

should happen. And so I'm

23:27

imagining the monolith is listening

23:29

to these prediction events, subscribe

23:31

to an S&S topic or SQ,

23:33

and I imagine, you know, thousands and

23:36

thousands and thousands of events

23:38

every second, which is wait, way too

23:40

much. And so I was thinking of like

23:42

all these clever ways to do cashing and

23:45

like to try to figure out when can

23:47

I drop events, so I don't need to

23:49

hit the database. Like when can I, when

23:51

can I, how can I come up with

23:54

these clever ways? to only make round trips

23:56

to the database when I really need to.

23:58

And that added time to the... development

24:00

and it also added a ton of complexity

24:03

so that when are two systems one were

24:05

like okay let's turn them on let's see

24:07

what happens the first error that comes out

24:09

because of course there's going to be an

24:11

error when you first try it out that was

24:14

really hard to debug it was really hard

24:16

to understand like is this a caching issue

24:18

is this actually an issue with

24:20

the data coming from the microservice

24:22

like it was really hard to isolate that

24:25

And then that obviously moves into

24:27

the third problem, the third mistake

24:29

was that we made too many

24:31

changes at one time. So all these were

24:33

ways that I tried to shoot myself in

24:35

my own foot while working on this very

24:37

important project. Makes sense?

24:40

I'm not sure if I understand it,

24:42

but so you're saying there's kind

24:44

of two separate things going on

24:47

here. Firstly, you're moving to a

24:49

completely new technology. So you're taking

24:51

a large part of the app

24:53

out, you're putting it in a

24:55

micro service. It's not very micro

24:57

if you're doing, what, 700,000 things

24:59

a day. That's not a micro

25:01

service. This is a, this is,

25:03

this is a macro micro, micro

25:06

or Medcro. And the second thing

25:08

you're doing is you're actually changing

25:10

the whole interface. So you're

25:12

saying you're going from, you

25:14

previously had a rescue batch

25:17

job. And now he's saying, no, no, no,

25:19

we're not going to do batching, we're going

25:21

to do it all immediately. Yeah,

25:23

Luke, that's a really a student observation.

25:26

So we were changing structure and

25:28

we were changing behavior at the

25:30

same time, which is something, since that

25:32

point, I've really tried to avoid

25:34

doing, even at like a micro PR

25:37

level, if I have a PR that I'm going

25:39

to push up. It's either going to

25:41

add behavior, change behavior, or it's

25:43

going to change the structure of the

25:45

structure of the code. So in this case,

25:47

we did the two things at the same

25:50

time. And at multiple points, we

25:52

should have known, or in retrospect,

25:54

we should have known to pause

25:56

and verify, right? And if we

25:59

couldn't verify. take that failure as

26:01

feedback and iterate. And I think

26:03

that that's really what really strong

26:05

engineers, experienced engineers know to do,

26:07

make one change at a time

26:09

and verify. Like I have a,

26:11

I guess it's kind of like

26:13

when I run our spec, right,

26:15

or our spec feels like supernatural

26:17

to me. So every time I

26:19

hit my test suite, I'm like,

26:21

okay, I expect this to happen.

26:23

And like sometimes I'm right. And

26:25

then sometimes I'm wrong and that's

26:27

like really interesting too. But making

26:29

that initial guess of what's going

26:31

to happen is super important. And

26:33

I think we did a slow

26:35

down and stop and say, okay,

26:38

here's our guess as to what's

26:40

going to happen, here's how we're

26:42

going to verify it, and if

26:44

we can't verify it, here's like

26:46

the corrected action we can take.

26:48

Yeah. So I think you're hitting

26:50

it like one thing that's like

26:52

really important, right? is not necessarily

26:54

like to avoid mistakes because shoot

26:56

for whatever reason, they keep happening

26:58

to me, right? It's, are you

27:00

good at cleaning up, right? Are

27:02

you good at figuring out that

27:04

something's not right, something's not right,

27:06

something smells funny, whatever it is,

27:08

can you go back and, you

27:10

know, sweep up your mess? You

27:12

know, either push it across the

27:14

finish line because you're close enough,

27:16

or say actually this is the

27:18

wrong path. We should back up,

27:20

go to the last intersection and

27:22

go down the other way. So

27:24

I mean, it's definitely, yeah, I

27:26

mean, I can speak to experiences

27:28

too, where I was just like,

27:30

man, I really like missed all

27:33

the flags. And then I kept

27:35

missing more flags and just kept

27:37

going. And why are we surprised

27:39

that we were in a bad

27:41

place? Because I like ignored everything.

27:43

So I hear you, man. And

27:45

kudos for recognizing it at some

27:47

point and doing something about it.

27:49

Yeah, so what do we do

27:51

about it? Unfortunately, I, well not

27:53

unfortunately, it was a good, it

27:55

was a good learning experience, but

27:57

obviously my happy place is, is

27:59

deep in a, in our leg.

28:01

Rail's application, finding all the pathways

28:03

there, but I had to really

28:05

push myself, get out of my

28:07

comfort zone. I had to pick

28:09

up Colin. Luckily, Colin is like

28:11

a super friendly language, I think,

28:13

to get into, especially for folks

28:15

who are coming from Ruby, because

28:17

I think that there's like kind

28:19

of like an emphasis on syntax,

28:21

on like, syntactic sugar, on like,

28:23

making the code actually like readable

28:25

and look nice, whereas that's not

28:27

always the case with all JVM

28:30

languages. So in that case, in

28:32

that sense, it was kind of

28:34

friendly. Also at Ibata, we were

28:36

able to like organize kind of

28:38

a learning group. So there were

28:40

lots of folks who were new

28:42

to Ibata or new to Colin

28:44

who were kind of trying to

28:46

solve these similar problems. So we

28:48

made a study group, we found

28:50

cool online coursework, we helped each

28:52

other accountable for making sure that

28:54

we were making progress on those

28:56

things. And I started writing Colin

28:58

in the microservice to. try to,

29:00

like you said, push across the

29:02

line. If we need to get

29:04

this across the line, and then

29:06

we can kind of circle back

29:08

and deal with some of the

29:10

underlying problems. And that's important because

29:12

like as engineers, like at the

29:14

end of the day, we're trying

29:16

to deliver value to the businesses

29:18

that we that we work for

29:20

and not just like trying out

29:22

any technology or like optimize what

29:25

what what technological solution should we're

29:27

we're implementing. So just to clarify

29:29

what what you're what you're saying

29:31

you kept. You kept ACCA. You

29:33

just pushed it across the finish

29:35

line despite all your problems. That's

29:37

how you resolve it. Okay. So

29:39

we had problems with it and

29:41

we decided, let's get this thing,

29:43

like we're close enough, even with

29:45

the problems, even with our lack

29:47

of understanding. We're close enough that

29:49

a total rewrite right now would

29:51

put us back a long way

29:53

and upset the business. So let's

29:55

push it across the line. and

29:57

then we can circle back and

29:59

figure out what to do. So

30:01

that was that delivering that value

30:03

even though it kind of felt

30:05

yucky, even though we knew that

30:07

there were things that were going

30:09

to need to change over the

30:11

long term, bought us some time

30:13

and brought us some credibility in

30:15

the organization. And I want to

30:17

circle back to the point you

30:20

made about moving too quickly or

30:22

too many changes happening at one

30:24

time. And I think that's something

30:26

that a lot of developers might

30:28

start to experience soon with the

30:30

whole. removal of jquery from bootstrap

30:32

five. So no, not my jquery.

30:34

No. Sorry. So the idea is

30:36

that bootstrap five no longer has

30:38

the jquery dependency. So you might

30:40

plan to upgrade your rails application.

30:42

the CSS framework from bootstrap for

30:44

to bootstrap five. And you might

30:46

say like, hey, well, why we're

30:48

making this change? Why don't we

30:50

go ahead and rewrite a lot

30:52

of our JavaScript that was also

30:54

JQI dependent? And so I would

30:56

say, instead of doing that, do

30:58

one thing. Either first, remove all

31:00

your JQI dependency within your application

31:02

and just have JQI be a

31:04

dependency of bootstrap. And then in

31:06

another iteration, upgrade your bootstrap version,

31:08

removing then J-queary entirely, or do

31:10

it vice versa, where you do

31:12

your bootstrap upgrade first, and then

31:14

you do your J-queary removal from

31:17

your application as a dependency. But

31:19

trying to do both side by

31:21

side, it's too big of a

31:23

task. You know, for one person

31:25

or one team to do right

31:27

away. I would just handle one

31:29

thing at a time, and moving

31:31

slower, you're going to say, okay,

31:33

what broke this? Was it the

31:35

new bootstrap framework that broke this

31:37

or was it our j-query rewrite?

31:39

So that way you're going to

31:41

be able to identify a lot

31:43

more problems quicker before they are

31:45

reported to you by the customer?

31:47

Absolutely. I think the most experienced

31:49

engine... one of the best engineers

31:51

I've worked with, his name is

31:53

Justin Hart, he was the, like,

31:55

did the first commit on the,

31:57

on the monoth that I work

31:59

on, and every time I work

32:01

with him, that is always my

32:03

experience, like, he is, like, we're

32:05

making this change, and we're going

32:07

to expect, and we're making this

32:09

change, and we're going to expect,

32:12

and we're going to do it,

32:14

and I'm always like, oh, why

32:16

don't we do this, this, this,

32:18

this, and this, and this, and

32:20

this, and this, and this, and

32:22

this, and this, and this, and

32:24

this, and this, and this, and

32:26

this, and this, and this, and,

32:28

and, and, and, and, and, and,

32:30

and, and, and, and, and, and,

32:32

and, and, and, and, and, and,

32:34

and, and, and, and, and, and,

32:36

and, and, and, and, and, and,

32:38

and, and, and in the projects

32:40

that I'm doing now, realizing, like,

32:42

okay, this thing that I want

32:44

to do, it's actually like four

32:46

different things, and I'm going to

32:48

do the first, the first thing

32:50

first, verify it, and then move

32:52

on from there. If you, all

32:54

your loved ones, are affected by

32:56

the removal of jquarie from Bootstrap,

32:58

why not check out the excellent

33:00

calls from Dev Chat, Dev Chat,

33:02

TV? You don't know JavaScript yet

33:04

30-day challenge available with the link

33:06

in the episode of this? Just

33:09

trying to just try to keep

33:11

a ship afloat man. Come on.

33:13

Come on. Oh, I was good.

33:15

So this goes along for me

33:17

with. No, no, no, no, no,

33:19

no, no, no. No, we have

33:21

to do jquery now because this

33:23

is really affecting me. I mean,

33:25

this is terrible news. I literally

33:27

can't write JavaScript without putting a

33:29

dollar sign in front of everything.

33:31

You can still have a dollar

33:33

sign. It just need to sign

33:35

something to the dollar sign. That's

33:37

all. One of the reasons I

33:39

started using Google, was because it

33:41

has a little dollar sign in

33:43

front of a data structure and

33:45

that kind of really makes me

33:47

feel at home. Plus, you know,

33:49

it feels like money, you know,

33:51

when I type the dollar sign

33:53

in, I feel like, yeah, this

33:55

is going to make me a

33:57

millionaire. Oh, I'm sorry, Luke. So

33:59

I guess, while we're on JavaScript,

34:01

I think another premature optimization, with

34:04

react. the dash at what back

34:06

equals react. Just thought through that.

34:08

in there for the arbitrary Bash

34:10

on React. I do like stimulus.

34:12

I do feel like I do

34:14

feel like we can we can

34:16

let React's go now. Yeah, I

34:18

mean, I don't know, have you

34:20

all checked out the Base Camp's

34:22

email? Hey, hey.com. Oh yeah. Yeah,

34:24

what do you think? Yeah, so

34:26

I've been using it and obviously

34:28

I'm a Base Camp fan boy,

34:30

but I think it's it's pretty

34:32

amazing what. what they're able to

34:34

do with, just HDML over the

34:36

wire for the most part, and

34:38

not relying on some of the

34:40

frameworks that have come out recently.

34:42

Also, I think it's super interesting,

34:44

like the 2020 Rubian Rail Community

34:46

Survey, if you look, the one

34:48

put out by Planet, Argonne, if

34:50

you look at like what JavaScript

34:52

libraries people are using, so I

34:54

was expecting that number one would

34:56

be react, maybe number two would

34:59

be, I don't know, or like,

35:01

Ember, or maybe, but number one

35:03

on there is j-quarie. And it's

35:05

like we're all in this in

35:07

this jquery world and it's not

35:09

it's not bad. I love your

35:11

query. So so for all of

35:13

you who along with Luke are

35:15

mourning jquery. I'm telling you stimulus

35:17

go check it out. Try to

35:19

understand it. I don't have an

35:21

answer. That's fair. I thought I

35:23

found it pretty easy. I thought

35:25

it gave me everything that I

35:27

felt like I was getting from

35:29

jquery before, which is a thing

35:31

happened on the page. And because

35:33

that thing happened, I mean, Jesse,

35:35

talking about event-based architecture here, come

35:37

on, this is exactly what JavaScript

35:39

is, right? And JQI especially, right?

35:41

Stimulus is exactly that. Event happens,

35:43

do something else because this thing

35:45

happened, right? Just saying, if you

35:47

love that event-based architecture style, stimulus

35:49

is that. It's just way prettier.

35:51

And I found it a lot

35:53

easier to use. And because I

35:56

don't like dollar signs, I was

35:58

happy about it. Yeah, look, if

36:00

you want a bunch of different

36:02

stimulus JS tutorials, check out. Drrifter

36:04

Ruby, man. I have a whole

36:06

bunch on there. It's a big

36:08

plug episode. Everyone quickly plugged their

36:10

thing. Do you know what? Actually,

36:12

Dave, I have already checked it

36:14

out and I still don't understand

36:16

it. So I need to check

36:18

it out again. It's definitely me.

36:20

I tell you what a problem

36:22

is, right? I've been leaning on

36:24

J query for so long and

36:26

I'm talking about 10 years, right?

36:28

and it's not just that I've

36:30

got my whole arsenal of weird

36:32

front-end stuff that I can pull

36:34

in and replacing that big long

36:36

list of handy widgets I know

36:38

will solve this problem is the

36:40

is what I'm lacking. So yeah

36:42

basic gum stuff fine you know

36:44

yes six or a way but

36:46

if I want to do something

36:48

weird if I want to do

36:51

something fancy The whole point of

36:53

stimulus, this doesn't have weird fancy

36:55

stuff, it's clean. So, so now

36:57

that we've decided that Jaycaree is

36:59

dying and that hopefully our morning

37:01

period is almost over. It is

37:03

dead. I know it's dead. I'm

37:05

just finding it hard to cope.

37:07

Okay, fair. So Dave, you earlier

37:09

had me a little bit on

37:11

this train and I kind of

37:13

wanted to go back to it

37:15

or whatever, talking about like changing

37:17

multiple things. Because there's so many

37:19

places where we can find ourselves

37:21

tricked or just like, I don't

37:23

know, seduced into it, right? Where

37:25

it just seems like the right

37:27

thing to do. So my, the

37:29

thing that like, I like to

37:31

tell people, because it made total

37:33

sense to me when I first

37:35

heard it, was, you know, Kemp

37:37

Beck's like, first you make the

37:39

change easy, then you go and

37:41

make the easy change, right? And

37:43

the important thing about that, right,

37:46

right, right, there are steps here,

37:48

right. Dude, engineering is all about

37:50

taking those like tiny steps. And

37:52

like you said, testing, right? And

37:54

we all know what happens when

37:56

we change a bunch of things

37:58

at once. And then we all

38:00

do it. And then we. Yeah,

38:02

so we go down the road

38:04

of mistakes. Engineering is a, it

38:06

requires self-discipline and whenever we don't

38:08

have self-discipline or whenever we, you

38:10

know, just break it because for

38:12

human beings, that stuff happens. I

38:14

almost feel like it's, I go

38:16

into like meditative state almost, it's

38:18

kind of like the flow state

38:20

kind of, and there are times

38:22

where it's like, like, I pick

38:24

up the story and say, okay,

38:26

yeah, this is easy, I can

38:28

just kind of like half think

38:30

about it about it and do

38:32

it and do it. And when

38:34

that doesn't work, which it almost

38:36

never dies, I have to like

38:38

get into this like very focused,

38:40

I'm making this one change verifying

38:43

state. It's almost like the cartoon

38:45

avatar going into your avatar state,

38:47

if you if you guys know

38:49

that amazing cartoon. Yes, I'm familiar

38:51

with it. Luke, I highly recommend

38:53

it. What's it called? Avatar in

38:55

the last air vendor, I think

38:57

came out like 10 years ago,

38:59

15 years ago. I was the

39:01

guy with a face, the face

39:03

tattoo. Yeah, exactly. You know exactly

39:05

what... I will check it out.

39:07

I've seen that art too. He

39:09

looks like he's really angry. Exactly.

39:11

He's got no hair, right? Yeah,

39:13

Baldhead. Yep. That's the guy. Yeah,

39:15

that's the guy. Yeah, all right.

39:17

Yeah, that's the guy. Yeah, all

39:19

right. All right. I'm a huge

39:21

anime watch too. That's a dangerous

39:23

gap in my knowledge. Yes, so

39:25

anyway, it's like you go into

39:27

the state. the first. And it

39:29

and unfortunately for me maybe it's

39:31

because of my age or whatever

39:33

but it getting into that really

39:35

focused state actually that costs something

39:38

right it takes like a little

39:40

bit out you can't maintain that

39:42

state forever there's like a manopool

39:44

almost of how long you how

39:46

long for me I can be

39:48

in that extreme focused state and

39:50

I think that when there are

39:52

times where I look at a

39:54

problem I'm like do it like

39:56

I almost like myself do I

39:58

need to be in this highly

40:00

focused day to get this thing

40:02

accomplished. And more often than not,

40:04

answer is yes but my initial

40:06

answer is no and so I

40:08

end up wasting time by not

40:10

doing things as systematically as they

40:12

need to get done. Am I

40:14

alone on this? Nobody else goes

40:16

into Qatar State when they when

40:18

they code? Go ahead. I find

40:20

that yeah people talk about this

40:22

flow state and I was I

40:24

was not a believer for a

40:26

while until I got something really

40:28

hard to work on. you know,

40:30

and it's those problems where you're

40:33

at the kind of limit of

40:35

what you can do when someone,

40:37

you're thinking, can I actually write

40:39

this? That is when I find

40:41

you get these kind of periods

40:43

of intense concentration and obviously you

40:45

don't want to be, you know,

40:47

the edge of your ability all

40:49

the time, you want to kind

40:51

of line up the nice easy

40:53

jobs or things tend to go

40:55

disastrously wrong. But one thing I

40:57

have found is that, you know,

40:59

when you're doing these really hard

41:01

problems, and you're like, can we

41:03

do it? You know, is it

41:05

possible? You always have to come

41:07

back and redo it. Once you

41:09

prove it's possible, then you have

41:11

to hit it again, and then

41:13

you come up the one. So

41:15

all of the stuff I've done

41:17

in a kind of state-intensive concentration

41:19

tends to get thrown away. But

41:21

it moves you forward down the

41:23

old, down the road, down the

41:25

road. And it's totally cool to

41:27

write really terrible code during naive

41:30

implementation. That's not the point of

41:32

it. The point is to get

41:34

from having nothing to having a

41:36

working thing. The important thing is

41:38

that after your naive implementation you

41:40

decide, you know, decide whether it's

41:42

worth refactoring when you're going to

41:44

refactor all that kind of stuff.

41:46

Yeah, I love that idea of

41:48

a naive implementation. I think the...

41:50

ACCA microservice was our naive implementation.

41:52

We didn't consider the scale at

41:54

all. We didn't consider the makeup

41:56

of the team changing. And as

41:58

a result, when we went back

42:00

to fix the naive, right when

42:02

we wanted to get rid of

42:04

the double loop, so whatever. We

42:06

ended up moving to Java, so

42:08

from Copland to Java and from

42:10

ACCA to camel. And the reason

42:12

that those tools made sense is

42:14

because they were very common in

42:16

our company. And what we didn't

42:18

have to reinvent the wheel. We

42:20

weren't seeing errors for the first

42:22

time in the entire company. It

42:25

was sort of a knowledge base

42:27

to draw from. So the current

42:29

state is that we have. Java

42:31

Camel Spring microservice talking to the

42:33

rails monolith. It's very stable. It's

42:35

performing super well now. And yeah,

42:37

I mean, getting, I guess a

42:39

question I have is like, how

42:41

can we speed up the process

42:43

of getting from our naive solution

42:45

to the more learning solution? I

42:47

think in my opinion, if I

42:49

had an answer for that, I

42:51

would be a super wealthy person.

42:53

Because in my experience, usually what

42:55

happens is when you get to

42:57

the point that you're calling something

42:59

a knife solution, right? It's usually

43:01

because you recognize that there's a

43:03

problem with it, right? And one

43:05

of the things that's going on,

43:07

I feel like, is that you're

43:09

kind of burning yourself out on

43:11

the problem at that point. When

43:13

you sort of have this realization,

43:15

you're also at the point that

43:17

you're kind of burning yourself out

43:19

in this problem. So if you

43:22

as a... as an organization don't

43:24

have the resources to sort of

43:26

like swap out people, you know,

43:28

bring somebody in the fresh, things

43:30

like that, right, or kind of

43:32

go back to a huddle and,

43:34

you know, somehow become re-energized. Like

43:36

all the things that we can

43:38

do are almost all like social

43:40

interactions, not engineering interactions, right? And

43:42

so if you have a culture

43:44

where you can kind of deal

43:46

with that, I feel like people

43:48

can... kind of turn around and

43:50

refactor and make reasonable choices or

43:52

whatever but If you don't, like

43:54

I feel like it's really hard.

43:56

It's really hard for you as

43:58

somebody who just burned yourself out,

44:00

pushing something across the finish line

44:02

that was really hard, to then

44:04

turn back around and be like,

44:06

I'm throwing you out and I'm

44:08

gonna redo this really hard problem

44:10

again. Right? Like that's just a

44:12

really hard thing to do, I

44:14

think. It's interesting that you say

44:17

that because at the point that

44:19

we made that decision, the problem

44:21

didn't seem hard anymore. At the

44:23

point that we were like... you

44:25

know, we can just do this

44:27

in job and camel, we had

44:29

wrapped our heads around the problem

44:31

enough that we really felt like

44:33

it wasn't hard anymore. And maybe

44:35

that's what it takes. You also

44:37

said a really important word there,

44:39

which is we, right? So one

44:41

of the things that you did

44:43

as a company to recover from

44:45

this problem is you went back

44:47

to this huddle and you said,

44:49

hey, I made some mistakes. And

44:51

then your team said, that's okay,

44:53

Jesse. we as a team are

44:55

taking ownership for this right like

44:57

and we as a team are

44:59

going to fix this right like

45:01

that that's what a lot of

45:03

that communicates right here and when

45:05

you do that like that's one

45:07

of those re-energizing moments or you

45:09

know things and then those decisions

45:12

become a lot easier in my

45:14

experience it breaks my heart every

45:16

time I have a have a

45:18

get commit with a kind of

45:20

more lines removed than added oh

45:22

man those are the proudest days

45:24

right right I know some people

45:26

like to take the code out

45:28

and they're like, oh, I go

45:30

knocked out, you know, loads of

45:32

that repeated code. I just think,

45:34

why couldn't I get it right

45:36

the first time? That's interesting. The

45:38

person who taught me how to

45:40

code, one of the first pieces

45:42

of advice that he gave me

45:44

was that you should write a

45:46

lot of code and throw a

45:48

lot of code out. And that's

45:50

how you'll get better at coding.

45:52

That is very, very sensible advice.

45:54

Do I, the context is a

45:56

big mistake. Now, just to be

45:58

clear, this big mistake has had

46:00

a happy ending. So this isn't,

46:02

this isn't the kind of. a

46:04

big mistake I got fired. This

46:06

is a big mistake. We pulled

46:09

through it and it all worked

46:11

out. Am I right? Thankfully. Yeah.

46:13

So I think so we did

46:15

a couple of things I think

46:17

that helped us. So the first

46:19

is, as John said, we were

46:21

open about these things. We didn't

46:23

try to hide that things where

46:25

I'm going as well as we

46:27

had wanted them to. And I

46:29

think that I bought a has

46:31

a pretty strong culture in the

46:33

sense that We're not trying to

46:35

throw people under the bus in

46:37

engineering. If something crashes, it's not,

46:39

who's going to get fired. It's

46:41

like, okay, how do we learn

46:43

from this? This is a mistake

46:45

that costs us some money. How

46:47

do we make sure that money

46:49

is actually teaching us something? So

46:51

that was part of it. And

46:53

then I think also we did

46:55

a good job of communicating to

46:57

external stakeholders. We communicated to the

46:59

finance team who are kind of

47:01

one of the main, main consumers

47:04

of the data that we were

47:06

producing. and really went through in

47:08

detail. Here's what we're at, here's

47:10

the timeline, like updating them, we

47:12

were checking in with them all

47:14

the time, and just keeping expectations

47:16

in line I think really helped

47:18

us out. So even though we

47:20

delivered a little bit later than

47:22

I think we thought we would

47:24

at the onset of the project

47:26

because we were able to communicate

47:28

that, we were not fired. And

47:30

yeah, I mean, not only that.

47:32

We've hired more people, we're still

47:34

hiring, and if you're thinking about

47:36

getting into the mobile coupon space,

47:38

there's a ton of really cool

47:40

problems, even if you're not passionate

47:42

about mobile coupons, and you might

47:44

get to talk to me in

47:46

the interview process, which will be

47:48

fun. Speaking of interviews, I understand

47:50

that you have a project which

47:52

you call Mena Swan interviewing. That

47:54

is... I've no idea what Minnesuan

47:56

sounds, but I see a lot

47:59

of people saying it in the

48:01

Ruby community, but I just, just

48:03

not, it's one of these weird

48:05

in jokes I think they have

48:07

in the Ruby community. it's so

48:09

nice and so i know i

48:11

know i know try humor luke

48:13

never pick up on your sarcasm

48:15

good lord i'm just over here

48:17

like like dying everything and that's

48:19

the what was it Dave sorry

48:21

it was seriously yeah go on

48:23

Matt's it's nice and so are

48:25

we yeah what the best thing

48:27

about reviews community it's a it's

48:29

a really It's a really great

48:31

language and that's a really strong

48:33

committee, really great events, even though

48:35

obviously the events this year have

48:37

been a bit difficult. So how

48:39

are you carrying that culture, that

48:41

tradition over into the interviewing process?

48:43

What's your gambit? Yeah, so maybe

48:45

everybody has experience or a lot

48:47

of folks having experience in, let's

48:49

just say, a not mean us

48:51

one interview interview. an interview that

48:53

maybe is a little more hostile

48:56

than we'd like and I think

48:58

I think a lot of us

49:00

have experienced kind of broken interviews

49:02

where it feels more like the

49:04

person on the other side of

49:06

the table is trying to prove

49:08

how much smarter they are or

49:10

how much better they are coding

49:12

than I am. And that's not

49:14

nice. Another thing that's not nice

49:16

is asking someone to do an

49:18

inordinate amount of work outside of

49:20

work. that's not paid in the

49:22

form of like a take-home project.

49:24

So I've done take-home projects that

49:26

have taken me entire weekends, multiple

49:28

days, and that's uncompensated work, and

49:30

that can bias your process against

49:32

people who have outside of work

49:34

commitments like families, and just, you

49:36

know, who don't want to be

49:38

working all the time. So I

49:40

thought... it would make sense to

49:42

kind of take the best part

49:44

of the Ruby community, this idea

49:46

that Matt is nice and so

49:48

we are nice and apply it

49:51

to interviewing. Let's actually be nice

49:53

to the people that we potentially

49:55

could be working with. And the

49:57

pandemic has been terrible in so

49:59

many ways, but it did offer

50:01

us this opportunity to kind of

50:03

dramatically rethink what our interview was

50:05

going to look like, because we're

50:07

not coming into the office. Everything

50:09

has to be remote. And basically

50:11

our HR team and our leadership

50:13

are like, how can we do

50:15

this? We've only been accustomed to

50:17

bringing people in, asking them super

50:19

tricky things that they have to

50:21

wipe word. How can we translate

50:23

this? to a remote interview. And

50:25

this is what I proposed and

50:27

this is like what we landed

50:29

on, which is an interview not

50:31

meant to trick the interviewee. It's

50:33

an interview meant to simulate what

50:35

the first couple days of work

50:37

is going to look like. And

50:39

it's supposed to give us as

50:41

an organization a sense of how

50:43

much we would enjoy this person

50:46

as a colleague, how successful they'll

50:48

be. And the message that we're

50:50

always trying to send is not,

50:52

hey, I'm so much smarter than

50:54

you. because I understand recursion or

50:56

because I understand how to do

50:58

whatever this type of model this

51:00

type of data structure. The message

51:02

is you're going to be successful

51:04

on our team and you're going

51:06

to like working with us and

51:08

we're going to like working with

51:10

you. So the way that we

51:12

do that is basically by giving

51:14

the person a sample of code

51:16

from our domain and it's highly

51:18

simplified and it's not and we

51:20

ask them to just read the

51:22

code and we say what is

51:24

this doing. What do you like

51:26

about this code? What do you

51:28

not like about this code? And

51:30

it's not a bug finding adventure.

51:32

We're not asking them to find

51:34

where an error is going to

51:36

be secretly raised or why a

51:38

test is failing. That's kind of

51:40

beyond, like you can't really do

51:43

that in a 20 or 30

51:45

minute conversation. We want to hear

51:47

how this person would approach an

51:49

alien code base, which is what

51:51

their first task is going to

51:53

be on the job. Then we

51:55

present them with some data, you

51:57

know, some award events from our

51:59

from our system. And we asked

52:01

them to manipulate that data with

52:03

a pretty simple. You even tell

52:05

them what the algorithm is. And

52:07

we ask them to code it.

52:09

And we say specifically, we don't

52:11

care about the answer here. The

52:13

answer is not interesting. We just

52:15

told you what the answer is.

52:17

We actually want to see what

52:19

it looks like when you code.

52:21

What's your approach? Are you systematic?

52:23

Are you making guesses about what

52:25

should happen and checking yourself? Those

52:27

are things that we're looking for

52:29

and we're not looking for some

52:31

for to see. Do you know

52:33

this random algorithm from your... from

52:35

your computer science education that you'll

52:38

never use as a web developer

52:40

at Iban. So those are the

52:42

big pieces and I'm stoked because

52:44

I got invited to talk at

52:46

Ruby Conf, which is coming up

52:48

in November, where I'm going to

52:50

be kind of outlining this. You

52:52

all got a preview of the

52:54

content there, exclusive to Ruby Rhodes.

52:56

Awesome. I'm definitely curious, as you

52:58

guys implement this, like, when you

53:00

kind of come back and say,

53:02

well, this didn't work or this

53:04

is like, like, this is how

53:06

we sort of had to come

53:08

up with a way to, you

53:10

know, because this is a subjective

53:12

thing, this is how we came

53:14

up with a way to make

53:16

this more objective than it originally

53:18

was, you know, and that helped

53:20

us, like definitely interested in seeing

53:22

this. I mean, all of you,

53:24

all of the pain points that

53:26

you're talking about, right, all this

53:28

homework that we have, all these,

53:30

you know, coding challenges that we

53:32

do, I mean, we, we're very,

53:35

I feel like as a community,

53:37

we talk about how well, where

53:39

we're where we're where we are.

53:41

that they don't work. But we

53:43

don't have alternatives yet. And so

53:45

people continue to use them regardless

53:47

because they're like, well, I don't

53:49

know what to do. And people

53:51

who are lost, you know, just

53:53

run back to where they came

53:55

from a lot of times, right?

53:57

It's like the squirrel. It's like

53:59

in the middle of the street

54:01

and the cars coming. It's like,

54:03

shoot, I gotta go all the

54:05

way back. So I kind of

54:07

feel like we do a lot

54:09

of that still. Yeah, there have

54:11

been various things like, there have

54:13

been various things like. they have

54:15

you come and spend like I

54:17

remember if it was like a

54:19

day or whatever, or half day,

54:21

but like a lot of time

54:23

with them, pairing, right? I know

54:25

that boot camp does that as

54:27

well, or base camp, sorry, wow,

54:30

whatever. Base camp does that as

54:32

well, or something similar. Though, actually,

54:34

I remember, they had an article

54:36

about they did some homework or

54:38

something recently, whatever. I thought base

54:40

camp does that as well, or

54:42

something similar. So. I'm super interested

54:44

to see all this turns out.

54:46

Yeah, I mean, so far, you

54:48

know, obviously hiring has been slower

54:50

than typical for iBata, but we

54:52

are still hiring. And so far,

54:54

the feedback from candidates has been

54:56

really interesting because, you know, we'll

54:58

get unsolicited emails about how much

55:00

people like the process and how

55:02

they hope that this is the

55:04

direction that the industry goes in.

55:06

A lot of folks who are

55:08

interviewing with us are interviewing at

55:10

other places as well. And I

55:12

see this as like a competitive

55:14

advantage for us, right? Does this

55:16

buy us like $10,000 in salary

55:18

or $5,000 in salary or something

55:20

like that? Like, does this make

55:22

our company more, you know, once

55:25

you get to a certain point,

55:27

there's like diminishing returns on all

55:29

these different levers that a company

55:31

can offer to a developer? And

55:33

I think that this is maybe

55:35

an unexplored lever or a lever

55:37

where there's a ton of room

55:39

to gain. And when someone goes

55:41

through our process and comes out

55:43

feeling like... hey I'm gonna I'm

55:45

gonna kick butt there the people

55:47

there are super nice they didn't

55:49

make me feel like an idiot

55:51

and they contrast that with other

55:53

interview experiences they've recently had hopefully

55:55

that that will play in our

55:57

favor that is certainly not the

55:59

current approach to coding interviews I've

56:01

been hearing recently mill more people

56:03

talking about make how to make

56:05

coding interviews harder if you go

56:07

on something like hacker news you

56:09

know they discuss which framework they

56:11

need to insert under the fingernails

56:13

of potential applicants. here if you

56:15

go to Facebook now and apply

56:17

for a job then they you

56:19

have to bring along your PhD

56:22

in computer science they then burn

56:24

it in front of you mix

56:26

it into old coffee and make

56:28

you drink it as part of

56:30

the interview process so it's really

56:32

I think it's really something that

56:34

the tech community needs we have

56:36

been doing a few episodes about

56:38

trying to make community more inclusive

56:40

trying to bring in people trying

56:42

to keep people in and that

56:44

sounds like a really great idea

56:46

for a talk you know why

56:48

don't we be be the the

56:50

nicer to people trying to get

56:52

the most out of them during

56:54

interviews and sort of subjecting them

56:56

to torture. I love that image

56:58

of shoving a framework under someone's

57:00

fingernails luke, but the thing the

57:02

thing that I noticed and I've

57:04

done a lot of interviews I've

57:06

done I would say easily 200

57:08

interviews over the past two years

57:10

we do a ton of hiring

57:12

so it's crazy and the thing

57:14

that I noticed is that when

57:17

we were asking people to whiteboard,

57:19

right, to answer these kind of

57:21

tougher algorithm questions, I didn't always

57:23

see a one-to-one correlation between the

57:25

people who crushed that kind of

57:27

question and the people who I

57:29

really enjoy working with in either

57:31

direction, right? They're false positives and

57:33

false negatives. And I just found

57:35

there to be way too much

57:37

noise in that type of diagnostic

57:39

for me to really make a...

57:41

a strong decision that I felt

57:43

confident with that proved out over

57:45

time. And I'm hoping that this

57:47

is a remedy for that. Yeah,

57:49

I mean, you always have to

57:51

check to make sure that whatever

57:53

test you're giving, right, like is

57:55

that you have to decide what

57:57

you're testing for and you have

57:59

to say, is the question that

58:01

I'm asking actually testing for the

58:03

thing that I'm testing for? And

58:05

too often, we jump to this

58:07

conclusion that, oh, yeah, this sort

58:09

of this sort of is related,

58:12

therefore it'll let me know. You're

58:14

making a subjective judgment call again

58:16

and calling it. So, yep, it

58:18

is, it is a thing that's

58:20

very painful. And maybe, maybe we

58:22

should, we should focus a little

58:24

bit on this in the future

58:26

and have a more in-depth chat

58:28

about this. I feel like that

58:30

would be an interesting subject. Yeah,

58:32

there was a Jazz Comp talk

58:34

last year. I believe the guy's

58:36

name is Eric, I'll throw it

58:38

in the, in the chat, and

58:40

second, he's from Test Double. And

58:42

he talked about, Test Double, like,

58:44

like, a really consultancy, based on

58:46

Columbus, I believe. And he talked

58:48

about their hiring process and more

58:50

in the sense of our goal

58:52

is to build a process that

58:54

lets candidates show off their strengths

58:56

and as opposed to helping us

58:58

find their weaknesses. And that's how

59:00

I've really informed our process that

59:02

I bought it. We're trying to

59:04

find people's strengths and trying to

59:06

figure out where they're going to

59:09

be able to make the most

59:11

impact in our company. Yeah, I

59:13

have gone for the past life.

59:15

I don't know, four or five

59:17

rails comps. I've gone to basically

59:19

every single talk that was sort

59:21

of like related to the subject.

59:23

I do, I care about it

59:25

quite a bit and I just

59:27

kind of hope that we get

59:29

to a better place. It's, yeah,

59:31

like I said earlier, my feelings

59:33

on this I think are basically

59:35

what I said earlier that I

59:37

think that a lot of people

59:39

are thinking about this. And I

59:41

just don't think there's a clear

59:43

answer. I think we're going to

59:45

get. a little better. Yeah. Well,

59:47

hey, looks like we're coming up

59:49

on the hour, so we need

59:51

to move things along. Jesse, if

59:53

people want to get in touch

59:55

with you, where should they go

59:57

in line and look? I would

59:59

say you can find me on

1:00:01

Twitter. I tweet from planet efficacy.

1:00:04

So if you can spell that,

1:00:06

you can find me. And you'll

1:00:08

find a lot of interesting ruby

1:00:10

contents, political outrage, and Vermont's progressive

1:00:12

rock. on that Twitter stream. Awesome.

1:00:14

Well, let's go ahead and move

1:00:16

over into. some picks look you

1:00:18

want to kick us off I

1:00:20

would my first pick is a

1:00:22

version of Windows 10 a bit

1:00:24

of a strange thing to pick

1:00:26

on a three rubs poke files

1:00:28

but this is called Windows 10

1:00:30

a mealyarated edition I found it

1:00:32

featured on the popular YouTube channel

1:00:34

a lot long ago and this

1:00:36

is a version of Windows temple

1:00:38

to spyware taken out which also

1:00:40

incredibly boosts the responsibility and performance

1:00:42

because it's not doing any Asink

1:00:44

network calls back home in the

1:00:46

background if you know what I

1:00:48

mean. This is the first operating

1:00:50

system I've ever found where to

1:00:52

download it I had to get

1:00:54

a bit torrent link from a

1:00:56

telegram channel. So if you want

1:00:58

some excitement in your operating systems

1:01:01

check out Windows 10. an ameliorated

1:01:03

edition. Never seen anything like it.

1:01:05

I just wanted to confirm something

1:01:07

really quick, Luke. Is this a

1:01:09

legal version of Windows 10? It's

1:01:11

got quite a big section on

1:01:13

Legality on the website. Okay. The

1:01:15

website does make it clear that

1:01:17

you do need to have a

1:01:19

Windows 10 license in order to

1:01:21

legally use the Windows 10 ameliorated

1:01:23

edition. I mean, come on, Pitorant

1:01:25

Link through a telegram channel. Wow!

1:01:27

That's quite a... Anyway, it's quite

1:01:29

troubling. It's a cool little project.

1:01:31

There's lots in the FAQ about

1:01:33

their rationale behind it, and it

1:01:35

does fly when it runs. It's

1:01:37

really quick. I heard there were

1:01:39

some issues with it, with not

1:01:41

being able to do certain things,

1:01:43

because it can't call home to

1:01:45

Microsoft. Even some like... simple rudimentary

1:01:47

things that you would think that

1:01:49

would be possible but just kind

1:01:51

of breaks it. Yeah, I mean

1:01:53

you could call it Windows 10

1:01:56

lobotomized. I mean there's barely anything

1:01:58

there at all, but it's it's

1:02:00

Oh, it's a lot of fun.

1:02:02

It flies doing nothing. Awesome. Well,

1:02:04

John, you wanna do some picks?

1:02:06

Yeah, so I have a different

1:02:08

pick this week too. You may,

1:02:10

by the time this comes out,

1:02:12

I have no idea if people

1:02:14

are even gonna still be playing

1:02:16

this game, but I have been

1:02:18

playing and streaming and absolutely having

1:02:20

a blast playing. I guess that's

1:02:22

redundant among us. So if you're not

1:02:24

familiar with it. If you're familiar

1:02:26

with like We're Wolf or mafia

1:02:28

or kind of games in that

1:02:30

nature where you there's another game that

1:02:32

people have like compared this to or

1:02:35

whatever where you like sort of have

1:02:37

like like townsfolk and then you have

1:02:39

like people that are pretending to be

1:02:41

townsfolk that are like trying to kill

1:02:44

everybody off and your goal is the

1:02:46

townsfolk or to try and find the

1:02:48

one in this game the imposters among

1:02:50

you. So this is like set in

1:02:52

space or whatever. But yeah. It's an

1:02:55

absolute blast. I really am not sure

1:02:57

what to say about it other than

1:02:59

it's like kind of like a great

1:03:01

social game. Yeah, it's it's pretty

1:03:04

awesome. So I highly recommend checking

1:03:06

it out. I definitely have been

1:03:08

doing a lot of streaming of

1:03:10

it lately. So yeah. I'll jump

1:03:13

in with a few picks. First

1:03:15

pick. bamboo flooring. So I love

1:03:17

bamboo and instead of the crummy

1:03:19

old chair mat, the little plastic,

1:03:21

thin plastic that I was using

1:03:23

for a long time, I finally

1:03:25

went to Home Depot and got

1:03:27

some bamboo flooring and I put

1:03:29

glued that to a piece of

1:03:32

plywood and I now use that

1:03:34

as my floor mat on my

1:03:36

carpeted floor. So my chair can just

1:03:38

slide around on it. It's really

1:03:40

really cool. And it's been a

1:03:43

life changing event here. So it's

1:03:45

pretty awesome. And the second thing

1:03:47

that I pick is the Elgato

1:03:49

Key Light. So I got those

1:03:51

for my streaming set up and

1:03:53

they make a huge difference. So

1:03:55

you won't be able to tell

1:03:57

on the podcast, but this is with.

1:04:00

lights off makes a really big difference

1:04:02

in quality. So having proper lighting on

1:04:04

doing any kind of video work is

1:04:06

an absolute must. Can you do that

1:04:08

again, Dave, just so everyone can see

1:04:11

on the podcast? Yeah, I gotta say

1:04:13

that is quite striking. Is that the

1:04:15

same company that did the shortcut bar?

1:04:17

Yes, the stream deck. Yeah, that's a

1:04:20

pretty cool thing as well. Big Elgato

1:04:22

fan. And Jesse, do you want to

1:04:24

jump in with some picks? Yeah,

1:04:26

I've got a couple of bits,

1:04:28

different categories. So first is a

1:04:30

talk that I watched recently

1:04:33

that maybe you all have seen

1:04:35

already, but I recommend revisiting

1:04:37

it. So Sandy Metz is

1:04:39

Rails' 2019 keynote, which is called

1:04:42

Lucky You. And I think it's

1:04:44

especially timely at the as

1:04:46

we approach election season and

1:04:48

thinking about. equality and

1:04:50

issues of inequality in our country.

1:04:52

So I highly recommend that talk.

1:04:54

It's amazing as all Sandy Met's

1:04:56

talks tend to be. Then I

1:04:58

have, when the pandemic started, we

1:05:00

started working remote, I made a

1:05:03

small investment in my work from

1:05:05

home setup that I highly highly

1:05:07

highly recommend. I went out

1:05:09

and I got an ergodox split

1:05:11

keyboard, having some wrist pain on the

1:05:14

keyboard I was using. So I know keyboards

1:05:16

can be a whole other podcast,

1:05:18

but Check out Urbodox, they make

1:05:20

a really really really good product

1:05:22

that's worth the price and I've

1:05:24

been a huge fan of it

1:05:26

since getting it. And then my

1:05:28

final pick, there's a book that

1:05:30

just came out, it's a guilty pleasure,

1:05:32

it's called The Trouble with Peace

1:05:34

by Joe Abercrombie, a British gentleman,

1:05:37

and it is in the grim

1:05:39

dark genre of fantasy literature

1:05:41

and I highly recommend it. Awesome.

1:05:43

Well, thank you for those picks and just

1:05:46

be sure to post them into our chat

1:05:48

section here so we can include them on

1:05:50

the show notes. Well, Jesse, thank you for

1:05:52

coming on today. It was a lot of

1:05:54

fun. I love these kind of talks where

1:05:56

we can just humble ourselves and talk about

1:05:59

past mistakes. and you know, what we

1:06:01

know, what we learned from them.

1:06:03

again. This again. Really interesting. to a long-time listener,

1:06:05

first-time guest, and this was this was, this

1:06:07

was awesome. Yeah, I appreciate it.

1:06:09

for Thank you for coming. all

1:06:12

Well, that's all for this episode

1:06:14

everyone. Thanks for listening. Take

1:06:16

care everybody.