0:00

for the Ruby on Rails podcast is

0:02

provided by fireside .fm. Are you

0:04

thinking of starting a podcast? Head

0:06

to fireside .fm to get started. You're

0:11

listening to the Ruby on

0:13

Rails podcast. Happy

0:17

Wednesday and welcome to the show. I'm your host,

0:19

Elise Schaefer, and I use she, her pronouns. Today,

0:21

we're continuing our series of highlighting

0:23

startups building with Rails. Every app

0:26

eventually runs into scaling requirements, whether that's

0:28

from a spike in traffic or persistent growth

0:30

or any of a number of reasons. That's

0:33

why we have autoscaling so that

0:35

we can automatically adjust to new

0:37

traffic. JudoScale is a

0:39

tool that makes autoscaling easy.

0:41

And today's guest is Adam McCrae,

0:43

who uses his hymn pronouns. Adam

0:46

has been a Rails developer since

0:48

2007. He started JudoScale in 2016

0:50

as a side project and went

0:52

full time on it in 2021.

0:55

Welcome to the show. Hi, Elise. Thanks

0:57

for having me. Thank you so much

0:59

for being here. I am excited about

1:01

this one because I feel like scaling is

1:03

the problem that everybody deals with and

1:05

everybody talks about, right? So this is

1:07

going to be a pretty interesting

1:09

conversation. You've been

1:11

using Rails since 2007. Can you kind of

1:13

give a little bit about how you

1:15

got introduced to Rails and what your journey

1:18

was like there? Absolutely. I've

1:20

been programming professionally since 2001.

1:22

and i started my career

1:24

in corporate it actually doing c

1:27

and then when i moved into web stuff

1:29

i was doing classic asp. And

1:31

i was happy to

1:33

be programming for a living but i really

1:35

did not enjoy the corporate environment i

1:37

didn't enjoy the technology that we were working

1:39

with. And i

1:42

think my first introduction to rails might

1:44

have been the getting real book from

1:46

thirty seven signals and i became sort of

1:48

obsessed with that book and. That

1:50

was when I was like, okay, I need to get

1:53

out of corporate IT. I need to get to do

1:55

a smaller company who works faster, works with better tools. And

1:57

I started playing with Rails at that point

1:59

as well. So that was

2:01

2006, 2007. And I

2:04

got my first Rails job, a

2:06

full -time Rails job in

2:08

2007 at an agency called Edgecase. And

2:11

I was there for six years, left

2:14

the consulting world and worked for a few

2:16

different product companies. Yeah,

2:18

I eventually found myself building

2:21

JudoScale as a side project while at one of

2:23

those product companies. Let's

2:25

talk about JudoScale. What was

2:27

the inspiration or impetus for building

2:29

JudoScale and building a custom auto -scale

2:31

tool? The company I was working

2:33

for at the time, we were

2:35

called Real HQ. Now it's called Agent

2:37

Pronto. It was basically a lead generation service

2:39

for real estate agents. We

2:41

were a small company, a small dev team. There

2:43

were just three of us on the dev team. I

2:47

had just recently joined and spearheaded

2:49

a move of our hosting from Engine

2:51

Yard to Heroku, which

2:54

was great. It worked great. It saved us a ton of money. But

2:56

we were still spending a lot

2:58

more than we needed to. We

3:00

were still scaled a lot higher than we

3:03

needed to be 24 hours a day, seven

3:05

days a week for an app that

3:07

was primarily used during business hours. So

3:10

we knew we wanted some kind of auto scaling. were

3:14

a couple options, but not many options

3:16

at that time. And we tried

3:18

them and just weren't very happy with them.

3:20

They were hard to use. They

3:23

didn't work very reliably. And

3:25

like we do as developers, I

3:27

had the notion that I could build a

3:30

better one and it didn't seem very hard.

3:32

So I asked my boss at the time,

3:34

if I build this thing as a

3:36

side project, would it be cool with

3:38

you? And would you be interested in us using it?

3:40

And like I said, we were a team of three,

3:42

very casual, very informal. He was like, yeah, absolutely, go for

3:44

it. So that's when I

3:46

started just sort of hacking on this

3:48

idea of a Heroku Autoscaler as

3:51

a little side project, which the

3:53

first version of which it was

3:55

truly, it was not even an MVP. It

3:57

was not viable. It had no

3:59

UI. The configuration was all done

4:01

through a production Rails console. It

4:04

was built for specifically one

4:06

app. Other customers, other clients couldn't

4:08

use it. So it was.

4:10

truly just a thrown together thing

4:12

to see if my concept

4:14

of an autoscaler could

4:16

work better than what we

4:18

had played with on the market. And that

4:20

worked great for us. So it kind of went from there.

4:23

In kind of doing a little bit

4:25

of research for this episode, I saw on

4:27

the main page of JudoScale, you talk

4:29

about how a lot of autoscaling is based

4:31

on like memory or CPU utilization. Can

4:34

you talk about what JudoScale does differently in that

4:36

regard? Yeah, absolutely. So yeah,

4:38

you're going to find when

4:40

you look at platform native auto scaling,

4:42

like the auto scaler that comes with Heroku

4:44

or the auto scaling that comes with a

4:47

platform like render or even

4:49

AWS, you're typically

4:51

going to find one of two things. It's

4:53

either going to be compute based auto scaling,

4:55

which is usually just CPU or on the

4:58

Heroku side, what they give you

5:00

is response time based auto scaling.

5:02

And each of those

5:04

have their own complications for response time.

5:07

The biggest problem is Every

5:09

app has slow endpoints and fast

5:11

endpoints. And if you're just autoscaling based

5:13

on response time, your slow

5:15

endpoints are slow no matter whether

5:17

your servers are at capacity or not.

5:19

And that can trigger autoscaling. And it's

5:21

just impossible to dial that in

5:23

effectively. And with

5:25

CPU, CPU can work

5:28

OK on web services. It does not

5:30

work great at all when you're trying

5:32

to autoscale. worker services, it's really

5:34

easy to get queue backups, even

5:36

when CPU is relatively low because

5:38

you have jobs that are IO

5:40

heavy or it's just your configuration,

5:42

you've hit your thread limit or whatever. So

5:45

yeah, so what JudoScale does differently,

5:47

we, on the website, we autoscale based

5:49

on request queue time, which is basically

5:51

the time from when a request

5:53

hits a load balancer or Heroku's

5:56

router till the time it gets picked up

5:58

by your Rails application. So it's not impacted by

6:00

how long the request takes within Rails

6:02

It's just how long requests wait before

6:04

they get picked up. And

6:06

that's the number that corresponds directly

6:08

with how strained your capacity

6:10

is. As you approach capacity, your

6:13

request queue time starts to

6:15

increase. On the worker

6:17

side, we do something similar, but instead of request

6:19

queue time, we're looking at job queue time

6:21

or queue latency. So if you

6:23

imagine your sidekick dashboard where you see the latency

6:25

of each of your queues, we're basically looking

6:27

at the same thing. and autoscaling based on that

6:29

so that you can say things like, well,

6:31

I don't want jobs in this queue to wait

6:33

longer than five seconds before they're picked up.

6:35

And we can autoscale based on that metric. It

6:39

seems kind of intuitively simple as

6:41

a thing, right? Like, I don't want

6:43

this to sit longer than this

6:45

before it's picked up, or I

6:47

don't want this request to have latency

6:49

higher than X, right? What

6:52

I'm curious about is, were there

6:54

things that were challenging about doing

6:56

it that way versus trying to

6:58

do things based on CPU memory

7:00

or why are the platform native

7:02

autoscalers not trying to do the

7:05

same thing? Yeah, there's two

7:07

things. One, the platform

7:09

native autoscalers, by nature, they

7:11

have to build a general

7:13

purpose solution that can work

7:15

for any language, any framework. So

7:18

they have to sit at a

7:20

higher level. Whereas when you're trying to

7:22

measure things like job queue latency,

7:25

for instance, That is tightly coupled to

7:27

a framework so we have adapters

7:29

that we've built for sidekick for solid

7:31

queue for delayed job for all

7:33

these job back ends. The

7:35

platform owners don't want to build that

7:37

stuff they want to build general purpose

7:39

solutions. And likewise with

7:41

request queue time that's something that

7:44

you have to measure within the

7:46

rails process so we actually have

7:48

a middleware that measures that and

7:50

reports it so yeah to use

7:52

these kinds of metrics. You

7:55

have to get into the frameworks. You

7:57

have to actually build those integrations, which the

7:59

platform owners don't want to do. When

8:01

we accepted that, yeah, we're going to build a

8:03

Ruby gem that captures these metrics, that

8:05

part was not hard at all. I mean,

8:07

capturing the metrics is easy. Reporting the metrics

8:09

is easy. The hard part

8:12

just came on the back end, just like,

8:14

okay, what do we do with these? How do

8:16

we aggregate them? How do we decide when

8:18

to scale up, when to scale down? People

8:21

often think because I run JudoScale, I'm a

8:23

DevOps guy. I'm not really a DevOps guy. I

8:25

don't know much about DevOps when it comes

8:27

down to it. JudoScale is

8:29

more of a data science problem than it

8:31

is a DevOps problem. And if I'm

8:33

being honest, I'm not a data science person

8:35

at all. I've been kind of faking

8:37

my way through it for years now, but

8:39

getting the job done. That's

8:41

kind of an interesting way to put

8:43

it. I know enough DevOps stuff to

8:45

break a bunch of stuff, basically. Okay,

8:48

so you did mention building a Ruby gem

8:50

to collect all these metrics and that kind of

8:53

is maybe a good way to pivot into

8:55

discussion about how you all are using Ruby and

8:57

Rails. My main question is,

8:59

I think if you went

9:01

to someone and said, hey, we're

9:03

gonna build auto scaling tooling

9:05

in Ruby, someone might

9:07

say, well, wouldn't it be better

9:09

to use Go or something?

9:11

And I guess I'm curious if

9:13

you have any thoughts there

9:15

about why Ruby. and why

9:18

rails. You might think that if you're

9:20

thinking that autoscaling is like a low -level

9:22

problem where it needs to be ultra

9:24

-fast or anything, but at the end

9:26

of the day, JudoScale doesn't work much

9:28

differently than other APM or analytics tools.

9:30

We capture a few metrics, not nearly

9:33

as many as APM tools collect. We

9:35

just capture a small handful of metrics

9:37

within the app and report them to

9:39

our back end, which is all rails.

9:41

It's just a rails monolith. And

9:44

from there, most of the heavy lifting on

9:46

the analytics side is

9:48

in our database. And we

9:50

use timescale DB, which

9:52

is built on top of Postgres with

9:54

some time series capabilities built into it.

9:56

And so it automatically aggregates the data

9:58

for us and rolls it up and

10:00

lets us handle retention. So we don't need

10:02

to worry about holding on to large

10:05

amounts of data. It handles a lot of

10:07

that heavy lifting for us. So

10:09

at the end of the

10:11

day, it's a fairly simple

10:13

Rails application with a fairly

10:15

simple Rails gem that sends

10:17

metrics to it. Do you

10:19

support other frameworks other than

10:21

Rails? We do.

10:23

On the Ruby side, we support Sinatra

10:25

and just straight rack applications in

10:27

addition to Rails. And then

10:30

we also have integrations with Python

10:32

and Node. So on Python, you

10:34

do support Django and Flask and

10:36

Fastify and some of their background

10:38

job frameworks. I would say probably

10:40

90 % of our business is still Rails. And

10:42

maybe that's for a couple of reasons. One

10:44

is that we were initially called Rails Auto Scale,

10:46

not Judo Scale. Okay. Because when I initially

10:48

built it, I did not have aspirations to expand

10:50

beyond Rails. I'm like, I'm all Rails all

10:52

the time. I am a Rails

10:54

developer and have been in the Rails community

10:57

for years. So it's a lot easier for

10:59

me to connect with the community and get

11:01

the word out there. I don't

11:03

know Python well, don't know Node that well,

11:05

not involved in those communities. So it's a lot

11:07

harder to get traction there. Is it a

11:09

similar thing where you're working inside the language, like

11:11

kind of more at a middleware or framework

11:13

level to get the match? I'm assuming it's like

11:15

the same thing across all of those. Yeah,

11:18

yeah. I mean, the patterns are exactly the same.

11:20

It's just the hooks into the frameworks are

11:22

different as all. You said that

11:24

you support other rack applications, Sinatra, that kind

11:26

of stuff. Does it work

11:28

for any rack application? Like would it

11:30

work for different web framework built on

11:32

top of rack? Yes, it just

11:34

wouldn't be quite as automatic. So with Rails,

11:36

we're able to use rail ties. you install

11:39

the gem and you don't have to do

11:41

anything. The rail tie inserts the middleware and

11:43

you don't have to think about it. If

11:45

you're using it outside of Rails for another

11:47

rack -based application, you just have to manually insert

11:49

that middleware, but that's it and then it

11:51

will just work. Okay, awesome,

11:54

awesome. So what features

11:56

of Rails are you leveraging and

11:58

is there any that you

12:00

particularly found helpful with building JudoScale?

12:03

It's hard to point to a single

12:05

feature. What really

12:08

enabled me to build JudoScale

12:10

as a side project and

12:12

grow it as a business

12:14

was really just the whole

12:16

ethos of Rails as the

12:18

one person framework. I

12:20

think what I've been able to do with JudoScale

12:22

is proof that it works. I'm not

12:24

one person anymore. I do have a full -time

12:27

developer. We're a whole team of two. That

12:29

was just last year that I

12:31

hired him and So yeah, it

12:33

has been just me for the

12:35

vast majority of the product's existence.

12:38

And honestly, it still could be just me if

12:40

I wanted it to be, but I enjoy

12:42

working with somebody else. So yeah, Rails is the

12:44

one person framework. It really can be done. And

12:47

I love that about it. This

12:49

is maybe an interesting thing to dive

12:51

a little deeper on because I think

12:53

that that's one of the things that

12:56

I'm trying to highlight in this series

12:58

of interviews is that when we think

13:00

of Rails, we think of GitHub of

13:02

Shopify of 37. So we think of

13:04

these big companies who started small and

13:06

then became these big companies. But there

13:08

really is a push from DHH and

13:10

from the Rails Foundation to highlight that

13:12

this is a one -person framework. So

13:15

maybe let's spend a little bit of

13:17

time talking about that because I feel like

13:19

we could harp on that for a

13:21

while and not run out of steam really.

13:24

I am curious when you were starting to

13:26

build judo scale and starting to think

13:28

of, oh, how is this going to work?

13:31

Were you just working by yourself on weekends to

13:33

try and get it done? You said you

13:35

were working at a separate company and trying to

13:37

solve a particular problem that that company had.

13:39

Was it part of that time that you spent

13:41

there? No, it was not part of that

13:43

time. It was evenings and weekends for five years,

13:45

really. I went full time on judo scale

13:47

in 2021. But I don't want

13:49

to make it sound like I was up until

13:51

2am every night and working every weekend. I was

13:53

really able to pace

13:55

myself in a way that

13:57

it was enjoyable and not

13:59

super stressful. The

14:01

initial version, like I said, was just for

14:03

us internally. And then I kind of

14:06

did the bare minimum to stick a UI on

14:08

it and allow people to sign up for it

14:10

and stick it in the Heroku marketplace. And

14:12

once it was there, I was just

14:14

kind of slowly adding features to it

14:16

and slowly growing the product. So yeah,

14:18

it was evenings and weekends, but not

14:21

at an unsustainable level, I don't think.

14:23

So this is one thing that I think

14:25

is very interesting about Rails, even when

14:27

I was learning it back in 2010. I

14:30

was surprised at how easy it

14:32

was to just pick it up because

14:34

of all of these little niceties

14:36

that made it easier to hold it

14:38

conceptually in my head, right? I've

14:41

said this on the show and I know a

14:43

ton of other people have said the same thing

14:45

that you could almost guess the method names and

14:47

the parameter order. And you'd be

14:49

right 80 % of the time or

14:51

something, right? So it

14:53

is good to hear that for

14:55

you, you're building it on your

14:57

own, but you weren't burning yourself

14:59

out to build it, which I

15:01

think is like an important thing

15:03

to maybe call out because Rails

15:05

gives us a lot of power

15:07

with relatively low like mental load,

15:09

I guess is the way to

15:11

put it. Yeah, and I think

15:13

especially if you keep your front

15:15

end simple, I think that's where

15:17

Rails applications can get out of

15:19

hand. For us, we don't use

15:22

any front end framework. We

15:24

don't use the full hot wire stack.

15:26

We do use turbo. We use AlpineJS

15:28

instead of Simulus, but they're conceptually similar.

15:31

And I think that's a huge

15:33

part of it. That's a huge

15:35

part of embracing the one person

15:38

framework is keeping the front end

15:40

part of things simple. You

15:42

mentioned learning Rails in 2010, how

15:44

smooth it was to get started. I

15:46

felt the same way in 2006,

15:48

2007. I feel terrible for anybody who

15:50

had to try to pick up rails during

15:52

the webpacker era. Oh my gosh, yeah. Fortunately,

15:55

we were both already part of it by

15:57

that point and now we've ridden through it and

16:00

we're on the other side. But

16:02

yeah, there were times where it had

16:04

to have been much harder to get

16:06

started with it. But I think we're

16:08

at a point now, I think we're

16:10

in the best era ever in terms

16:12

of rails being accessible for someone new

16:14

stepping into it and embracing it as

16:16

that one person framework. Yeah,

16:18

I definitely agree. All of the new

16:20

features of the framework in the last couple

16:22

of years, for a while, especially during

16:25

the Webpacker era, it seemed like every Rails

16:27

company was an API -only backend and then

16:29

a React single -page application on the front

16:31

end. And you'd have to have two

16:33

separate teams to manage each side of that.

16:35

And now it really feels like we're

16:37

kind of returning to the good old days

16:39

of full stack Rails development where you

16:41

can really get very far. Just with what's

16:44

in the box and you don't have

16:46

to kind of deal with all of this

16:48

complexity and build tooling for your JavaScript,

16:50

which personally I like a lot. I will

16:52

say one personal caveat. I've been on

16:54

the import maps train for a couple years

16:56

now, but I think if I were

16:58

to start new today, I probably would not

17:00

recommend someone use import maps. I feel

17:03

like every time I need to bring in

17:05

a third party library, I'm jumping through

17:07

hoops trying to understand how to make it

17:09

work with import maps and I'd much

17:11

rather just be able to. follow

17:13

the instructions on whatever third party

17:15

library does to import the quote

17:17

unquote normal way using a lightweight

17:20

build process. But yeah, that's one

17:22

place where I would maybe recommend

17:24

diverging from the golden path. But

17:26

yeah, that's one small thing. I

17:29

haven't really used the import map stuff

17:31

yet. In a lot of ways, all the

17:33

new stuff in Rails excites me. But

17:35

then in another way, I haven't really gotten

17:37

to use it outside of just playing

17:39

with it for a couple hours on a

17:41

weekend or whatever. So that is

17:43

interesting to know. Is it just that figuring

17:45

out how to get them and pin them

17:47

from some JS deliver or something? Is it

17:49

just that managing that ends up being a

17:52

chore? Yeah, it's figuring out the

17:54

right way to pin them and then the right

17:56

way to import them after they're pinned. And

17:58

that sounds simple. That sounds like two things that

18:00

a smart developer who's been doing this for

18:02

a while should be able to figure out, but

18:04

I have banged my head against that too

18:06

many times. Alright,

18:08

let's talk about deploying code without

18:10

having a full -blown panic attack. You

18:13

ever push something live and immediately regret

18:15

it? Like, oh no, I just nuked the

18:17

homepage. Well guess what? Flippers got your

18:19

back. Ship your code whenever you want,

18:21

roll out features on your schedule, and if

18:23

something goes sideways, boom, roll

18:25

it back. No redeploy, no

18:27

drama. Plus, you get multi -environment sync,

18:29

audit history, and fine -grain permissions

18:32

so you don't wake up to

18:34

a WTF -happened Slack message. Best part,

18:36

sign up at flippercloud .io slash

18:38

rails and get 10 % off your

18:40

first year. Flip the switch and

18:42

you can chill out. I

18:45

feel like there's always something,

18:47

right? Like back in 2010, it

18:49

was having to differentiate between

18:51

different versions of Internet Explorer and

18:53

different CSS things and prototype

18:55

versus jQuery. Like it was a

18:57

whole different set of things

19:00

and then we just, it's always

19:02

something. One thing that

19:04

I am curious about is With

19:06

Rails, I imagine that being

19:08

inside the framework helps JudoScale, as

19:10

you've said, do a whole

19:12

bunch of metrics that the platform

19:14

vendors can't. You said that

19:16

it was pretty trivial to get

19:18

those metrics. Is there anything

19:20

about doing what you're doing that

19:22

was difficult or hard to

19:24

accomplish? The biggest challenges for me

19:26

was when I stepped out of Ruby

19:28

and needed to build that Python adapter and

19:30

another adapter. But no, on the

19:33

Ruby side, collecting the metrics,

19:35

was not difficult. Yeah,

19:37

I think I mentioned this a little

19:39

bit, but figuring out the best way

19:41

to aggregate the data and the actual

19:43

autoscaling algorithm itself, that definitely took a

19:45

lot of iteration. And

19:47

we're still iterating on it, honestly.

19:49

We're still trying to find ways

19:51

to do it better. Even now,

19:53

we're exploring some additional metrics to

19:55

collect, because request queue time, even

19:58

while it is much better than

20:00

what we see in the alternatives,

20:02

it's still a reactive metric. Request

20:04

queue time is only going to

20:06

start to increase as you approach

20:08

kind of a capacity point. And

20:10

we definitely have some customers who want

20:12

to be more proactive and say, I

20:14

only want to be at 50 %

20:16

utilization so that when we get a

20:18

huge spike, we can ride

20:20

that spike without waiting for a

20:23

reactive autoscaling because our autoscaler is

20:25

fast. You can only be as

20:27

fast as the platform can provision

20:29

new dinos or new instances or

20:31

whatever. So we definitely have some

20:33

customers with spikes in traffic that

20:36

are faster than even JudoScale can

20:38

handle right now. So we're always

20:40

exploring better ways of doing this.

20:43

So when you configure JudoScale, like

20:45

let's say I've got a

20:47

Rails app and I've got JudoScale

20:49

attached and it's running in

20:51

the Rails app and we're on

20:53

a Dino serving responding to

20:55

a web request. So

20:57

if we're on that worker and

20:59

we get a request and we realize

21:02

we need to make some sort

21:04

of auto -scaling thing. Is there a

21:06

separate dino or pod or whatever running

21:08

that's like, oh, I got to

21:10

communicate to our platform vendor to tell

21:12

them, hey, scale this up? Or

21:14

is that happening from within that same

21:16

worker that's responding to the web

21:18

request? Yeah, the flow looks like this.

21:20

So we capture the queue time

21:22

for each request as it comes in

21:24

in that middleware. And then

21:26

we just have a separate

21:28

thread. that's just running

21:31

a loop and every 10 seconds

21:33

it ships off those metrics to

21:35

Judo scale. So then on the

21:37

Judo scale side, on the server

21:39

side, we're capturing those metrics, running

21:41

some analytics and deciding, hey, based

21:43

on the configuration that this customer

21:45

has set up, do we need

21:47

to scale them up? Do we

21:49

need to scale them down? And

21:51

then we just hit that platform's

21:53

API, whether it's Heroku's API or

21:55

AWS or whatever. So building

21:57

this yourself, I am curious, Is there anything

21:59

that you would do differently this time?

22:01

Like if you had to start over to

22:03

build something like this, would you do

22:05

something different? Would you? Well,

22:07

JudoScale has gone through a

22:09

lot of iterations because I

22:11

started it in 2016. So

22:13

it's seen coffee script. It's seen

22:16

Hamel. We weren't always on time scale.

22:18

It was originally just Postgres and

22:20

Redis with our homegrown sort of time

22:22

series stuff. And fortunately, because

22:25

it was just me, I was able to

22:27

make the decisions along the way to, you know,

22:29

I'm going to rewrite this part. I'm going

22:31

to tear this out. So if I were

22:33

to redo it today, I wouldn't make all those

22:35

mistakes I made before. I'd use all the

22:37

stuff I'm using now. That's

22:39

an easy answer. I think

22:41

one thing that I often

22:44

debate in my mind is

22:46

if I was starting it

22:48

today, would I still be

22:50

multi -language or would I just

22:52

embrace Rails? Because

22:54

when I think about the ROI

22:58

whenever there's an issue with our Python

23:00

adapter or anything like that, that takes a

23:02

lot more time than with Ruby because

23:04

I don't know it as well. And we

23:06

have a lot fewer customers on that

23:08

platform. So that's the debate in my mind

23:10

is I don't like to use the

23:12

word regret, but do I regret making it

23:14

multi -language? It's multi -language now when

23:16

we're sticking with it. So I don't dwell on

23:18

it too much, but that is something if I were

23:20

to start over today, I have to think long

23:22

and hard about that. I

23:24

feel the same way about When I

23:27

have to go and look at

23:29

React, I feel like anything I have

23:31

to do in React LAN takes

23:33

me probably twice as long or more

23:35

as the equivalent thing in Ruby

23:37

or Rails. Yeah, I feel that. Yeah.

23:40

Python isn't even the same language that

23:42

it was the last time I

23:44

looked at it. So you don't have

23:46

to answer this question because it

23:48

is a little bit future roadmap -y.

23:51

But along those lines, do you think

23:53

you'll add other languages or? Do

23:55

you think you'll stick to the ones

23:57

you have? Right now, no plans

23:59

to add other languages. Our focus in

24:01

the past year or so has

24:03

been expanding on the hosting platform front,

24:05

because we were originally just Rails, and

24:08

then over the past year

24:10

or two, we've expanded to

24:12

Render and Fly and AWS

24:14

and Railway. I think

24:16

Fly was the one we most recently integrated with,

24:18

and I think we're pausing on that for

24:20

a bit. while we kind of catch

24:22

our breath and do some other things that

24:24

are more fun than platform integrations. But

24:27

yeah, I think if and when we

24:29

expand more, it's probably going to be

24:31

more on the platform front than the

24:33

language front. Mostly because of what

24:35

I just said about it, about not knowing

24:37

the languages and that being difficult. But also,

24:39

I think if we were to, I

24:42

think the next sort of most

24:44

popular language that it would make sense

24:46

to dive into would be PHP

24:48

Laravel. They just got 50

24:50

-some million dollars to build Laravel Cloud.

24:53

I'm not going to compete with that. Yeah.

24:57

No, it's funny because Laravel was what I had

24:59

in my mind when I asked that question. It's

25:02

kind of funny because I feel

25:04

like there was a whole rivalry between

25:06

Rubyists and PHP developers a while

25:08

back. I have not used Laravel,

25:10

but from what I understand from talking

25:12

to people, it's very, they borrowed a lot

25:14

of lessons from Rails. Yeah, I feel

25:16

like we're best friends now. I haven't used

25:18

Laravel, but I feel like we are

25:20

so spiritually aligned with the Laravel framework and

25:22

community. I would love to support Laravel

25:24

for that reason. If I were

25:26

starting a new Laravel app today, would

25:28

I host it on Heroku or Laravel Cloud?

25:30

I think the default is going to

25:32

be Laravel Cloud for everybody starting a new

25:34

Laravel app. And they've got

25:37

their own autoscaling. I think it probably

25:39

has the same limitations as the other

25:41

platform native autoscalers that I've talked about.

25:43

So yeah, I don't know. Is there an opportunity

25:45

for a Judo scale for Laravel Cloud in

25:47

the future? I don't know. There might be. That

25:49

would be cool. So with all

25:51

the back ends, this also seems like

25:54

an interesting topic of discussion because some

25:56

of the most challenging things to do

25:58

on the back end are integrating with

26:00

third -party services. And so I'm

26:02

curious, were there any hiccups

26:04

with that? Are there differences in how

26:06

you interact with them? Did you

26:08

have to do any kind of weird

26:10

architecture stuff on your back end

26:12

to handle that? It's interesting. So the

26:14

first platform we integrated with after

26:16

Heroku, was render and that took us

26:18

a long time, not because render

26:21

was difficult to integrate with, which is

26:23

because it was like peeling layers

26:25

off of the onion and finding more

26:27

ways that our code, my code,

26:29

at the time was tightly coupled with

26:31

Heroku. She was like, all these

26:33

assumptions built in like, oh, no, this

26:35

is assuming Heroku. So we had

26:37

to unpeel all those Heroku assumptions to

26:39

be able to even make it

26:41

multi -platform at all. So render

26:44

took a while for that reason. And

26:46

then the one after render was AWS. And

26:48

that took a while just because it's

26:50

AWS. Permissions were a

26:52

lot more difficult because we

26:54

had to do the IAM stuff.

26:57

I don't have much experience with AWS. I've

26:59

always been kind of anti AWS, honestly.

27:01

I think it's overkill for the vast majority

27:03

of applications. So I've tried to avoid

27:05

it, but just realized that it was crazy

27:07

for us not to integrate with it.

27:09

So finally bit the bullet and that took

27:11

a while. And since then, the

27:13

other integrations have been easier because

27:15

it's mostly just been plugging into the

27:17

APIs of the various platforms, which

27:19

are generally pretty decent. When you make

27:21

the call out to the other

27:23

service, let's say in this case, Heroku,

27:26

is that happening like in a

27:28

job? Is that happening doing a single

27:30

request? Do you have to babysit

27:32

it or check in on it? It's

27:34

happening in a sidekick job. Yeah.

27:36

So we have one sidekick job, technically

27:38

two sidekick jobs that run our

27:40

auto scaling algorithm. We have one for

27:43

upscaling and one for downscaling. Those

27:45

run every 10 seconds across all of our

27:47

apps, what needs to upscale, what needs to downscale.

27:50

And then it fans out

27:52

separate jobs to do

27:54

the actual scaling. The

27:56

interesting things for me around this are

27:58

each service being a little different. I

28:00

am curious when you said you had

28:02

to peel back the layers of the

28:05

onion on the Heroku, did

28:07

you end up doing something akin

28:10

to like an adapter pattern to like move

28:12

stuff or when the request comes in, are

28:14

you just picking a service and then it

28:16

goes down a bunch of different paths or

28:18

is it largely the same from your perspective

28:20

with just when you get to this point,

28:22

this service is different? Probably pretty close to

28:25

an adapter pattern, I would say. The

28:27

way we handle it is in

28:29

JudoScale, you can have multiple teams. You

28:31

can be a part of multiple

28:33

teams. Each team is going to

28:35

be tied to a platform. So

28:37

if you happen to have

28:39

a Heroku application and an

28:41

AWS application. Those would

28:44

be two separate teams in Heroku

28:46

and that team in the code.

28:48

We actually have subclasses of our

28:50

team for each platform where we

28:52

override the specific methods that are

28:54

going to be unique for that

28:56

platform. That does sound kind of

28:58

like what I would expect though, right?

29:00

That it's dealing with a bunch of different

29:02

services and so it's got different adapters

29:04

to different services. That's kind of what I

29:06

would have expected in my head. I

29:09

think now might be a good time to

29:11

ask a question. You

29:13

built this by yourself. And

29:15

I'm sure we have people listening to

29:17

the show that are interested in building

29:20

things by themselves. And I'm curious if

29:22

you have advice or encouragement or anything

29:24

for someone who is like, hey, maybe

29:26

I could build something by myself. I'm

29:29

always a little hesitant to give advice

29:31

because I'm not an experienced entrepreneur in

29:33

the sense that I've done this a

29:35

bunch of times. I've done this one

29:37

time. and it's gone all right. But

29:39

based on that one time, what

29:41

I would say to somebody who's

29:43

considering doing this is start out

29:45

really small. What's the smallest thing

29:48

that you can build that will

29:50

actually be useful for someone? In

29:52

an ideal world, they'd pay for it. But even

29:54

if they don't pay for it, that someone is actually

29:56

going to use on a regular basis and not

29:58

just say they're going to use, not just install it

30:00

because they're your friend, but actually use it. If

30:03

you can't get to that point, it's

30:05

probably not a viable business idea. Someone

30:07

did an interview on like another

30:10

podcast and they said, the best

30:12

market validation is someone paying you

30:14

money. Absolutely. And that's

30:16

a quote that is stuck in my mind.

30:18

I remembered my other question. Do

30:20

you think that you would ever allow people

30:22

to do kind of multi -cloud stuff or

30:24

do you think you would leave that to

30:26

other vendors? So like someone using

30:28

both Heroku and Fly, for example, or? You

30:30

can do it right now. You just

30:32

create multiple teams in Judo scale. that

30:35

works well enough and

30:37

honestly we've had very few

30:39

people have that need. Honestly

30:42

the one need that I didn't foresee

30:44

that's similar is before we integrated with

30:46

AWS because I didn't have a ton

30:48

of experience with AWS, I

30:51

didn't realize how common it was

30:53

to put a staging in a

30:55

production environment on completely separate AWS

30:57

accounts. And because of the way

30:59

we Model judo skill that means

31:01

two separate teams in judo skill, which means

31:03

two separate stripe subscriptions, which is kind of

31:05

annoying But that's the way it's gonna be

31:07

because that would be a lot more Unpeeling

31:09

of a whole nother onion if we were

31:11

to restructure that so this is what it

31:14

is lesson learned. Yeah. Yeah, that's awesome So

31:16

we are just about at time. So I

31:18

want to say thanks for coming on the

31:20

show. It's been great This conversation has been

31:22

awesome. I'm glad we got to like dive

31:24

into some scaling things because I feel like

31:26

that's maybe a topic we've under covered on

31:28

this show. I feel like we could go

31:30

on for a while. Yeah. So

31:32

thanks again for coming on the show. I

31:34

appreciate you being here. Thanks for having me.

31:37

This was fun. You can check out JudoScale

31:39

at judoscale .com. We'd love to hear from

31:41

you. If you have comments about this episode,

31:43

send an email to comments at the Ruby on

31:45

Rails podcast .com. You can include a text comment

31:47

or attach a file from Voice Memos or

31:49

Google Recorder and we'll respond to some of them

31:51

in a future show. Thanks to

31:53

Paul, our wonderful editor over at Peachtree

31:55

Sound for making us sound like professionals.

31:57

Thanks to our producer, John Nunamaker, and

31:59

thank you for listening. You're a gem. gem.