DOP 64: Do We Really Want To Use Serverless?

Transcript

Viktor Farcic 0:00
Do you want to focus more on your business and less on everything else? That's the eternal question. I think.

Darin Pope 0:04
This is episode number 64 of DevOps Paradox. Do we really want to use serverless?

Darin Pope 0:14
Welcome to DevOps Paradox. This is a podcast about random stuff in which we Darin and Viktor, pretend we know what we're talking about. Most of the time, we mask our ignorance by putting the word DevOps everywhere we can, and mix it with random buzzwords like Kubernetes, serverless, CI/CD, team productivity, islands of happiness, and other fancy expressions that make us sound like we know what we're doing. Occasionally, we invite guests who do know something, but we do not do that often, since they might make us look incompetent. The truth is out there and there is no way we are going to find it. PS: it's Darin reading this text and feeling embarrassed that Viktor made me do it. Here are your hosts, Darin Pope and Viktor Farcic.

Darin Pope 1:06
So last week, Viktor, we tried our attempt at a serverless 101 episode, our take on what a serverless 101 would look like. We talked the current managed functions, lambda, Azure Functions, Google functions, all GA, all very useful.

Darin Pope 1:33
The containers as a service. I don't like using that we didn't use that phrase before, but we'll call it that the ECS Fargates, the cloud run, whatever the Azure one is. What was the Azure one? Azure containers. okay. GA-ish, right? GA-ish. But now the question is, okay, you're so we see each other. So people listening, you can't see it. But he's, he's nodding his head sort of Yes, no, maybe not really. But it's Thursday. So, okay.

Viktor Farcic 2:16
The reason why I'm saying is because I think that now we are in a, in a phase where Kubernetes as a service in the big providers is is a good and stable thing. Now, container as a service is a service on top of their Kubernetes. So kind of like, if we just arrived to the point where Kubernetes as a service is stable, good thing to do in the Big Three providers, then it's a logical conclusion, even if we don't know anything else, that container as a service must be like a year behind or two years behind, right? Just logically speaking, even if I know nothing about it.

Darin Pope 3:00
And the only reason that I would choose container as a service over Kubernetes today is if I don't want to run Kubernetes. Right? That would be the only logical reason why I would potentially even do that.

Viktor Farcic 3:15
I'm gonna say that I mean, depends how you define if you don't want to run Kubernetes. If you mean literally, then I wouldn't agree. I would say if you don't want to deal with, with all the stuff that you need to do, like networking, storage, monitoring, scaling, you know, all that stuff. So it's, yeah, you can call that if I don't want to run Kubernetes really depends how you define it.

Darin Pope 3:44
Yeah. If you didn't watch one of our live streams a couple of weeks ago, we had a picture and you may have seen it on Twitter. It was a sort of a Lego take off. And it was a bunch of bricks. And it was, hey, here's the StatefulSet brick Here's a Deployment brick. Here's a storage class brick, right? It's like, Here, look at all the fun, you can have using Kubernetes, sort of doing a play on Lego at the same time. But it is that's what Kubernetes is. It is a building block set of things. And you may not have the time to run it, you might still be using if you're on AWS, you might still be using Elastic Beanstalk. Because that's just simple for you to use. This still gets us back to the point of do we really want to use serverless either as a function or as a container.

Viktor Farcic 4:42
Let me rephrase that question. Let's forget about functions for a second. Let's say your question is, do we want to use containers? And if we now make a kind of worldwide question like just a simple one. Do you feel that you should be using containers? I think that the majority of people would say yes today not necessarily for everything because there is a lot of legacy and stuff like that. But let's say, for the new stuff that you're doing, would you use containers? The The answer is overwhelmingly I think today, yes. We might be fighting, whether it is Docker, whether it's Kubernetes, whether it's this or that, but only container, almost everybody sees value in it. We all interpret that value in different ways in different technologies. But container is a container, I think it's a given today. So if you assume that what I just said is correct. Then the next question is, do you want to use containers alone or do you feel that or are you doing something more serious that requires scheduling of those containers and then we are entering into this core story of Swarm, Kubernetes and all those things and most people also maybe a fewer number of people, but significant number. So yes, I need some kind of those containers to run on a cluster level. It's not one server anymore, right? And that's how we got to Kubernetes. And then the next question is, do you want to bother with Kubernetes? Then we got Kubernetes as a service, right? kind of let let Google run my Kubernetes cluster. And then the next question is, do you want to bother with all the stuff that you need to set up around your applications like scaling, for example, networking and all those things? It's kind of like it's, each question opens the next question. And each of those next questions is about can I remove more from my plate of the things that are not really related to my business? That's how I interpret it right? And now, we have those two approaches. Now I'm coming back to functions and containers. We have those two approaches is, do you want to give me source code or do you want to give me container image? Because guess what, when you do run function as a service, you're running a container. Just to clarify kind of like it's a container. Maybe not Docker container, maybe it's directly the mind stopped. The thing in Linux doesn't matter.

Darin Pope 7:28
lxc?

Viktor Farcic 7:30
Yeah, exactly. But some form of containers is used. I can guarantee you that for every single implementation of, of serverless functions in this case. So it's obviously a container. The real question is, do you prefer to give somebody source code or a binary, let's say like, you know, jar, or you prefer to give a container image? That's the real question. And depending on that choice, the implementations are different, right? Like, for example, if you give me source code, each request of that something is going to be an instance. If you give me a container image, then maybe I can actually give you up to 1000 requests per instance, right? More control, more optimization, potentially more complexity. If you're talking about containers as a flavor of serverless. Yet, what I really wanted to say is that I see two groups of people, people going towards function as a service, and people going towards containers, which might end up on serverless side of that, depending on how much which of those many layers I mentioned they want to give away to somebody else.

Darin Pope 8:50
Right, because from a container as a service, those are persistent running things. They're always running right?

Viktor Farcic 9:00
No,

Darin Pope 9:01
you? Well...

Viktor Farcic 9:02
Container as a service can be nothing is running if there are no requests.

Darin Pope 9:07
ECS for ECS Fargate does work that same way. I'm still stuck with I'm still thinking ECS EC2. Which there is?

Viktor Farcic 9:17
there is there is always, if it's a container as a service in the context of serverless because I get that somebody might think that container as a service is something else. It's I mean, there are no requests, it's nothing when there are requests, there is something. One of the major differences is that unlike function as a service where one instance is associated with one request, with container as a service one instance might be associated with one request, or 1000 requests depending on what you choose.

Darin Pope 9:50
right, because you could have a cooldown period of an hour and anything that comes in within that hour keeps it alive for another, the clock, the reset clock. happens, right? It's not like so. So if you go extreme and you think if you go back to the old Perl CGI days, to where every request was a, there was one a one to one mapping, it ran, it was done. There was no reuse at all. But now, even with function as a service or container as a service, you can have this cooldown period.

Viktor Farcic 10:24
Exactly. If there are no requests for 15 minutes or one minute, then shut it down completely. Right or things like that.

Darin Pope 10:33
So that way, if you if something goes completely dormant, you're not spending money-ish. You're still spending a little bit of money but you're not spending as much.

Viktor Farcic 10:44
But now here comes I think this is an interesting point is now when you have a huge load on a single service, function, from one perspective, you might say I don't care really because yeah, I have a million requests per second. It's AWS job to spin up million instances, right? It's not my problem anymore. But in a way it is because their costs reflect your costs. Right. So if I have sporadically some requests, sporadic requests, I would say that function is a better choice, right. But if I have more than sporadic requests to something, then container with that flexibility to choose, oh, I can handle up to 10,000 requests concurrent requests is a much more cost effective, because you are ultimately paying for it one way or another. It's not really free.

Darin Pope 11:45
Yeah, so the the the catch with it is, let's say you have 1000 endpoints, and of those thousand endpoints 900 of them. Not even that. You have 1000 endpoints but five of the endpoints are your really hot endpoints. Those are the ones taking all all the traffic all the time. So those 995 endpoints are at least candidates for moving towards function as a service. It doesn't mean that they should, but they are candidate,

Viktor Farcic 12:19
yes,

Darin Pope 12:20
but those five, now, those five still could be kept separate. You could have five different services at that point. It could be a function. And because they're, they're running all the time, it's probably never once it starts up, it's never going to shut down. Or it's a container as a service, or it's more traditional. You've just got instances running there. Because your pricing is going to burn you at some point at some, just like how we've talked about you're on prem, you move to the cloud, you see a cost savings. You're lazy and you don't manage your spend in cloud well, and you finally figure out, it's going to be cheaper for you to come back on premise. You're going to go through that same bell curve of spend with Functions as a Service if you're not careful.

Viktor Farcic 13:13
Exactly. It's like, I don't know if you remember a while ago, we had Carlos as a guest. I think he said sentence that I'm paraphrasing, that cloud providers are actually earning money from the things you don't use. Now, I would actually rephrase, I would say that they're earning money for you misusing services, either by not shutting down or using the wrong service. You know, because you chose everything should be a function or every or nothing should be a function, right? either of those two choices is actually where they're earning money. If you make a bad choice, if you make suboptimal choice.

Darin Pope 13:53
Well, and this is the problem, maybe we can go back to Twitter for just a minute way back when, or initially if I remember the story right, Twitter was fully Ruby on Rails. But then at some point, they had to peel off parts of it because Ruby on Rails was not performing as well as something else. They needed to scale. It worked great up to a point there was an inflection. And then they needed to replace. Otherwise, Justin Bieber couldn't have five racks of servers to support him. Or whatever his crazy number was. Just because you make the decision today that this is the right decision, don't be surprised that your decision becomes wrong later on. Because it's you may prematurely optimize, which may cause more spend and then it's like that's giving you heartache. But the thing about it is you may have prematurely optimized, but eventually when that inflection point hits, you would quickly see Oh crap, I wouldn't have done that. we wouldn't be able to handle that the thundering herd because we got featured on the front of TechCrunch or something else.

Viktor Farcic 15:00
That's what leads me almost the beginning of this conversation is that containers can hardly ever be a bad choice if what you're doing potentially can run on an operating system that supports containers. It's very hard to find a use case where that is a bad choice. Kubernetes might be a bad choice, right? So let's not associate Kubernetes with containers. Packaging something as a container and running it as a container in whichever form there are many forms, right? It's hardly a bad choice. Kubernetes could be. Nomad. Definitely. ECS right, kind of things above. Yes. Depends on the use case. But package as a container that's almost always a good use case. And that now then leads me to why I have a strong preference towards containers as a services function as a service. Because I believe that container images are the common denominator that works in most of the cases There are very few such common denominators. But container images are one of those. And then you choose, we want to run this container is directly on a VM, right? Docker run, you want to run it with compose, do you want to run it with Kubernetes? Do you want to run it as serverless. Right? OpenFaaS or Cloud Run? Right? What so many choices can be many choices, and we could have a long debate, which would always end up with depends, right? But container is a good common denominator. And that's why I like container as a service over function as a service, even if I'm going to have a function,

Darin Pope 16:51
right. Because I could still produce the underlier as what we were saying before. Even if you just give provider X a zip file with your code, they are still going to containerize it in some way, shape or form.

Viktor Farcic 17:09
Exactly.

Darin Pope 17:09
We don't know what it is we'd really don't care because it just runs. But if I'm able to just give them the image, there's no question about what is happening under the hood. This is my runtime.

Viktor Farcic 17:22
Exactly. This is, this is, this is NodeJS that I want. Not the one you choose not the one I don't know. This is not this, this is what I want. I'm in and now I can be considered time kind of not maybe advanced enough and kind of, but this is the level of control the division that I think it makes sense. I choose what is my application including the runtime. You make sure it's running.

Darin Pope 17:50
Right. So this is I actually have an example of this is a friend of mines company. They have been running on Elastic Beanstalk for quite a while and they've been using the Python flavor of Elastic Beanstalk, and specifically the 3.4 version of Python on Elastic Beanstalk. And if you know anything about Elastic Beanstalk, that runtime is being tombstoned soon, like sometime in 2020, being tombstoned. I can't remember exactly when, let's say it's the end of the year. What if we're just to change that Python app into a Docker container? And then I could still stick on Elastic Beanstalk for now, I don't care. Doesn't really matter. But I'm running the Docker flavor of it, and then I don't have to worry about upgrading from 3.4 to something else. You know, it's a stopgap. It's should should we still be on? 3.4? No, we shouldn't still be on 3.4. It's been dead for three, four years. Right, different conversation, but as a stopgap to get me over over the line so I don't get impacted. crazily, then it's worth doing. So container is still the right answer.

Viktor Farcic 19:00
Yeah, exactly. So it's basically level of control you want to give away. That's, that's the major.

Darin Pope 19:12
Do we want to use serverless? The answer's yes.

Viktor Farcic 19:16
Sometimes,

Darin Pope 19:17
well hang on a second. I believe the answer is always yes. But you may not want to use Functions as a Service, that specific implementation of serverless. What we're saying is we we like the container as a service. That to us that is serverless in its purest form. Right? A Docker, a Docker image, spun up as a container is the pure because that is massively portable.

Viktor Farcic 19:50
Exactly. It's and, and it avoids quite a few nasty things like I don't have to write my function or application certain way. I can do anything I want. I don't want you to tell me you need to write this and that right kind of it's whatever I want. You don't decide what it is. So and that avoids certain level of vendor lockin right There is always vendor lockin, let's clarify but but it's my image. I build it as I want, you run it. And if I don't like you tomorrow, somebody else can run it, I don't care. It's still a container image common denominator. And that's very, very powerful.

Darin Pope 20:33
Okay, but maybe I still like you. But I also want to have these other players available as well. Maybe let's play out this scenario here. I'm going to use Cloud Run on Google. I'm going to use EKS on AWS and I'm going to go to Digital Ocean and use DOKS or whatever their Kubernetes service is right. And then I pick a fourth party, since I picked three, a fourth party that does global traffic load balancing across those three providers. So wherever you are, maybe I'm maybe I'm using D, I don't remember DO's footprint, but maybe DO is sitting over in EMEA and they have Digital Ocean. Again, I'm making stories up right now. Digital Ocean has the best GDPR and the most compliant and everything else. And I know if I go with Digital Ocean over there, I'm not going to have any problems. But what I don't want to have to do is set up three different applications and manage them all separately. I want my one control plane. It gets deployed out to all the places no matter what flavor it is, it's just a Docker image to me. Now, my implementation might be different. I'm doing an Helm upgrade on EKS. I'm doing wherever the command is for Cloud Run on Google and I'm doing probably a Helm upgrade on Digital Ocean over in EMEA. So having just containers, I'm not having to rewrite my app to run in multiple providers. Not that I'm upset with the provider. It gives it's, you know, building in some business risk mitigation that I don't have to think about now, the back ends of those things. It's a different conversation. Okay, where do you keep your back end? I don't care that's different. Well let that sit somewhere else.

Viktor Farcic 22:26
Yeah. But even backend sort of still a similar story, right? I would always choose, you know, you want some kind of compatibility or some form of standard, like, if you give me my Postgres API, and it's important, I said, API, I don't really care whether you're running Postgres or you have homebaked something. I really don't care. I just care that from my application, it's Postgres or whatever I chose, right? And then you figure out if you can do better than running original Postgres, go ahead, that's okay. Right kind of it's your problem. I don't mind that with providers. And and that's why this whole story is important is that how the way we are evolving, more and more standards are being established. And software vendors or service vendors are competing on top of those standards with additional value, right. That's a general story how the world works for a long time, right? And today's that today, that common denominator is containers, and slowly becoming not slowly becoming Kubernetes. We're going to see whether actually. But yes, we can say Kubernetes is also becoming that common denominator. And, and this brings me to the important thing, and this is now placing a bet, right, as if I'm buying a lottery ticket with a small chance to win, right. But I think that the next common denominator. And this is related to serverless would be Knative. Google adopted it and I believe that Azure is going to adopt it and I know for for quite a few others are adopting it. So next level might be in the serverless story is that not give me a container. But here is the Knative yaml, run it. Because, because right now, if I go to let's say AWS Yes, I give a container image to everybody, that's clear, right? But now there is still autoscaling policy, right? Oh, it's like this here is like that, you know, kind of like different definitions. There is a bunch of other stuff that I need to give to AWS or Azure or whom not. We will get to the next level and that will be whatever is the common denominator and then one less thing that is different between those and they will figure out new stuff to compete with.

Darin Pope 24:51
So today's world, July 2020, it's a container. Do I really want to use serverless? Absolutely. I want to use serverless. I could hey if I never had to run kubectl again. It wouldn't hurt my feelings. Right? I don't want to, or I don't want to run AWS ECS. Right, I just want to be able to do today's world I want to be able to do. I'm willing to do helm upgrade, and I'm willing to do whatever the version of that is for something else.

Viktor Farcic 25:23
At least you as, let's say at least 95% of people in a company there is that maybe 5% that care a lot about things that majority of people don't know. And they're going to measure differences in performance. You know, do do funky stuff that but for vast majority of people in a company kind of here you go.

Darin Pope 25:44
Yeah, go run. Again, it all comes back to the Heroku model. Here's my code, go run it.

Viktor Farcic 25:51
Yes,

Darin Pope 25:52
right. what's what's funny to me right now is everybody's basically trying to reinvent what Heroku has been doing for years. What our company did for a number of years. What a bunch of other companies did for a number of years as platform as a service. We're trying to get back to platform as a service, which is what we haven't really said.

Viktor Farcic 26:15
Yes, we are. But there are, you know, world, our industry is full of Herokus that you have a good idea, but the technology or the market is not ready for your idea.

Darin Pope 26:30
Right.

Viktor Farcic 26:31
I mean, I could say that Docker is in the same boat as a company as Heroku, correct. Brilliant idea I've never seen before, fail to be the thing.

Darin Pope 26:42
Right. And I doubt that Heroku would have still I doubt Heroku would still be alive, at least at the point that it's at right now if Salesforce would not have picked them up. So do we where do we really want to use serverless? The answer is yes. How do you want to do it? It's your choice. Our opinion today, July 2020, is you probably want to stick with a container to give you best portability from desktop to serving your end users.

Viktor Farcic 27:17
Let me rephrase your question when you said that,

Darin Pope 27:20
go ahead.

Viktor Farcic 27:21
Haha, sorry. Do you want to focus more on your business?

Darin Pope 27:25
Yes.

Viktor Farcic 27:27
And serverless is just one of the ways. There are many others kind of do want to focus more on your business and less than everything else? That's the eternal question. I think.

Darin Pope 27:36
And by the way, if your answer is no, I want to focus on the technology, and you're not working at a technology company, you need to go get a new job. Because the correct answer is, do I want to help my business? Because by the way, as much as we like to think of think of ourselves, as technologists in the vast majority of companies, we are cost centers. We are not generating revenue for the company. We are overhead just to put it bluntly. So if you want to care about the technology, that's great. If your company allows you to care about it, that's great. But if you think that technology is going to get a bigger budget ever than marketing, you're fooling yourself. Did I say that out loud or was that my inside voice?

Viktor Farcic 28:31
You said it aloud. It was good.

Darin Pope 28:33
Okay. That's usually your thing to do. But okay, so one final thing, one final statement. Do we really want to use serverless? Yes, we want to use serverless. Is it always going to work? No. There are use cases where it doesn't make sense today, as we've defined it. Now next week, we have a guest coming on. I'm not going to talk anymore about it. But we have a guest coming on that's going to be interesting that you may want to listen to. Actually, let me rephrase that. You will, you will want to listen to.

Viktor Farcic 29:15
Can I rephrase it?

Darin Pope 29:16
Yeah, go ahead, take over take over.

Viktor Farcic 29:20
If you got if you got to the end of this episode that means that you want to listen to that guy. If you wouldn't want to listen, then you wouldn't get this far. You're not listening to us already. You're somewhere else.

Darin Pope 29:34
We hope this episode was helpful to you. If you want to discuss it or ask a question, please reach out to us. Our contact information and the link to the Slack workspace are at https://www.devopsparadox.com/contact. If you subscribe through Apple Podcasts, be sure to leave us a review there. That helps other people discover this podcast. Go sign up right now at https://www.devopsparadox.com/ to receive an email whenever we drop the latest episode. Thank you for listening to DevOps Paradox.

DOP 64: Do We Really Want to Use Serverless?

Show Notes

Hosts

Darin Pope

Viktor Farcic

Links

Rate, Review, & Subscribe on Apple Podcasts

Signup to receive an email when new content is released

Transcript