[ale] [OT] Kubernetes vs AWS EC2?

Thu Sep 12 10:44:08 EDT 2024

I'm assuming on-prem k8s vs EC2[1] with automation managing deployment,
configuration and scaling of nodes/containers. Your decision will be
different from other folks', but here are what I would consider the primary
axes of comparison:
- Cost
  - Hardware
    - k8s: High up-front, but amortized over time. Possibly includes
support contracts.
    - ec2: Per instance based on time used on an ongoing basis[2]. And
there is not an upper bound if anything gets misconfigured, so best be very
careful about putting in limits.
  - Infra (network, storage, AC, etc)
    - k8s: Depends on your pre-existing infrastructure.
    - ec2: Ongoing cost per MB for both network and persistent storage.
  - Time
    - k8s: Much longer initial setup time. In addition, ops/eng folks will
need to ramp up on the necessary skills to properly manage the cluster
*and* deploy services to it.
    - ec2: You can be up and running in a few hours. Ops/eng setup and
testing can be done via GUI initially, with automation and "config as code"
deployment coming later.
- Scalability
  - Scaling up
    - k8s: For nodes, it's based on lead time for hardware acquisition and
installation. For containers (assuming adequate compute/storage is
available), from dozens of seconds to minutes depending on application
complexity.
    - ec2: On the order of single-digit minutes (assuming proper
auto-scaling) or low double-digit minutes (if done manually).
  - Scaling down
    - k8s: Your workloads can be managed dynamically, but your nodes are
assumed to stay in place until they are decommissioned and removed once the
hardware reaches end of life.
    - ec2: On the order of minutes.
- Ongoing maintenance
  - k8s: The versions of the OS and of Kubernetes will need to be
periodically updated. Generally done by completely rebuilding a node, then
re-introducing it to the running cluster (if the k8s upgrades allow it); or
taking several nodes, doing upgrades, creating a new cluster with them,
migrating workloads, then upgrading and migrating individual nodes when
demand is low enough to allow it.
  - ec2: The suggested method is to deploy new instances with
software/OS/"hardware" upgrades, then add them to the service rotation
while removing old instances. Once everything is net new and there are no
problems, old ec2 instances are shut down/deleted.
- Management Complexity
  - For both options, this will depend on your comfort with the platform
and will require an initial learning period as well as refreshing knowledge
on an ongoing basis.
  - k8s: Will require more effort/knowledge to properly maintain and
provide supporting services (IAM, Storage, etc)
  - ec2: Lets you avoid handling complexity for things you don't want to
tackle right away. But you will pay for it on an ongoing basis

High level:
- If I'm part of an existing org; the costs (time, money, space, etc) are
small compared to the overall budgets; the org is willing to invest in the
operations/engineering staff; and the ops/eng staff are interested/eager to
learn.... I'd go with k8s. If I could make use of k8s to run other existing
services in a more efficient fashion[3], this would tip the scales toward
k8s even more.
- If I'm building a small startup and this is my primary app, I'd deploy to
the cloud. Having a lower initial investment a) should everything go pear
shaped or b) if I need to devote resources to another part of the business;
would be my priority.
- In most/all other cases, the cost-benefit analysis would depend on what,
if any, available resources are there; what is the requirements and
criticality of the service(s) to be provided; what is the priority (and,
therefore, organizational support) for doing what needs done to stand up
the service(s).

In general on-prem kubernetes has a higher up-front cost in terms of time,
money, complexity, and knowledge; and EC2 has much lower initial investment
but you will always be paying for what you use as long as you use it.  It's
similar to whether you buy or rent your living abode. If you're in a
position to commit to the long-term, buying has a significant upside. If
you're not sure you'll be in the same city in five years, renting allows
the required flexibility, even if you accrue no equity during the time you
rent.

Just some idle thoughts on how I've come to think about these decisions.

[1]: Substitute GCP, Azure, or any other cloud provider of choice here.
[2]: You can get discounts for reserved instances, but the complexity of
determining the optimum choice has significant overhead.
[3]: In case you didn't know, AWS grew from Amazon trying to make more
money from all the compute resources they built for peak times during the
other 98% of the year.

On Wed, Aug 28, 2024 at 9:41 PM Leam Hall via Ale <ale at ale.org> wrote:

> Dozens to hundreds. EC2 can be created via automation, and autoscaling can
> expand as needed, right?
>
> On Wed, Aug 28, 2024, 20:01 DJPfulio--- via Ale <ale at ale.org> wrote:
>
>> On 8/28/24 17:53, Leam Hall via Ale wrote:
>> >
>> > What am I missing?
>>
>> Do you have 4 instances or 200+ instances?
>> _______________________________________________
>> Ale mailing list
>> Ale at ale.org
>> https://mail.ale.org/mailman/listinfo/ale
>> See JOBS, ANNOUNCE and SCHOOLS lists at
>> http://mail.ale.org/mailman/listinfo
>>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> https://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>

-- 
Dylan Northrup
"Now that is a powerful cat"
  - Aesop Rock
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.ale.org/pipermail/ale/attachments/20240912/1430afa8/attachment.htm>