Cloud Engineering Archives - Software Engineering Daily

Kubernetes continues to mature as a platform for infrastructure management. At this point, many companies have well-developed workflows and deployment patterns for working with applications built on Kubernetes. The complexity of some of these deployments may be daunting, and when a new employee joins a company, that employee needs to get quickly onboarded with the The post Tilt: Kubernetes Tooling...

Kubernetes has become a highly usable platform for deploying and managing distributed systems.  The user experience for Kubernetes is great, but is still not as simple as a full-on serverless implementation–at least, that has been a long-held assumption. Why would you manage your own infrastructure, even if it is Kubernetes? Why not use autoscaling Lambda The post Kubernetes vs. Serverless w...

Every software company is a distributed system, and distributed systems fail in unexpected ways.  This ever-present tendency for systems to fail has led to the rise of failure testing, otherwise known as chaos engineering. Chaos engineering involves the deliberate failure of subsystems within an overall system to ensure that the system itself can be resilient The post Distributed Systems Research...

Amazon’s virtual server instances have come a long way since the early days of EC2. There are now a wide variety of available configuration options for spinning up an EC2 instance, which can be chosen from based on the workload that will be scheduled onto a virtual machine. There are also Fargate containers and AWS The post AWS Virtualization with Anthony Liguori appeared first on Software Enginee...

Serverless computing is a way of designing applications that do not directly address or deploy application code to servers. Serverless applications are composed of stateless functions-as-a-service and stateful data storage systems such as Redis or DynamoDB.  Serverless applications allow for scaling up and down the entire architecture, because each component is naturally scalable. And this The pos...

When a developer spins up a virtual machine on AWS, that virtual machine could be purchased using one of several types of cost structures. These cost structures include on-demand instances, spot instances, and reserved instances. On-demand instances are often the most expensive, because the developer gets reliable VM infrastructure without committing to long-term pricing. Spot The post Reserved In...

A large cloud provider has high volumes of network traffic moving through data centers throughout the world. These providers manage the infrastructure for thousands of companies, across racks and racks of multitenant servers, and cables that stretch underseas, connecting network packets with their destination. Google Cloud Platform has grown steadily into a wide range of The post Google Cloud Netw...

A cloud provider gives developers access to virtualized server infrastructure. When a developer rents this infrastructure via an API call, a virtual server is instantiated on physical machines. That virtual server needs to be made addressable through the allocation of an IP address to make it reachable from the open Internet. When the virtual server The post Go Networking with Sneha Inguva appeare...

Distributed systems are required to run most modern enterprise software. Application services need multiple instances for scalability and failover. Large databases are sharded onto multiple nodes. Logging services, streaming frameworks, and continuous integration tools all require the orchestration of more than one server. Deploying a distributed system has historically been difficult because the...

The ridesharing infrastructure of Lyft has a high volume of traffic that is mostly handled by servers on AWS. When Vicki Cheung joined Lyft in 2018, the company was managing containers with an internally built container scheduler. One of her primary goals at the company was to move Lyft to Kubernetes. In today’s episode, Vicki The post Lyft Kubernetes with Vicki Cheung appeared first on Software E...

Cloud infrastructure is usually consumed in the form of virtual machines or containers. These VMs or containers are running on a physical host machine that is also running other VMs and containers. This is called multitenancy. Servers across cloud providers such as AWS have a high utilization because there are multiple virtual instances running on The post Packet: Baremetal Infrastructure with Zac...

Edge computing is the usage of servers that are geographically close to the client device. The first common use case for edge computing was CDNs: content-delivery networks. A content delivery network placed media files such as images and videos on multiple servers throughout the world. These are big files, and they take lots of bandwidth The post Edge Computing Platform with Jaromir Coufal appeare...

Amazon EC2 (Elastic Compute Cloud) is a virtualized server product that provides the user with scalable compute infrastructure. EC2 was created in 2006 as one of the first three AWS services along with S3 and Simple Queueing Service. Since then, EC2 has provided the core server infrastructure for many of the companies that have been The post Amazon EC2 with Dave Brown appeared first on Software En...

Amazon’s container offerings include ECS (Elastic Container Service), EKS (Elastic Kubernetes Service), and Fargate. Through these different offerings, Amazon provides a variety of ways that a user can manage Kubernetes clusters and standalone container instances. The choice of which containerization system to choose depends on the needs of the user, and the tradeoffs they want The post Amazon Kub...

When the Kubernetes project was started, Amazon Web Services was the dominant cloud provider. Most of the code that runs AWS is closed source, which prevents an open ecosystem from developing around AWS. Developers who deploy their application onto AWS are opting into a closed, controlled ecosystem, which has both costs and benefits. The software The post Kubernetes Progress with Kelsey Hightower...

Cruise is a company that is building a fully automated self-driving car service. The infrastructure of a self-driving car platform presents a large number of new engineering problems. Self-driving cars collect vast quantities of data as they are driving around the city. This data needs to be transferred from the cars onto cloud servers. The The post Kubernetes at Cruise with Karl Isenberg appeared...

The container orchestration wars ended in 2016 with Kubernetes being the most popular open source tool for deploying and managing infrastructure. Since that time, most large enterprises have been implementing a “platform strategy” based around Kubernetes. A platform strategy is a plan for creating a consistent experience for software engineers working throughout an enterprise. At The post Linkerd...

Kubernetes has created a widespread system for deploying and managing infrastructure. As Kubernetes has been increasingly adopted, companies are thinking about how to leverage that common layer of infrastructure. With the common infrastructure abstraction of Kubernetes, it becomes easier to adopt other abstractions that are uniform across the entire company.  This has created a market The post Ist...

A cloud provider gives a developer low-cost compute infrastructure on-demand.  Cloud providers can be divided up into two categories: Layer 1 cloud providers and Layer 2 cloud providers. A Layer 1 cloud provider such as Amazon Web Services owns server hardware and sells compute infrastructure as a commodity. A Layer 2 cloud provider purchases compute The post Heroku Infrastructure with Mark Turner...

New software abstractions always take advantage of the abstractions that have been built before. Software libraries allow us to import code that sits on the same host as a new program. Open source software let us copy and paste existing code, or clone entire repositories. Cloud providers offer hosted tools and APIs that we can The post Cloud Dependencies with Mya Pitzeruse appeared first on Softwa...

“Serverless” is an execution model where applications are scheduled and deployed to servers that are not directly managed by the application developer.  In serverless execution, an application only loads and operates when a user actually needs to get a response from that application. This saves on resources, because many applications do not need to run The post Cloudflare Serverless with Zack Bloo...

Java programs run in a different environment than they did ten years ago. Modern infrastructure runs on containers sitting in a Kubernetes cluster. The optimal configuration for a Java program in that context is different than it was for an environment dominated by virtual machines and bare metal. When you are co-scheduling your services with The post GraalVM Quarkus: Java Acceleration with Guilla...

Dark Lang is a programming language that is tightly integrated with the cloud.  Dark takes an opinionated approach that most developers are going to want to run their applications in the cloud, and this perspective influences how Dark looks at deployments, IDEs, exception handling, and other aspects of software development. Paul Biggar is the founder The post Dark Lang with Ellen Chisa and Paul Bi...

Modern applications are distributed systems. These applications require an installation mechanism that can run and update the software across multiple nodes.  When a SaaS company starts to work with large enterprise customers, that company needs to figure out a way to deliver their product to the enterprise. This requires the SaaS company to deploy the The post Gravity: Distributed Application Del...

Render is a cloud provider built on top of Amazon Web Services and Google Cloud. Render uses the compute abstractions provided by the major cloud providers to build a second layer cloud provider with the goal of providing a better user experience. Anurag Goel is the founder of Render, and he returns to the show The post How To Build A Cloud Provider with Anurag Goel appeared first on Software Engi...