Cloud Engineering Archives - Software Engineering Daily

Cloud providers created the ability for developers to easily deploy their applications to servers on data centers. In the early days of the cloud, most of the code that a developer wrote for their application could run on any cloud provider, whether it was Amazon, Google, or Microsoft. These cloud providers were giving developers the The post Crossplane: Multicloud Control Plane with Bassam Tabbar...

Originally posted on 16 June 2017. John Looney spent more than 10 years at Google. He started with infrastructure, and was part of the team that migrated Google File System to Colossus, the successor to GFS. Imagine migrating every piece of data on Google from one distributed file system to another. In this episode, John The post Google Early Days with John Looney Holiday Repeat appeared first on...

Originally posted on 14 February 2017. Most tech companies are moving toward a highly distributed microservices architecture. In this architecture, services are decoupled from each other and communicate with a common service language, often JSON over HTTP. This provides some standardization, but these companies are finding that more standardization would come in handy. At the The post Service Prox...

Software products are distributed across more and more servers as they grow. With the proliferation of cloud providers like AWS, these large infrastructure deployments have become much easier to create. With the maturity of Kubernetes, these distributed applications are more reliable. Developers and operators can use a service mesh to manage the interactions between services The post Linkerd Servi...

Not every company wants to move to the public cloud. Some companies have already built data centers, and can continue to operate their business with their own servers. Some companies have compliance issues with the public cloud, and want to operate their own servers to avoid legal risk. Operating a data center is not easy. The post On-Prem Cloud with Bob Fraser appeared first on Software Engineeri...

Cloud computing changed the economics of running a software company. Before the cloud, a software company had to purchase physical machines which often required thousands of dollars paid up front. The cloud allowed developers to deploy their applications for free, to operate a business for cheap, and to scale without hiring a dedicated team to The post Cloud Costs with Ran Rothschild appeared firs...

Originally published on July 6, 2016. Scheduling is the method by which work is assigned to resources to complete that work. At the operating system level, this can mean scheduling of threads and processes. At the data center level, this can mean scheduling Hadoop jobs or other workflows that require the orchestration of a network The post Schedulers with Adrian Cockcroft Holiday Repeat appeared f...

The software release process is a barrier between written code and a live production environment that affects users. A software release process can involve a variety of different practices. Code might be tested for bugs using automation and manual testing. Static analysis tools can look at the code for potential memory leaks. A software release The post Liquid Software with Baruch Sadogursky appea...

Serverless computing abstracts away the idea of a server node. Serverless lets programmers treat compute resources as high-level, reliable APIs, rather than unreliable, low-level compute nodes that might fail. Serverless dramatically improves the efficiency of programmers. Instead of thinking of a database as a set of servers that need to be sharded and replicated, the The post Fission: Serverless...

Matt Klein has worked for three rapidly growing Internet companies. At AWS, he worked on EC2, the compute-as-a-service product that powers a large percentage of the Internet. At Twitter, he helped scale the infrastructure in the chaotic days before Twitter’s IPO. Today he works at Lyft, building systems to allow for ride sharing infrastructure to The post Scaling Lyft with Matt Klein appeared firs...

A smart security camera takes in a high volume of video images and processes those images using a set of machine learning models. Those models can be used to identify interesting snippets of movement throughout the day, and decide which of those snippets to keep. Some of the video snippets might contain movement of birds–but The post Flogo: Event-Driven Ecosystem with Leon Stigter and Matt E...

DevSecOps emphasizes moving security out of a siloed audit process and distributing security practices throughout the software supply chain. In the past, software development usually followed a waterfall development process. Each step in building software was serialized, one after another. First, software was planned. Then it was built. Then it was tested. Finally, the software The post DevSecOps...

Deepak Singh is the director of compute services at AWS, where he works on cloud products relating to containers, Linux, and High Performance Computing. In today’s show, Deepak describes how the market for containers and serverless has evolved, and how Amazon thinks about product strategy. Back in 2014, Docker containers were becoming a popular way The post AWS Containers with Deepak Singh appeare...

The Windows operating system is one of the most widely used pieces of software in history. Windows was started before there was any alternative to a monolithic codebase, because Microsoft was building software before the Internet was widely used by consumers. Networked computers gave rise to web applications, and software engineers began to rethink how The post DevOps at Microsoft with Martin Wood...

Internet of Things is a concept that describes lots of devices that you interact with regularly being connected to the Internet and networked together. Technologists have been dreaming of the world of IoT for many years, where our connected refrigerator can detect that we are out of food, and automatically order more food. Or our The post Android Things with Wayne Piekarski appeared first on Softw...

Unity is a game engine for building 2-D and 3-D experiences, augmented reality, movies, and other applications. Unity is cross-platform, so that applications can be written once and deployed to iOS, Android, web, and other surfaces. Unity has been around for 13 years, and has grown in popularity with the rise in gaming and game The post Unity and WebAssembly with Brett Bibby appeared first on Soft...

Front is a shared inbox application that has seen rapid adoption within companies. Front allows multiple members of a company to collaborate together on a conversation–whether that conversation is in email, Twitter, or Facebook Messenger. This is useful when a customer email needs to be shared between the sales and engineering teams, or when a The post Front Engineering with Laurent Perrin a...

Google has two consumer operating systems: Android and Chrome. The Android operating system has been widely deployed on mobile devices. Chrome is an operating system for laptops and tablets, originally based around the Chrome browser. For several years, these two ecosystems were mostly separate–you could not run Android apps on a Chrome operating system. Shahid The post Android on Chrome wit...

Kubernetes is an open source container management system. Kubernetes is sometimes described as “the Linux of distributed systems” and this description makes sense: the large numbers of users and contributors in the Kubernetes community is comparable to the volume of Linux adopters in its early days. There are many different distributions of Linux: Ubuntu, Red The post Kubernetes Distributions with...

Continuous integration and delivery allows teams to move faster by allowing developers to ship code independently of each other. A multi-stage CD pipeline might consist of development, staging, testing, and production. At each of these stages, a new piece of code undergoes additional tests, so that when the code finally makes it to production, the The post Continuous Delivery Pipelines with Abel W...

A company runs a variety of distributed systems applications such as Hadoop for batch processing jobs, Spark for data science, and Kubernetes for container management. These distributed systems tools can run on-prem, in a cloud provider, or in a hybrid system that uses on-prem and cloud infrastructure. Some enterprises use VMs, some use bare metal, The post Orchestrating Kubernetes with Chris Gaun...

Engineering organizations can operate more efficiently by working with a continuous integration and continuous deployment workflow. Continuous integration is the process of automatically building and deploying code that gets pushed to a remote repository. Continuous deployment is the process of moving that code through a pipeline of environments, from dev to test to production. At The post Kuberne...

Kubernetes is in production clusters around the world with hundreds of thousands of containers. Kubernetes provides a distributed systems management environment for small startups and giant enterprises with applications ranging from microservices to machine learning pipelines. Because the use cases are already so wide-ranging, and the project has had so much adoption, the focus of The post Kuberne...

The main user interfaces today are the smartphone, the laptop, and the desktop computer. Some people today interact with voice interfaces, augmented reality, virtual reality, and automotive computer screens like the Tesla. In the future, these other interfaces will become more common. Developers will want to be able to expose their applications to these new The post Android Slices with Jason Monk...

Back in 2014, platform-as-a-service was becoming an increasingly popular idea. The idea of PaaS was to sit on top of infrastructure-as-a-service providers like Azure, AWS, or Google Cloud, and simplify some of the complexity of these infrastructure providers. Heroku had built a successful businesses from the idea of platform-as-a-service, and there was a widely held The post Helm with Michelle Noo...