Cloud Engineering Archives - Software Engineering Daily

Since Kubernetes came out, engineers have been deploying clusters to Amazon. In the early years of Kubernetes, deploying to AWS meant that you had to manage the availability of the cluster yourself. You needed to configure etcd and your master nodes in a way that avoided having a single point of failure. Deploying Kubernetes on The post Kubernetes on AWS with Arun Gupta appeared first on Software...

A single user request hits Google’s servers. A user is looking for search results. In order to deliver those search results, that request will have to hit several different internal services on the way to getting a response. These different services work together to satisfy the user’s request. All of these services need to communicate The post Istio Motivations with Louis Ryan appeared first...

Docker was released in 2013 and popularized the use of containers. A container is an abstraction for isolating a well-defined portion of an operating system. Developers quickly latched onto containers as a way to cut down on the cost of virtual machines–as well as isolate code and simplify deployments. Developers began deploying so many containers The post Kubernetes Usability with Joe Beda...

In the first 10 years of cloud computing, a set of technologies emerge that every software enterprise needs; continuous delivery, version control, logging, monitoring, routing, data warehousing. These tools were built into the Cloud Foundry project, a platform for application deployment and management. As we enter the second decade of cloud computing, another new set The post Cloud R&D with O...

Cloud Foundry is an open-source platform as a service for deploying and managing web applications. Cloud Foundry is widely used by enterprises who are running applications that are built using Spring, a popular web framework for Java applications, but developers also use Cloud Foundry to manage apps built in Ruby, Node and any other programming The post Cloud Foundry with Rupa Nandi appeared first...

Google Docs is used by millions of people to collaborate on documents together. With today’s technology, you could spend a weekend coding and build a basic version of a collaborative text editor. But in 2004 it was not so easy. In 2004 Steve Newman built a product called Writely, which allowed users to collaborate on The post High Volume Logging with Steve Newman appeared first on Software Enginee...

Duolingo is a language learning platform with over 200 million users. On a daily basis millions of users receive customized language lessons targeted specifically to them. These lessons are generated by a system called the session generator.   Andre Kenji Horie is senior engineer at Duolingo. He wrote about the process of rewriting the session The post Scala at Duolingo with Andre Kenji Horie appe...

Ten years ago, if you wanted to build software, you probably needed to know how to write code. Today, the line between “technical” and “non-technical” people is blurring. Website designers can make a living building sites for people on WordPress or Squarespace–without knowing how to write code. Salesforce integration experts can help a sales team The post Cloud Marketplace with Zack Bloom ap...

Remember when the best game you could play on your phone was Snake? In 1998, Snake was preloaded on Nokia phones, and it was massively popular. That same year Half-Life won game of the year on PC. Metal Gear Solid came out for Playstation. The first version of Starcraft also came out in 1998. In The post Scalable Multiplayer Games with Yan Cui appeared first on Software Engineering Daily .

The Internet was designed as a decentralized system. Theoretically, if Alice wants to send an email to Bob, she can set up an email client on her computer and send that email to Bob’s email server on his computer. In reality, very few people run their own email servers. We all send our emails to The post Decentralized Objects with Martin Kleppmann appeared first on Software Engineering Daily .

Developers can build networked applications today without having to deploy their code to a server. These “serverless” applications are constructed from managed services and functions-as-a-service. Managed services are cloud offerings like database-as-a-service, queueing-as-a-service, or search-as-a-service. These managed services are easy to use. They take care of operational burdens like scalabil...

Functions as a service are deployable functions that run without an addressable server. Functions as a service scale without any work by the developer. When you deploy a function as a service to a cloud provider, the cloud provider will take care of running that function whenever it is called. You don’t have to worry The post Serverless Scheduling with Rodric Rabbah appeared first on Software Engi...

Are we a media company or a technology company? Facebook and the New York Times are both asking themselves this question. Facebook originally intended to focus only on building technology–to be a neutral arbiter of information. This has turned out to be impossible. The Facebook newsfeed is defined by algorithms that are only as neutral The post React and GraphQL at New York Times appeared fi...

Cloud computing changed the economics of running a software company. A cloud is a network of data centers that offers compute resources to developers. In the 1990s, software companies purchased servers–an upfront capital expense that required tens of thousands of dollars. In the early 2000s, cloud computing started, and turned that capital expense into an The post How IBM Runs Its Cloud with...

Thumbtack is a marketplace for real-world services. On Thumbtack, people get their house painted, their dog walked, and their furniture assembled. With 40,000 daily marketplace transactions, the company handles significant traffic. On yesterday’s episode, we explored how one aspect of Thumbtack’s marketplace recently changed, going from asynchronous matching to synchronous “instant” matching. In t...

The labor market is moving online. Taxi drivers are joining Uber and Lyft. Digital freelancers are selling their services through Fiverr. Experienced software contractors are leaving contract agencies to join Gigster. Online labor marketplaces create market efficiency by improving the communications between buyers and sellers. Workers make their own hours, and their performance is judged The post...

Facebook serves interactive content to billions of users. Google serves query requests on the world’s biggest search engine. Uber handles a significant percentage of the transportation within the United States. These services are handling radically different types of traffic, but many of the techniques they use to balance loads are similar. Vivek Panyam is an The post Load Balancing at Scale with...

As a system becomes more complex, the chance of failure increases. At a large enough scale, failures are inevitable. Incident response is the practice of preparing for and effectively recovering from these failures. An engineering team can use checklists and runbooks to minimize failures. They can put a plan in place for responding to failures. The post Incident Response with Emil Stolarsky appear...

There is a quote from Jeff Bezos: “70% of the work of building a business today is undifferentiated heavy lifting. Only 30% is creative work. Things will be more exciting when those numbers are inverted.” That quote is from 2006, before Amazon Web Services had built most of their managed services. In 2006, you had The post Run Less Software with Rich Archbold appeared first on Software Engineering...

A popular software application serves billions of user requests. These requests could be for many different things. These requests need to be routed to the correct destination, load balanced across different instances of a service, and queued for processing. Processing a request might require generating a detailed response to the user, or making a write The post High Volume Event Processing with J...

As the gig economy grows, that growth necessitates innovations in the online infrastructure powering these new labor markets. In our previous episodes about Uber, we explored the systems that balance server load and gather geospacial data. In our coverage of Lyft, we studied Envoy, the service proxy that standardizes communications and load balancing among services. The post Fiverr Engineering wit...

In an event driven application, each component of application logic emits events, which other parts of the application respond to. We have examined this pattern in previous shows that focus on pub/sub messaging, event sourcing, and CQRS. In today’s show, we examine the intersection of event driven architecture and serverless architecture. Serverless applications can be The post Serverless Event-Dr...

The Netflix API is accessed by developers who build for over 1000 device types: TVs, smartphontes, VR headsets, laptops. If it has a screen, it can probably run Netflix. On each of these different devices, the Netflix experience is different. Different screen sizes mean there is variable space to display the content. When you open The post Netflix Serverless-like Platform with Vasanth Asokan appea...

Serverless architecture is software that runs without an addressable server. Serverless is made possible by two types of technology: platform as a service providers like Auth0, and functions as a service like AWS Lambda. With both of these technologies, we can program logic that runs without being deployed to a server. Functions as a service The post Serverless Authentication with Bobby Johnson ap...

Augmented reality applications are slowly making their way into the world of the consumer. Pokemon Go created the magical experience of seeing Pokemon superimposed upon the real world. IKEA’s mobile app lets you see how a couch would fit into your living room, which has a significant improvement on the furniture buying process. Augmented reality The post Augmented Reality with Scott Montgomerie ap...