Cloud Engineering Archives - Software Engineering Daily

Monitoring translates machine data into actionable business metrics, and is a key component of a modern software company. James Turnbull’s new book “The Art of Monitoring” describes how organizations can build their monitoring infrastructure.     James joins the show today to outline the strategies that a company can use to proactively monitor their systems. The post The Art of Monitorin...

Lee Atchison spent seven years at Amazon working in retail, software distribution, and Amazon Web Services. He then moved to New Relic, where he has spent four years scaling the company’s internal architecture. From his decade of experience at fast growing web technology companies, Lee has written the book Architecting for Scale, from O’Reilly. As The post Scalable Architecture with Lee Atchison a...

Scheduling is the method by which work is assigned to resources to complete that work. At the operating system level, this can mean scheduling of threads and processes. At the data center level, this can mean scheduling Hadoop jobs or other workflows that require the orchestration of a network of computers. Adrian Cockcroft worked on The post Schedulers with Adrian Cockcroft appeared first on Soft...

In 1999, it took $50,000 to buy a server. Once you bought that server, you had to know how to operate and maintain it. Today, cloud service providers have changed how we build software. Servers, load balancers, networking, storage–these hardware concerns have been turned into software. Don Pezet joins the show today to discuss the The post Cloud Providers with Don Pezet appeared first on Sof...

Six years ago, Twitter was experiencing outages due to high traffic. Back in 2010 Twitter was built as a monolithic Ruby on Rails application. Twitter migrated to a microservices architecture to fix these problems. During this migration, the engineers at Twitter learned how to build and scale highly distributed microservice architectures. William Morgan was an The post Scaling Twitter with Buoyant...

Mass customization is the process of making customized, personalized products that are accessible to individuals and small businesses. The process involves manufacturing, assembly lines, supply chains, and software at every step along the way. Today’s guests are Jim Sokoloff and Maarten Wensveen, who work on infrastructure and technology at Cimpress, a mass customization platform. Cimpress The pos...

The unit of computation has evolved from on premise servers to virtual machines in the cloud to containers running in those virtual machines. Serverless computation is another stage in the evolution of computational unit management. With a serverless architecture, a function call to the cloud spins up a transient container, calls the function on that The post Serverless Code with Ryan Scott Brown...

Google’s site reliability engineers are responsible for maintaining the highly available services that power the Google software that we all use on a regular basis. O’Reilly recently published the book “Site Reliability Engineering: How Google Runs Production Systems”, and the book provides a comprehensive window into how the site reliability engineering role works. Todd Underwood The post Google&...

Dropbox has been storing files on Amazon Web Services for 8 years, and Dropbox’s core business is storing files. For the past three years, Dropbox has been working on a project to migrate its file storage from Amazon Web Services to its own custom-built infrastructure. Magic Pocket is the name of Dropbox’s new infrastructure layer, The post Dropbox’s Magic Pocket with James Cowling appeared...

Distributed systems products are often marketed with terms like “real-time data” and “hassle-free scaling”, but what do those terms actually mean? Is data in a distributed system ever reliably “real time”? Do we ever have strong enough plans about our scalability strategy to say that scaling will be “hassle free”? Camille Fournier joins us today The post Distributed Systems Tradeoffs with Camille...

Exception monitoring services and log management services are two sides of a gradient. Exception monitoring services capture and aggregate the problems that occur on your application. Log management services aggregate all of your logs, so that you can decide for yourself what constitutes a problem. Brian Rue from Rollbar joins the show today to talk The post Distributed Systems and Exception Monit...

Kubernetes is an open source system for automating deployment, operations, and scaling of containerized applications. Google developed Kubernetes after fifteen years of running containers in production. Brendan Burns is a founder of the Kubernetes project, and he joins us to talk about the lessons learned as Google has built containerized applications to distribute across its The post Google’s Con...

“You need to build more things yourself to be highly available, but one of the very good consequences of being bare metal is that the prices are very low compared to what you could get on the cloud provider.” Engineers who want to add search to their application usually deploy Elasticsearch, or write their own The post Search as a Service with Julien Lemoine appeared first on Software Engineering...

“We’re not always in control of other people’s networks.” CDN stands for content delivery network. A content delivery network is a system of distributed servers that delivers web pages and other web content. Without CDNs, the internet would be much slower, because CDNs function as a caching layer for most web resources. Carl Gustas is The post Managing a CDN with Carl Gustas appeared first o...

“You write the code, but you don’t run it? That’s just preposterous.” Software applications are constantly generating logs. These logs are necessary to understand how an application is functioning, and logs are key to debugging. As applications have gotten more complex, logging infrastructure has become complex as well. Storing and managing all of our log The post Logging and NoOps with Christian...

“As the scale continues to increase, certain effects of architecture become less and less efficient.” When you spend money online, you expect a receipt to come in your email. When you register for a new web site, you need to verify your sign up in your email. These types of emails are called “transactional email” The post Scaling Email with J.R. Jasperson appeared first on Software Engineering Dai...

“SaaS, whether we want it or not, in enterprise technology or in our data centers, is coming.” Application delivery has become more complex as software architectures have moved into the cloud. Data center infrastructure has turned into code to be manipulated, and software engineering teams are adjusting their strategies. HashiCorp is a company that builds The post Automating Infrastructure at Hash...

“It’s an amazing era for software developers – we have all this amazing infrastructure behind the scenes that we can build upon.” Ten years ago, building a highly scalable image delivery service would require millions of dollars in upfront costs, and hours of work configuring hardware server infrastructure. Today, it is possible to bootstrap this The post Bootstrapping a SaaS for Developers with I...

“Its sort of like the old joke in computer science – what do you do when you have a problem? Well, add a layer of abstraction.” Today’s guest is Calvin French-Owen, the CTO of Segment, a tool that companies use to aggregate their analytics into once place. As Segment has scaled, the company has had The post Developer Analytics with Calvin French-Owen appeared first on Software Engineering Daily .

“It’s Friday night and you’re basically out of the office on your way to meet with friends. And you just merge this thing and put it into production because you have that trust – that the system will capture any kind of problem.” Continuous integration and deployment are important tools for modern software development. With The post Continuous Delivery and Test Automation with Flo Motlik app...

This episode is a republication from my interview with Leslie Lamport on Software Engineering Radio. Leslie Lamport won a Turing Award in 2013 for his work in distributed and concurrent systems. He also designed the document preparation tool LaTex. Leslie is employed by Microsoft Research, and has recently been working with TLA+, a language that is The post Distributed Systems with Leslie Lamport...

“A lot of our customers are kind of AWS refugees who really don’t want all the stuff amazon has, and they just want a really easy to use system that is reliable.” Continue reading… The post Engineering Cloud Services with Sam Kottler appeared first on Software Engineering Daily .

“You can have a monolith, and it can be a perfectly good thing.” Continue reading… The post Moving to Microservices at SoundCloud with Lukasz Plotnicki appeared first on Software Engineering Daily .

“If an engineer is doing something repeatedly over and over again, their mind is immediately going to jump to that place where it’s like ‘Okay how can I make this faster?’ or ‘How can I save myself time?’” Quora is a Q&A website where questions are asked, answered, edited and organized by its community of The post Engineering at Quora with Shreyes Seshasai appeared first on Softwar...

“If you can make a system that can survive this random failure testing, then you will more or likely survive whatever other chaotic conditions exist.” Continue reading… The post Scaling Uber with Matt Ranney appeared first on Software Engineering Daily .