Software Engineering Daily

JavaScript programming usually is done through the use of frameworks, such as ReactJS, AngularJS, and EmberJS. These frameworks abstract away some of the messy details of JavaScript, and simplify web development so that engineers can build products at a faster pace. When we build software using JavaScript frameworks, we are missing out on some of the richness of the JavaScript language itself. Kyl...

Most episodes of Software Engineering Daily are interviews with an expert about a technical software concept. Over the past year I have done a few experiments that are more editorial in nature, and both were very popular. The first editorial was about 10 Philosophies for Engineers, and the second was about how poker relates to software engineering. Today’s episode is an editorial about the creativ...

Virtual machines were the unit of cloud computation for many years. Amazon Web Services pioneered the democratized model of allowing anyone to deploy a service to the cloud, running on a virtual machine on Amazon’s servers. After virtual machines, containers have become the unit of scale in the cloud. We break up our virtualized servers into even smaller units of computation called containers. Tod...

Engineering managers start out as engineers. Eventually, there is a fork in their career road where an engineer can choose to move up into management or continue on as an engineer in a more senior role. Changing to management involves an increase in responsibilities, a different set of goals to focus on. Jon Emerson was working at Google as an engineer when a project he was working on started to g...

The war against spam has been going on for decades. Email spam blockers and ad blockers help protect us from unwanted messages in our communication and browsing experience. These spam prevention tools are powered by machine learning, which catches most of the emails and ads that we don’t want to see. TrueCaller is a company that is bringing this quality of spam detection to our phone call systems....

The software industry has a severe lack of women. There are numerous root causes of this diversity problem. Families do not encourage women to enter math and science. The media portrays most programmers as white males. Our industry often picks up on the signals of the broader society and perpetuates them. Reversing this trend of low female involvement in computer science could have tremendous posi...

Smart phone apps have better performance than web apps. When we have an application that we use on a regular basis, we download that application to a smart phone rather than using the browser based version on our mobile browser. Google’s Polymer Project wants to improve the gap between native app performance and mobile web app performance. The key problem with mobile web is that we are sending hug...

Most programmers spend lots of their time reading content about software. Since our field changes so rapidly, engineers consume news and editorials voraciously, trying to keep up with the impossibly fast pace. The Practical Dev is a collection of blog posts, editorials, and interviews that was created to help with that end. Ben Halpern is the creator of Practical Dev, and he joins the show to disc...

Ten years ago, databases were much simpler. Most companies would only have one or two types of databases in production. Today, the age of one-size-fits-all is over. Companies have multiple databases to deal with different types of use cases, and databases have become distributed to multiple nodes in order to be scalable. Ozgun Erdogan of Citus Data joins the show to give us a modern look at databa...

Kubernetes is the container management platform that came out of Google’s experiences managing data centers. Kubernetes abstracts away many of the frustrations of distributed systems management. OpenShift is a platform built on top of Kubernetes to provide an additional layer of usability. Clayton Coleman is the lead engineer of OpenShift, and in our conversation today, we start with the basics of...

Coding boot camps are a subject of controversy. Critics of boot camps defend the conventional university system, and argue that boot camp graduates do not have enough experience to write quality software. But the reality is that some boot camp graduates have found success from this new educational path. After graduating high school, Kenny Tran attended one coding boot camp, then spent some time li...

Infrastructure-as-code is a trend that has been popularized over the past decade, as cloud computing and distributed systems have become a part of every technology company. Tools like Salt, Puppet, Chef, and Ansible allow us to manage servers and processes from the command line. David Boucha works at SaltStack, the company that makes Salt. Salt is a platform that provides configuration management...

Solar energy is a growing market. Improvements in hardware have led to some people predicting that solar energy will be powering the world within the next few decades. Undoubtedly, a large percentage of our current energy infrastructure will be replaced by solar in the near future. Replacing our old, inefficient power grid requires massive investment. On Software Engineering Daily, the resources w...

Minecraft is a sandbox video game in which players build constructions out of 3-D cubes in a procedurally generated world. Minecraft is the best-selling PC game of all time. But Minecraft is not just a game. It is a platform for creativity, used by players within the game as well as programmers outside of it. Gabriel Simmer is a 16-year-old programmer who build NodeMC, a tool that wraps around the...

Rust is a systems programming language being developed at Mozilla. Rust has features of a high-level functional language like Scala and a low-level, performance-driven language like C++. Steve Klabnik is a developer program member with Mozilla. In this episode, he discusses how Rust looks at memory management, type safety, mutability, and concurrency. We also dive into a discussion of the low leve...

The process of building a software project requires us to make so many architectural decisions. Which programming languages should be used? Which cloud service provider? Which database? A newer type of building block is the analytics platform. Companies need to track events, aggregate metrics, and change the user’s experience based on aggregated data. Dan Kador is a co-founder of Keen IO, and he j...

Erlang is a programming language with primitives that help software engineers build distributed systems. When a process is malfunctioning in Erlang, the philosophy of the language is to let the process crash–and in a distributed system where unexpected faults happen on a regular basis, this philosophy of “let it crash” simplifies how we reason about an Erlang system. Other distributed systems adva...

Google has built a microservices architecture on top of the internal container management system called Borg. These services communicate over an internal protocol known as Stubby. Borg and Stubby are tightly coupled to Google’s infrastructure–it would not make sense for Google to open source them–but Google has worked with the open source community to develop open source projects with the core fun...

At Netflix, 500 billion events and 1.3 petabytes of data are ingested by the system per day.  This includes video viewing activities, error logs, and performance events. On today’s episode, we dive deep into the data pipeline of Netflix, and how it evolved from their 1.0 version to the modern 2.0 version. Before listening to this episode, check out the blog post that inspired it.

Dropbox has been storing files on Amazon Web Services for 8 years, and Dropbox’s core business is storing files. For the past three years, Dropbox has been working on a project to migrate its file storage from Amazon Web Services to its own custom built infrastructure. Magic Pocket is the name of Dropbox’s new infrastructure layer, and it gives Dropbox more control and improved economics. James Co...

Almost a year ago, Software Engineering Daily aired a week of shows about decentralized technologies like Bitcoin, Ethereum, and IPFS. Bitcoin has established itself as a stable network, but it can only be used for financial transactions. Ethereum is a global computer built on a blockchain, but it does not have the adoption of Bitcoin. IPFS is a distributed data store with an incentivization layer...

Distributed systems products are often marketed with terms like “real-time data” and “hassle-free scaling”, but what do those terms actually mean? Is data in a distributed system ever reliably “real time”? Do we ever have strong enough plans about our scalability strategy to say that scaling will be “hassle free”? Camille Fournier joins us today to discuss distributed systems in practice. Like eve...

Distributed databases are difficult to operate, and Crate.io wants to change that. Crate is a fast, scalable, easy-to-use SQL database that is built to run in containerized environments. An average software company runs several databases–MySQL for relational store, MongoDB for a document database, HDFS for blob storage and data warehouse, elastic search for search. On today’s show, Jodok Batlogg f...

Microsoft has built a suite of technologies on top of its Azure infrastructure as a service. Today, we discuss Azure Stream Analytics, a real-time event processing engine developed at Microsoft. Azure Streaming allows for constant querying of incoming data streams, and my guest Santosh Balasubramanian discusses Azure and the movement from batch processing to stream processing.

Apache Spark is a framework for fast, distributed, in-memory analysis. Apache Cassandra is a distributed database management system that provides high availability and fast throughput. Today, we are collecting fast, big data streams from user behavior, smart phones and sensors, and the disk checkpointing of and query language of Hadoop MapReduce is no longer adequate. Tim Berglund from Datastax ca...