Software Engineering Daily

Google Earth allows users to explore the imagery of the real world. Imagery for Google Earth is taken from satellites, cars equipped with cameras, and other sources. Google Earth renders a data intensive 3-D model of the world on a client application such as a desktop browser or virtual reality system.  WebAssembly is a runtime for executing code other than JavaScript in a browser-based environmen...

FoundationDB is a multi-model distributed key-value store. It is fully ACID compliant and horizontally scalable. FoundationDB is not usually used directly by an application developer–FoundationDB is a foundational building block for higher level distributed systems such as the metadata store for data warehousing tool Snowflake. Ryan Worl is a software engineer who specializes in FoundationDB. He j...

FindCollabs is a platform for finding collaborators and building projects. Three months ago we had our first hackathon, with lots of projects being created and collaborated on. In an earlier episode, we showcased the first place winner ARhythm.  Today’s show features two more interviews with winners from the first FindCollabs hackathon. Kitspace is an open source hardware registry, and Rivaly is a...

A new database company needs to solve numerous problems in order to succeed. There are already lots of existing database companies, so a new company needs to find a way to strongly differentiate itself. Databases are core infrastructure, so a new database company must earn trust with its customers. A database is a complicated distributed system, so a database company must have strong engineering t...

Programmers are in high demand, and software engineering is a career path that is fun, creative, and lucrative. There are many people who want to transition into a career in software and are looking for the right path toward writing code. The traditional college computer science curriculum teaches some software engineering skills, but the time and financial cost of attending a university is prohib...

A grocery store contains fruit, vegetables, meat, bread, and other items that can expire. In order to keep these items in stock, the store must be aware of how much food has been sold and what has gone bad. When a food item is low in stock, the store needs to order more of that food from a central distribution system. Managing food inventory is not simple. Some kinds of meat might expire faster th...

Apache Airflow is a system for scheduling and monitoring workflows for data engineering. Airflow can be used to schedule ETL jobs, machine learning work, and script execution. Airflow also gives a developer a high level view into the graph of dependencies for their data pipelines. Chaim Turkel is a backend data architect at Tikal. He joins the show to discuss a case study of using Airflow to rearc...

Software projects are organized and planned using project management software. Examples of project management software include JIRA, Trello, and Asana. There are hundreds of tools for managing a software project because there are infinite ways that a project could be managed. Google Docs changed project management by allowing documents to be easier to share and collaborate on. Newer SaaS tools suc...

Niantic is the company behind Pokemon Go, an augmented reality game where users walk around in the real world and catch Pokemon which appear on their screen. The idea for augmented reality has existed for a long time. But the technology to bring augmented reality to the mass market has appeared only recently. Improved mobile technology makes it possible for a smartphone to display rendered 3-D ima...

WebAssembly allows for web-based execution of languages other than JavaScript. Programs written in Rust or C++ can be compiled down to WebAssembly and shipped over the browser for on-the-fly execution in a safe, memory controlled environment. WebAssembly has been in development for more than two years, and is still an immature ecosystem because building the necessary tooling for WebAssembly is har...

Sheng Liang was the lead developer on the original Java Virtual Machine. Today he works as the CEO of Rancher Labs, a company building a platform on top of Kubernetes. Sheng joins the show to discuss his experiences in the technology industry. The container orchestration wars had many victims. The competing standards for how an enterprise should manage its numerous containers caused several compan...

Kubernetes has made distributed systems easier to deploy and manage. As Kubernetes has become reliable, engineers have started to look for higher level abstractions we can define on top of Kubernetes. An operator is a method of packaging, deploying, and managing a Kubernetes application. Operators are useful for spinning up distributed systems such as Kafka, Redis, or MongoDB. These data systems a...

Cloud computing was popularized in 2006 with the launch of Amazon Web Services. AWS allowed developers to use remote server infrastructure with a simple set of APIs. But even with AWS, it was still not simple to deploy and manage a web application. In 2007, Heroku launched a platform built on top of AWS. Heroku focused on the developer experience by optimizing for users who were deploying Ruby on...

Software engineering is an art and a science. To manage engineers is to manage artists and scientists. Software companies build practical tools like payment systems, messaging products, and search engines. Software tools are the underpinnings of our modern lives. You might expect this core infrastructure which modern humans rely on to have been constructed with pure formulaic rigor. But the best s...

Machine learning allows software to improve as that software consumes more data. Machine learning is a tool that every software engineer wants to be able to use. Because machine learning is so broadly applicable, software companies want to make the tools more accessible to the developers across the organization. There are many steps that an engineer must go through to use machine learning, and eac...

The software market changes every year. As individuals and enterprises become more willing to buy software, there are new markets for entrepreneurs to sell software to. Good software has high margins and high retention, so even a niche software business can prove profitable. As software spreads across the world, developing countries are showing a willingness to buy the same software tools as the d...

Google Cloud was started with a vision of providing Google infrastructure to the masses. In 2008, it was not obvious that Google should become a cloud provider. Amazon Web Services was finding success among startups who needed on-demand infrastructure, but the traditional enterprise market was not yet ready to buy cloud resources. Googlers liked the idea of becoming a cloud provider. But was it th...

Containers are made to fail gracefully. When your container shuts down due to a hardware or software failure, your distributed application should be able to tolerate that failure. One simple way to be able to tolerate such a failure is to make all of your application logic “stateless.” If your application does not maintain state, then shutting it down in the middle of a computation is not a proble...

Amazon Web Services is the leading cloud provider by a large margin. Amazon established its lead by being first to market in 2006, with Google and Microsoft taking several years to catch up to the huge business opportunity of the cloud. Since 2008, Google Cloud has been working on cloud products for developers. It started with App Engine, which is widely used internally at Google, but has not had...

Containers offer a lightweight abstraction for running a server. Cloud providers are able to manage billions of containers from different users, allowing for economies of scale so that each user can pay less. Today, there is a variety of ways that users can deploy containers on a cloud provider. These containers can run in managed Kubernetes clusters, in functions-as-a-service, or in long-lived st...

Each cloud provider offers a different set of services which are not always compatible with each other. What are the challenges of building an application that interoperates with multiple different clouds? The first issue is API compatibility. Most cloud providers have a managed SQL offering, a bucket storage system, and server abstractions like virtual machines and containers. But these tools mig...

Kubernetes has evolved from a nascent project within Google to a thriving ecosystem of cloud providers, open source projects, and engineers. Tim Hockin is a principal software engineer who has been with Google for 15 years. Tim joins the show to talk about the early days of the Kubernetes projects, and the engineering efforts that are under way five years into the project. At KubeCon EU 2019, two...

Google’s cloud business was long regarded as a place where startups could build a business, but not established enterprises. For serious workloads, enterprises chose Amazon almost unanimously. This phenomenon of Amazon as the default was described by a phrase that harkened back to the days of IBM’s dominance: “nobody ever got fired for choosing AWS.” With the rise of Kubernetes, Google has establi...

The CNCF was formed under the looming shadow of AWS. The CNCF was seeded with the donation of Kubernetes by Google. Much like the Linux community was positioned as a rebellious movement in reaction to Microsoft’s dominance, the Kubernetes community represents a fervent desire to open up the market to cloud providers beyond the tight-lipped, proprietary dominion of Amazon. With such a deep spirit o...

Cloud computing, open source, and mobile computing are trends that affect every organization. When a large organization adapts to these trends, it is commonly referred to as a “digital transformation”. Digital transformation causes many companies to reframe their business as a software company. A candy manufacturer now must think of itself as a software company that makes candy. An insurance compa...