Software Engineering Daily

Every digital system has vulnerabilities. Cars can be hacked, locked computers can be exploited, and credit cards can be spoofed. Security researchers make a career out of finding these types of vulnerabilities. Samy Kamkar’s approach to security research is not just about dissection–it’s also about creativity. For many of the technologies he hacks on, Samy open-sources code that summarily describ...

Self-contained systems is an architectural approach that separates the functionality of a system into many independent systems. Each self-contained system is an autonomous web application, and is owned by one team. Communication with other self-contained systems or 3rd party systems is asynchronous where possible. As Eberhard Wolff explains in this episode, self-contained systems is not the same t...

Many people find themselves going down a career path that does not bring them satisfaction. A lawyer finds himself constantly working cases he doesn’t care about. A student in medical school gets completely burned out from hospital bureaucracy. An investment banker no longer finds joy in the accounting statements that she used to enthusiastically study. Startups offer a different career path. With...

In the 1990s, the barriers to starting a company were significant. Not only did you need an idea, you needed $200,000 for servers and Oracle licenses. With cloud computing, the up-front financial costs of getting a company off the ground have been mostly eliminated–but the idea of starting a company is still perceived as risky. The process of building software has changed dramatically in the last...

A biologist wants to study the genetic makeup of an organism. A pharmaceutical researcher wants to test the effects of an experimental drug. These types of experiments require a deep knowledge of the scientific domain as well as the lab techniques to produce the data that will eventually yield a result. Transcriptic is a robotic biology laboratory that allows you to make requests for experiments t...

Application performance monitoring helps an engineer understand what is going on with an application. An application on a single machine is often monitored by inserting bytecode instructions into the application after it has been interpreted. Distributed cloud applications with functionality broken up across multiple servers often use distributed tracing. Andi Grabner from Dynatrace joins today’s...

Online marketplaces and social networks often have a trust and safety team. The trust and safety team helps protect the platform from scams, fraud, and malicious actors. To detect these bad actors at scale requires building a system that classifies every transaction on the platform as safe or potentially malicious. Since every social platform has to build something like this, Smyte decided to engi...

Every song you hear on the radio is written with a computer. Computer musicians mostly use synthesizers and samples to compose these songs. A sample is a snippet of recorded sound, sometimes taken from a songs, a movie, or another source. The more samples a musician has access to the better. SampleFocus is a platform where musicians upload and download samples to build songs with. Daniel Trostli i...

In the last five years, companies started hiring data engineers. A data engineer creates the systems that manage and access the huge volumes of data that are accumulating on cheap cloud servers. As the saying goes, “it’s more expensive to throw out the data than to store it.” Pete Soderling joins the show to discuss the rise of the data engineer, and how that role interacts with data scientists, s...

A huge percentage of online advertisements are never seen by humans. They are viewed by bots–automated scripts that are opening web pages in a browser and pretending to be a human. Advertising scammers set up web pages, embed advertisements on those pages, and then pay for bot traffic to come and view those advertisements. This aspect of the internet is bizarre and alarming. Think about it–how muc...

For many years, software companies have been breaking up their applications into individual services for the purpose of isolation and maintainability. In the early 2000s, we called this pattern “service-oriented architecture”. Today we call it “microservices”. Why did we change that terminology? Did the services get smaller? Not exactly. Jonas Boner suggests that the movement towards cloud and the...

Some tasks are simple, but cannot be performed by a computer. Audio transcription, image recognition, survey completion–these are simple procedures that almost any human could execute, but the machine learning models have not gotten consistent enough to do them accurately. Scale is an API for human labor, created by Lucy Guo and Alexandr Wang. Similar to Amazon Mechanical Turk, Scale sends small,...

Over the next few years, bots will pervade our lives more and more. These smart, conversational text interfaces provide a new way of engaging with the computer systems that we have been mostly interacting with through web and mobile app interfaces for the last decade. Bots are a necessary complement to the voice interfaces of the future, because we don’t always want to talk to the computer, and na...

Mesos and Kubernetes are tools for distributed systems management. Kubernetes is built with an emphasis on running services, whereas Mesos is commonly used for a wider variety of workloads, including data infrastructure like Spark and Kafka. Mesos can also be used as a platform to provide resource management for Kubernetes. Dharmesh Kakadia is the author of Apache Mesos Essentials, and has spent t...

Advertising fraud is easy, legal, and extremely profitable. A fraudster can set up a website, scrape content from the internet, and run programmatic advertisements against that website. The fraudster can then purchase bot traffic. Those bots will visit the page, consume advertisements, and return profit to the owner of the page. In a past life, Shailin Dhar worked for a company that set up these t...

Cloud Foundry is an open-source platform as a service. Companies use Cloud Foundry as a control plane to deploy and manage applications. It provides abstractions for microservices and continuous integration. Cornelia Davis joins the show to discuss Cloud Foundry. I interviewed her at the DevOps Enterprise Summit, where enterprises share their stories of improving their culture and their technology...

Caching is a fundamental concept of computer science. When data is accessed frequently, we put that data in a place where it can be accessed more quickly–we put the data in a cache. When data is accessed less often, we leave it in a place where the access time is slow or expensive. Netflix has a huge variety of data, and a huge variety of access patterns for how that data gets retrieved from stora...

Software Engineering Daily was started a year and a half ago, based on what I learned from my podcasting experience on Software Engineering Radio. Last week, I interviewed Robert Blumen, the editor of Software Engineering Radio, about how that podcast is produced. In today’s episode, Robert interviews me about this podcast. If you are thinking about starting a podcast about software engineering, t...

When you are working on a program, a lot of things are going through your head. In some sense, you become part machine when you are programming. Learnable Programming is a concept that facilitates this, by showing developers what the computer is doing in real time, before compiling. In this episode, Josh Varty, co-founder of Code Connect Inc., talks to Edaena Salinas about incorporating concepts f...

When a human passes away, we create a tombstone as a memorial. Friends and family visit a grave to remember the times they had with that person while they were still alive. Memorial bots are another way to celebrate the life of someone who has passed away. A memorial bot is created by taking the messages sent by a deceased person and passing it through a machine learning model in order to make a b...

When you are deciding who to marry, you are using an algorithm. The same is true when you are looking for a parking space, playing a game of poker, or deciding whether or not to organize your closet. Algorithms To Live By is a book about the computer science of human decisions. It offers strategies for how to think through everyday life like a computer scientist. Brian Christian has a background i...

There are multiple paths to constructing a piece of software from start to finish. An individual programmer can build an entire product from scratch in a couple days. A giant corporation can commission a project and delegate responsibilities to hundreds of people. An open source community can use the wisdom of the crowds to efficiently build an operating system. Today’s episode is about another pa...

Eight years ago, MongoDB was an internal project at 10gen, a company that was trying to build a platform-as-a-service out of open-source components. The team at 10gen realized that the platform-as-a-service play would be too complex, and difficult to build. Since MongoDB was the most valuable component of that project, they narrowed their focus to this new document-oriented database. In today’s ep...

Four years ago, I started volunteering for a popular podcast about software–Software Engineering Radio. For the next two years, I learned about the process of making a quality podcast about engineering. With its emphasis on preparation, timeless engineering principles, and attention to the listener, Software Engineering Radio continues to be one of the most popular shows about engineering. Softwar...

C# .NET is the framework that is most often used to write software for the Microsoft Windows operating system. For many years, the C# .NET framework was closed source, but Microsoft’s recent push towards open source has led to the creation of .NET Core, a fork of C# .NET composed of a small subset of features from the original C# .NET stack. This episode takes us through a history of .NET, with tw...