Software Engineering Daily

The Bitcoin main chain is a large distributed ledger of transactions. Bitcoin is useful for maintaining a trusted record of payments, but is not practical for small day-to-day payments. Bitcoin payment channels allow users to issue small payments to each other without paying the high transaction cost and latency of going through the main chain. When payment channels are connected to each other, a...

Demand for live streaming video over the internet is increasing. After the emergence of early live streaming platforms, like Twitch and Facebook Live, more forms of video have become accessible over live streams, such as sports. Live streaming is a harder engineering problem than delivering a static video file because the information distributed on a live stream is constantly changing. DZN, spelle...

Not every company wants to move to the public cloud. Some companies have already built data centers, and can continue to operate their business with their own servers. Some companies have compliance issues with the public cloud, and want to operate their own servers to avoid legal risk. Operating a data center is not easy. Operating systems need to be updated and security vulnerabilities need to b...

Podcasts have surged in popularity, but the podcast ecosystem remains difficult to work with. Podcast listeners have difficulty finding episodes. Podcast creators have difficulty finding out how to get started. The advertising marketplaces for podcasts are immature, and it can be difficult to build a business as a podcaster. Podcasting is unlike almost any other media format that we consume on the...

Cloud computing changed the economics of running a software company. Before the cloud, a software company had to purchase physical machines which often required thousands of dollars paid up front. The cloud allowed developers to deploy their applications for free, to operate a business for cheap, and to scale without hiring a dedicated team to manage the servers. Building in the cloud is cheap, bu...

Slack is a real-time messaging system for work communication. On Slack, chat rooms as big as 100,000 people have productive conversations. This might sound like the same problem solved by social networks like Facebook, where billions of users communicate over a newsfeed. But the engineering constraints of a messaging system are different than that of a social network. On a newsfeed, the order in w...

Many people have saved some money which they want to invest for the future. Some people are happy investing their money in a roboadviser, which programmatically puts money into long-term investments. Other people want a more personal approach involving a certified financial planner (CFP). A CFP is a human who allocates capital for an individual based off of that individual’s preferences. A CFP spe...

Parity is a company that builds blockchain infrastructure. Parity has built several open source projects and works with enterprises to put blockchain technology in production. Gavin Wood is the founder of Parity, and he joins the show to talk about the state of blockchain technology and what his company is currently focused on. Four years ago, Gavin helped start the Ethereum project, so he has lot...

Originally posted on June 23, 2016. Pieter Hintjens grew up writing software by himself. The act of writing code brought him great pleasure, but the isolated creative process disconnected him from the rest of the world. As his life progressed he became involved in open source communities, and he discovered a passion for human interaction. Open source software succeeds or fails on the strength of t...

Originally posted on May 12, 2016. When you are deciding who to marry, you are using an algorithm. The same is true when you are looking for a parking space, playing a game of poker, or deciding whether or not to organize your closet. Algorithms To Live By is a book about the computer science of human decisions. It offers strategies for how to think through everyday life like a computer scientist....

Originally posted on May 12, 2015. Humans have now been defeated by computers at heads up no-limit holdem poker. Some people thought this wouldn’t be possible. Sure, we can teach a computer to beat a human at Go or Chess. Those games have a smaller decision space. There is no hidden information. There is no bluffing. Poker must be different! It is too human to be automated. The game space of poker...

Featured Image Photo Credits Originally posted on July 11, 2016. Negotiation is an important skill for software engineers. The salary you negotiate at the beginning of your job could be a difference of tens of thousands of dollars over the course of an engineer’s career, but intimidating recruiters and exploding offers scare many engineers from negotiating at all. Today, Haseeb Qureshi returns to...

Originally published on July 6, 2016. Scheduling is the method by which work is assigned to resources to complete that work. At the operating system level, this can mean scheduling of threads and processes. At the data center level, this can mean scheduling Hadoop jobs or other workflows that require the orchestration of a network of computers. Adrian Cockcroft worked on scheduling at Sun Microsys...

The volume of data in the world is always increasing. The costs of storing that data is always decreasing. And the means for processing that data is always evolving. Sensors, cameras, and other small computers gather large quantities of data from the physical world around us. User analytics tools gather information about how we are interacting with the Internet. Logging servers collect terabytes o...

The software release process is a barrier between written code and a live production environment that affects users. A software release process can involve a variety of different practices. Code might be tested for bugs using automation and manual testing. Static analysis tools can look at the code for potential memory leaks. A software release might go out to a small percentage of the total user...

Modern software consists of sprawling international networks of servers. Users contact these servers to access applications. Microservices talk to each other to fulfill complicated requests. Databases and machine learning frameworks crunch terabytes of information to provide complicated answers. Across this infrastructure, there is a lot of different activities–and a lot of vulnerabilities. Withou...

Serverless computing abstracts away the idea of a server node. Serverless lets programmers treat compute resources as high-level, reliable APIs, rather than unreliable, low-level compute nodes that might fail. Serverless dramatically improves the efficiency of programmers. Instead of thinking of a database as a set of servers that need to be sharded and replicated, the programmer can think of a da...

Policies define which users and applications can access and modify resources in a computer system. In a file system, a user might have permission to read or write to a file. In a cloud infrastructure deployment, a user might have the rights to deploy a new server. One microservice may or may not have the necessary permissions to talk to another microservice. All of these are use cases where a “pol...

TLA+ is a formal specification language. TLA+ is used to design, model, and verify concurrent systems. TLA+ allows a user to describe a system formally with simple, precise mathematics. TLA+ was designed by Leslie Lamport, a computer scientist and Turing Award winner. Leslie joins the show to talk about the purpose of TLA+. Since its creation in 1999, TLA+ has been used to discover bugs in systems...

Mapillary is a company that processes high volumes of images to develop a labeled 3-D model of the physical world. Mapillary’s APIs allow developers to build applications that are aware of stop signs, buildings, streets, trees, and other physical objects in real-world space. The potential use cases for Mapillary are numerous, ranging from self-driving cars to augmented reality. We can now build a...

An instruction set defines a low level programming language for moving information throughout a computer. In the early 1970’s, the prevalent instruction set language used a large vocabulary of different instructions. One justification for a large instruction set was that it would give a programmer more freedom to express the logic of their programs. Many of these instructions were rarely used. Thi...

Open source projects benefit from the network effects of a large audience of developers. A popular open source project will be contributed to and used by thousands of developers, who are continuously testing, deploying, and improving the software. The open source movement has created massive communities and a thriving, collaborative economy. Infrastructure software companies are increasingly built...

Open source software powers everything we do on the Internet. Google runs on Linux servers. Content sites are served by WordPress. Our data is queued in Kafka clusters and stored in MongoDB instances. The success of an open source project often leads to the creator of that open source software becoming wealthy. An open source project can be monetized through enterprise add-ons, or consultation, or...

Matt Klein has worked for three rapidly growing Internet companies. At AWS, he worked on EC2, the compute-as-a-service product that powers a large percentage of the Internet. At Twitter, he helped scale the infrastructure in the chaotic days before Twitter’s IPO. Today he works at Lyft, building systems to allow for ride sharing infrastructure to work more safely and reliably. Hypergrowth Internet...

Online labor marketplaces are widely used for one-to-one transactions. On Uber, a rider hires a driver for transportation. On TaskRabbit, a homeowner hires a cleaner to come clean their kitchen. These types of marketplaces are not as widely used for one-to-many transactions, but they can be just as useful. A warehouse owner would want to hire a group of workers to help with holiday shipments. A co...