Software Engineering Daily

Biology research at Microsoft is focused on three main areas: molecular programming, synthetic biology, and stem cell biology. At the intersection of biology and computing there are implications for health, medicine, and efficient computing techniques. The field of Biological Computation is in its early days, and there is still lots of work to be done. Colin Gravill works in the computational scie...

Rust is a systems level language that is built to prevent crashes and eliminate data races. A language like C++ gives you high speed and lots of control, but it is easy to have segfaults, data races, and other problems if you aren’t careful. On this spectrum of control versus safety, we can plot other languages like Java, Go, and Haskell–but none of these languages have the unique feature set of R...

Microservices are a widely adopted pattern for breaking an application up into pieces that can be well-understood by the individual teams within the company. Microservices also allow these individual pieces to be scaled independently and updated in isolation. Past Software Engineering Daily episodes have covered the microservice architectures of Twitter, Netflix, Google, Uber and other companies....

There is a hierarchy of ways to access and store data in a computer system. The cheapest, slowest way to store and retrieve data is disk. On the faster end, we have memory. As we architect systems with increasing complexity, we have additional considerations–network latency, transient compute nodes, and numerous caching layers. Apache Geode is a distributed, in-memory system for the JVM. It provid...

Many programmers listening to this podcast are working at a big company, and they would prefer to be running their own software business. Patrick McKenzie has been writing about this topic for several years on his blog Kalzumeus.com. Almost a decade ago, he was working as an enterprise developer at a large company in Japan. Over time, his side projects started making enough money to justify his le...

Botnets have a massive influence on the Internet. As we have seen recently with the Mirai Botnet, IOT bots can take down companies as big as Netflix. In our recent episodes about advertising fraud, we’ve talked about how bots are being used to take billions of dollars of revenue from advertisers. Derek Muller is one of those advertisers who has spent money on ads and gotten nothing but fake traffi...

Slack is a chat client that has reached wide adoption. The rise of Slack has coincided with the rise of chatbots. A chatbot is a simple, conversational interface into a computer program that may have simple functionality, like telling you some simple statistics, or more complex functionality, like helping you manage your continuous integration pipeline. Bot design and engineering is a new field, a...

Large technology companies have no shortage of data. But raw data itself does not provide a competitive advantage. Many companies are bottlenecked by a shortage of data scientists who can query that data effectively. This results in an organizational dysfunction where people lining up to ask questions of the data science team are unable to move as fast as they want to. Tomasz Tunguz saw this when...

Amazon Web Services changed the economics of building an internet application. Instead of having to invest tens of thousands of dollars up front for hardware, developers can pay for services over time as their application scales. As AWS has grown to be a gigantic platform, the documentation about how to use cloud infrastructure has become insufficient. As an answer to this, Joshua Levy initiated T...

Bot Day was an O’Reilly conference featuring talks from some of the leading figures in the bot industry. Before I attended Bot Day, I knew there were lots of applications for chatbots, but I didn’t realize how good the tooling has gotten–it’s very easy to get started with chatbots today so if you are a developer and you have a basic idea, I suggest checking out the videos from Bot Day, as they are...

The blueprint for a typical startup involves investing heavily in cloud services–either from Amazon, Google, or Microsoft. The high costs can quickly eat away at all of the money that startup has raised. In today’s episode, Avi Freedman outlines some of the infrastructure mistakes that can set back a company severely–cloud jail, hipster tools, and lack of monitorability. Avi is the CEO of Kentik,...

When Facebook acquired Instagram, one of the first systems Instagram plugged into was Facebook’s internal spam and fraud prevention system. Pete Hunt was the first Facebook engineer to join the Instagram team. When he joined, the big problems at Instagram were around fake accounts, harassment, and large volumes of spammy comments. After seeing the internal Facebook spam prevention tools clean up I...

Ben Northrop was sitting at a tech conference, listening to a presentation about a new piece of technology, when he was struck by the sense that history was repeating itself. For the twenty years that Ben has worked as a software engineer, he has been hearing about new technologies that claim they will be able to disrupt everything, and he has relentlessly been keeping up as best he can with the l...

Jeff Bezanson’s university thesis described the motivation for a new programming language. He discussed the shortcomings of “array based programming environments” and his desire to create a more performant language with the best qualities of Lisp, Python, Ruby, Perl, Mathematica, R, and C. The Julia Language is a high performance language designed to suit technical users that crave the flexibility...

Legacy code is code without automated tests. Most companies have lots of legacy code, and most developers don’t like working on legacy code. Why is that? What is it that makes legacy code so difficult to work with? And why does a large amount of legacy code slow down an organization so severely? Andrea Goulet is the CEO of Corgibytes, a consultancy that specializes in remodeling legacy codebases....

Indie Hackers is a website that profiles independent developers who have made profitable software projects, usually without raising any money. These projects make anywhere from a few hundred dollars a month to more than $100,000 as in the case with park.io, one of the services profiled by Indie Hackers. Courtland Allen is the creator, engineer, and interviewer behind Indie Hackers. For each busine...

When you visit a web page, that web page can write data to a file on your computer, known as a cookie. Scripts on that page can also read from your cookie file to understand where you have been in the past. All of this data about you is getting shared between advertising companies like Google, Facebook, and AppNexus. Ghostery is a browser extension that allows you to limit what you share with thes...

Chat bots are your newest co-worker. Slack, HipChat, and other chat clients allow developers and other team members to communicate more dynamically than the limits of email. Companies have started to add bots to their chat rooms. These bots can give you technical information, restart a server, or notify you that a build has finished. Jason Hand is the author of ChatOps: Managing Infrastructure in...

Online advertising is heavily affected by a set of delusions and fraudulent practices that few people in the adtech industry have an interest in stopping. This is the curious, perverse nature of the world of online media. Some of the online advertising we see fulfills its job, when an ad successfully conveys a meaningful message from a marketer to a consumer. But there are reasons to be extremely...

Music collaboration software that works over the Internet is a software challenge that has not been fully tackled. On today’s Internet, users collaborate intensively on programming projects, journalism, and other projects, but the tools for collaborating on music online have not yet become popular. Blend.io is a social music collaboration tool–a github for musicians. I have been using it myself an...

Advertising fraud takes billions of dollars out of the economy every year. We don’t know exactly how much money is being lost, because we don’t know what percentage of Internet users are bots. Are You A Human is a company designed to solve that exact problem, and provide a service for verifying whether a user is real or automated. Ben Trenda is the CEO of Are You A Human. In this episode, we discu...

Blockchain technologies like Bitcoin and Ethereum have not impacted the lives of most consumers today. The theoretical breakthroughs that blockchain enables will eventually happen–I will be able to pay 1 cent to a knowledge worker in Africa without having to pay a 5 cent transaction fee. My servers will be able to pay other servers for small compute jobs. We will have decentralized versions of sha...

Kafka is a distributed log for producers and consumers to publish messages to each other. We’ve done many shows about Kafka as a key building block for distributed systems, but we often leave out the discussion of the complexities of setting up Kafka and monitoring it. Kafka deployments can be a complex piece of software to manage. Tom Crayford is an engineer at Heroku, where he helped engineer th...

Netflix has a highly interactive user interface. As I move my mouse around the page, hovering over titles and inspecting movie descriptions, there is a lot going on under the hood. One component of this UI is RxJS, a library for building reactive JavaScript. Reactive programming uses the observer pattern to create objects that emit streams of events. We can compose these streams together to create...

Bot fraud, the New York tech scene, RethinkDB and open source; these topics and more are discussed in today’s episode. Two of the most popular guests return to the show to explore a variety of topics. Ben Halpern is the creator of The Practical Dev, a massively popular Twitter account and blog that you may recognize from its parody O’Reilly book covers. Haseeb Qureshi is an engineer at Airbnb, and...