Software Engineering Daily

During 2015, Uber was going through rapid scalability. The internal engineering systems were constantly tested by the growing user base. Over the next two years, the number of internal services at Uber would grow from 500 to 2000, and standardizing the monitoring of all these different services became a priority. After working with a variety of available tools, Uber’s engineering team decided that...

A knowledge base assembles information from a wide variety of sources into a central platform. The most popular knowledge base is Wikipedia, which covers a wide variety of concepts through a system that attempts to remain authoritative and impartial. Other open knowledge platforms include Stack Overflow, which focuses on programming concepts, and Quora, which adds a social element to the process o...

Coding bootcamps allow anyone to become a programmer at a faster pace than the traditional computer science education system. In the last five years, coding bootcamps have grown rapidly in popularity, with thousands of people gaining the necessary skills to work as a software engineer. Career Karma is a platform that allows individuals to find the best coding bootcamp. There are many coding bootca...

A hackathon is an organized event where participants work together to build a product or tool. Hackathons are about creativity, learning, and exploration. A developer that is participating in a hackathon is often working on something that is outside of their normal day-to-day focus.  Hackathons can provide significant value to the participants. Hackathons have led to friendships, new companies, an...

Videos play a key role in the continuing education of a software engineer. Video can capture many different types of content that is useful for engineers: conference talks, tutorial videos, and podcast-style interviews are all popular formats of online video. YouTube has become the predominant source for video content about software engineering. The open nature of YouTube’s format allows for a tre...

“Internet of Things” is a term used to describe the increasing connectivity and intelligence of physical objects within our lives.  IoT has manifested within enterprises under the term “Industrial IoT,” as wireless connectivity and machine learning have started to improve devices such as centrifuges, conveyor belts, and factory robotics. In the consumer space, IoT has moved slower than many people...

Data engineering involves numerous tools–a data lake, databases, data warehouses, numerous APIs, streaming systems, and microservices. There is no shortage of ways to interact with data and manage data, but many companies are struggling to figure out design patterns and best practices for how to manage data and build data infrastructure. Zhamak Dehgani is a principal consultant and portfolio direc...

Software companies such as Slack, Zoom, and Uber have recently gone public. When a company goes public, they issue a document called an S-1. Within the S-1, there is a wealth of information about the company, providing a detailed story about the company’s business model, economics, and future prospects. The S-1 describes the operating model and the philosophy of a newly public company. Going publi...

Envoy is an open source edge and service proxy that was originally developed at Lyft.  Envoy is often deployed as a sidecar application that runs alongside a service and helps that service by providing features such as routing, rate limiting, telemetry, and security policy. Envoy has gained significant traction in the open source community, and has formed the backbone of popular service mesh proje...

In the developed world, it is easy to take for granted that we grew up with computers. Technology is so pervasive in the United States that we have debates about how early in child development a human should be given a smartphone. Across much of Africa, there is a shortage of access to computers. Children grow up without much exposure to computers at all. Smartphones are starting to proliferate th...

The Hadoop ecosystem provided every company with the tools to store and query large amounts of data at a low cost. Since 2005, that ecosystem has expanded with more and more open source applications for data infrastructure.  Data infrastructure includes databases, data lakes, distributed queues, data warehouses, query engines, web applications, on-prem software, closed source, open source, cloud p...

From Africa to India to Asia to South America–computer science and programming are rising in popularity in every emerging market. Each of these markets has regional needs for technology. Just like every culture develops its own food and television, every culture needs different types of applications to run their lives. In Vietnam, the day-to-day life of a citizen is different than it is in the Uni...

In 2011, Facebook had begun to focus its efforts on mobile development. Mobile phones did not have access to reliable, high bandwidth connections, and the Facebook engineering team needed to find a solution to improve the request latency between mobile clients and the backend Facebook infrastructure. One source of latency was recursive data fetching. If a mobile application client made a request t...

Facebook has released open source software projects that have changed the industry. The most impactful projects to date are the React frontend user interface tools: ReactJS and React Native. Before React became popular, there were multiple competing solutions for the dominant frontend JavaScript framework. React became the most prominent because of its invention of JSX, its one-way data flow, and...

Facebook generates high volumes of data at a rapid pace. Dhruba Borthakur joined Facebook in 2008 to work on data infrastructure. His early projects at Facebook were around Hadoop, the distributed file system and MapReduce computation platform that laid the foundation for the “big data” movement.  At the time, Facebook was generating as much data as any other startup, and the company needed to sta...

Facebook moves fast because of vision, collaboration, and trust. The fast pace of development is enabled by constantly improving infrastructure and a sense of unity throughout the company.  In Facebook’s early days, there was an emphasis on rapidly deploying new code to drive constant improvement and experimentation.  Product quality was maintained by engineers closely checking each other’s code r...

Facebook was built using PHP, a programming language that was used widely in the late 90s and early 2000s. PHP allows developers to get web applications built quickly and easily, although PHP has a reputation for being difficult to scale.  In the early days of Facebook, the company was scaling rapidly on every dimension.  New users were piling into Facebook. Existing users were increasing their in...

Data engineering allows a company to take advantage of the large quantities of data that the company has generated. In many companies, new data has been produced rapidly for many years, but the company has not been able to take full advantage of it.  Creating large data sets does not provide immediate value for a company. A company needs to perform data engineering and data science to take full ad...

Software engineers have a wide variety of media to choose from, including podcasts, blogs, YouTube videos, conferences. The amount of software engineering media that is available is growing and accelerating. Eight years ago, there were not as many options for information about software. Charles Max Wood founded Devchat.tv to create a network of podcasts and other content for software engineers. To...

Software applications running within a host operating system need to be isolated. Isolation prevents security vulnerabilities, such as one application accessing the memory of another. In modern cloud environments, a single physical host might be running multiple virtual machines on top of a hypervisor. Those virtual machines might be divided up into containers. The different virtual machines and c...

Creating a software company has never been easier. Software engineers are increasingly seeing entrepreneurship as a viable career path. But the path to being an independent software developer is not always clear.  Most engineers spend some of their career working at a software company. Even an engineer who intends to build a company someday can thrive within the right environment. Lior Kanfi is a...

Edge computing allows for faster data access and computation. When your client application makes a request, that request might be routed to the edge. Edge servers are more numerous and more widely distributed than normal data centers, but an edge server might not have all of the data or the complete application logic for the backend to serve your request. Edge servers have historically been used f...

Today’s episode is a keynote I gave at Full Stack Tech Radar Day in Tel Aviv. The talk is called “You Are Not a Commodity”. This talk is also available as a YouTube video. The slides can be accessed here. The world of commodity engineering is coming to an end. Developers are becoming more productive, more flexible, and more entrepreneurial. How does this impact you as an engineer? And how does it...

ListenNotes allows users to find podcasts by categories, popularity, and search queries. ListenNotes is not a podcast client that you download onto your phone, but it allows users to build playlists. These playlists are automatically published to an RSS feed, so that users can curate a playlist on desktop and subscribe to that playlist from any client. Podcasts are growing in popularity. There are...

Open source software allows developers to take code from the Internet and modify it for their own use. Open source has allowed innovation to occur on a massive scale. Today, open source software powers our consumer client applications and our backend cloud server infrastructure.  Linux powers single node operating systems and Kubernetes is the foundation for new distributed systems. Hadoop created...