Software Engineering Daily

AWS offers over 200 services as part of its IaaS platform, and that number continues to grow. Organizing all of these services, and tracking the costs they incur, can be a significant challenge, often requiring teams of AWS-certified sysadmins working together to get a handle on an enterprise-scale system.  Vantage provides an alternative, streamlined AWS console that makes it easier to manage AWS...

A data warehouse is a centralized repository that an enterprise may use to store selected data from production systems. Data is transformed into a structured form that makes it easily accessible for business intelligence or other operational users. SQL-compliant databases are frequently used for data warehouses due to the popularity of SQL as a tool in business data analytics. PostgreSQL is a free...

Build automation tools automate the process of building code, including steps such as compiling, packaging binary code, and running automated tests. Because of this, build automation tools are considered a key part of a continuous delivery pipeline. Build automation tools read build scripts to define how they should perform a build. Common build scripts include Makefile, Dockerfile, and bash.  Ear...

Static analysis is a type of debugging that identifies defects without running the code. Static analysis tools can be especially useful for enforcing security policies by analyzing code for security vulnerabilities early in the development process, allowing teams to rapidly address potential issues and conform to best practices. R2C has developed a fast, open-source static analysis tool called Sem...

Modern SaaS products are increasingly delivered via the cloud, rather than as downloadable, executable programs. However, many potential users of those SaaS products may need that software deployed on-prem, in a private network. Organizations have a variety of reasons for preferring on-prem software, such as security, integration with private tools, and compliance with regulations. The cost of set...

Cloud platforms are often categorized as providing either Infrastructure-as-a-Service or Platform-as-a-Service. On one side of the spectrum are IaaS giants such as AWS, which provide a broad range of services for building infrastructure. On the other are PaaS providers such as Heroku and Netlify which abstract away the lower-level choices and focus on developer experience.  Digital Ocean has carve...

Yelp.com is a crowdsourced review platform focused on restaurants and local businesses. Originally created as an email-based recommendation service, Yelp re-launched in its modern form in 2005. At the time, its focus on user-created reviews and social interactions was fairly novel, and made it stand out from competitors such as Angie’s List and CitySearch. Since then, Yelp has become a worldwide b...

Studies show that people in “maker” professions such as developers and writers are most productive when they can carve out dedicated time for focused work, without the frequent context-switching that comes with an irregular meeting schedule. Meetings and other non-development work are necessary parts of the job, but a team will be much more productive with deep work time in mind.  Okay is an engin...

A “co-location” center is a data center that leases out networking and compute infrastructure to retail clients. Co-location centers host clients with a wide variety of infrastructure strategies, from small retail customers, to medium-size teams running hybrid cloud models, to large corporate clients who prefer not to incur the capital cost of building their own data center. While Equinix is alrea...

Prediction Markets provide an exchange for trading based on the outcome of events. Most prediction markets are centralized- they operate like a casino, where betting takes place under the supervision of one central governing organization. This makes the market less efficient than it otherwise might be: the central organization is a business, and it makes money by extracting value from the trades t...

In the past several years, Kubernetes has become the de-facto standard for orchestrating containerized, stateless applications. Tools such as StatefulSets and Persistent Volumes have helped developers build stateful applications on Kubernetes, but this can quickly become difficult to manage as an application scales. Tasks such as machine learning, distributed AI, and big data analytics often requi...

Over the past few years, the conventional wisdom around the value proposition of Big Data has begun to shift. While the prevailing attitude towards Big Data may once have been “bigger is better,” many organizations today recognize that broad-scale data collection comes with its own set of risks. Data privacy is becoming a hotly debated topic both in the technology industry and in regulatory agenci...

Serverless computing refers to an architectural pattern where server-side code is run on-demand by cloud providers, who also handle server resource allocation and operations. Of course, there is a server involved on the provider’s side, but administrative functions to manage that server such as capacity planning, configuration, or management of containers are handled behind-the-scenes, allowing th...

Blockchain technology has a wide variety of potential applications. Fields such as finance, supply chain management, and even voting have seen innovations driven by the development of distributed applications built on blockchains, called DApps. However, developing a DApp on a blockchain often requires low-level knowledge about cryptographic protocols or particular networks. Since no one blockchain...

The incredible advances in machine learning research in recent years often take time to propagate out into usage in the field. One reason for this is that such “state-of-the-art” results for machine learning performance rely on the use of handwritten, idiosyncratic optimizations for specific hardware models or operating contexts. When developers are building ML-powered systems to deploy in the clo...

A data-driven organization collects a wide variety of data to help in strategic decision-making. The cost of storing large amounts and variety of data has dropped dramatically in the last two decades, but too much unstructured data may not improve decision-making, and can even lead to “analysis paralysis.” Organizations react by extracting the most important, actionable data and placing it into a...

Open source software is software distributed along with its source code, using a permissive license that allows anyone to view, use, or modify it. The term “open source” also refers more broadly to a philosophy of technology development which prioritizes transparency and community development of a project. Typically, development is managed by a governing body, whether a company, foundation, or jus...

Video calling over the internet has experienced explosive growth in the last decade. In 2010, surveys estimated that around 1 in 5 Americans had tried online video calling for any reason. By May of 2020, that number had nearly tripled. A significant factor in the growth of video calling has been an open-source project called WebRTC, or “Web Real-Time Communication.” WebRTC makes it possible to cap...

Cilium is open-source software built to provide improved networking and security controls for Linux systems operating in containerized environments along with technologies like Kubernetes. In a containerized environment, traditional Layer 3 and Layer 4 networking and security controls based on IP addresses and ports, like firewalls, can be difficult to operate at scale because of the volatility of...

In a distributed application, observability is key to handling incidents and building better, more stable software. Legacy monitoring methods were built to respond to predictable failure modes, and to aggregate high-level data like access speed, connectivity, and downtime. Observability, on the other hand, is a measure of how well you can infer the internal state of a system from its outputs in or...

Embedded Software Engineering is the practice of building software that controls embedded systems- that is, machines or devices other than standard computers. Embedded systems appear in a variety of applications, from small microcontrollers, to consumer electronics, to large-scale machines such as cars, airplanes, and machine tools.  iRobot is a consumer robotics company that applies embedded engi...

Security is more important than ever, especially in regulated fields such as healthcare and financial services. Developers working in highly regulated industries often spend considerable time building tooling to help improve compliance and pass security audits. While the core of many security workflows is similar, each industry and each organization may have its own idiosyncratic needs or particul...

Microservices are built to scale. But as a microservices-based system grows, so does the operational overhead to manage it. Even the most senior engineers can’t be familiar with every detail of dozens- perhaps hundreds- of services. While smaller teams may track information about their microservices via spreadsheets, wikis, or other more traditional documentation, these methods often prove unsuita...

Reinforcement learning is a paradigm in machine learning that uses incentives- or “reinforcement”- to drive learning. The learner is conceptualized as an intelligent agent working within a system of rewards and penalties in order to solve a novel problem. The agent is designed to maximize rewards while pursuing a solution by trial-and-error.  Programming a system to respond to the complex and unpr...

For startups that are still seeking product/market fit, pre-seed investments are critical to funding initial investments in the product and in the infrastructure needed to scale. Afore Capital is a pre-seed fund that invests in innovative companies across a wide variety of verticals. Afore focuses on startups with unique product insights and novel distribution approaches. Gaurav Jain is a co-Found...