Software Engineering Daily

Flutter is a UI toolkit developed by Google that helps developers build natively compiled applications for mobile, web, desktop, and embedded devices from a single code base. Development is fast because the screen “hot reloads” as you develop, the architecture is layered for fast and expressive designs, and its widgets incorporate all critical platform differences such as scrolling, navigation, ic...

Data exploration uses visual exploration to understand what is in a dataset and the characteristics of the data. Data scientists explore data to understand things like customer behavior and resource utilization. Some common programming languages used for data exploration are Python, R, and Matlab.  Doris Jung-Lin Lee is currently a Graduate Research Assistant at the University of California, Berke...

Arun Kumar is an Assistant Professor in the Department of Computer Science and Engineering and the Halicioglu Data Science Institute at the University of California, San Diego. His primary research interests are in data management and systems for machine learning/artificial intelligence-based data analytics.  Systems and ideas based on his research have been released as part of the Apache MADlib o...

Cloud data warehouses are databases hosted in cloud environments. They provide typical benefits of the cloud like flexible data access, scalability, and performance.  The company Firebolt provides a cloud data warehouse built for modern data environments. It decouples storage and compute to operate on top of existing data lakes like S3. It computes orders of magnitude faster performance from gigab...

Running applications in containerized environments involves regularly organizing, adding and replacing containers. This complex job may involve managing clusters of containers in different geographic locations with different configuration requirements. Platforms like Kubernetes are great for managing this complexity, but include steep learning curves to efficiently get anything off the ground. The...

Apache Superset is an open-source, fast, lightweight and modern data exploration and visualization platform. It can connect to any SQL based data source through SQLAlchemy at petabyte scale. Its architecture is highly scalable and it ships with a wide array of visualizations. The company Preset provides a powerful, easy to use data exploration and visualization platform powered by Apache Superset....

Application Programming Interfaces (APIs) are interfaces that enable multiple software applications to send and retrieve data from one another. They are commonly used for retrieving, saving, editing, or deleting data from databases, transmitting data between apps, and embedding third-party services into apps. The company BaseTen helps companies build and deploy machine learning APIs and applicatio...

The company Skynet Labs provides an open protocol for hosting data and web applications on the decentralized web. Skynet allows for decentralized, censorship-resistant, highly redundant storage and applications that are available around the globe. Developers don’t pay for their application’s storage, can launch apps with access to a user’s data right away, are free from corporations pulling access...

Columnar databases store and retrieve columns of data rather than rows of data. Each block of data in a columnar database stores up to 3 times as many records as row-based storage. This means you can read data with a third of the power needed in row-based data, among other advantages. The company Altinity is the leading enterprise provider for ClickHouse - an open-source column-store analytic data...

Apache Spark is a popular open source analytics engine for large-scale data processing. Applications can be written in Java, Scala, Python, R, and SQL. These applications have flexible options to run on like Kubernetes or in the cloud.  The company Data Mechanics is a cloud-native Spark platform for data engineers. It runs continuously optimized Apache Spark workloads on a managed Kubernetes clust...

Apache Hudi is an open-source data management framework used to simplify incremental data processing and data pipeline development. This framework more efficiently manages business requirements like data lifecycle and improves data quality. Some common use cases for Hudi is record-level insert, update, and delete, simplified file management and near real-time data access, and simplified CDC data p...

An application programming interface, API for short, is the connector between 2 applications. For example, a user interface that needs user data will call an endpoint, like a special URL, with request parameters and receive the data back if the request is valid. Modern applications rely on APIs to send data back and forth to each other and save, edit, delete, or retrieve data in databases. The num...

The traveling salesman problem is a classic challenge of finding the shortest and most efficient route for a person to take given a list of destinations. This is one of many real-world optimization problems that companies encounter. How should they schedule product distribution, or promote product bundles, or define sales territories? The answers to these questions constantly change because busine...

Microservice architecture has become very common over the past few years because of the availability of containers and container orchestrators like Kubernetes. While containers are overall positive for scaling apps and making them more available, they’ve also introduced hurdles like persisting data and state, and container restarts or pod failures. Development teams put significant work into desig...

Natural Language Processing (NLP) is a branch of artificial intelligence concerned with giving computers the ability to understand text and spoken words. “Understanding” includes intent, sentiment, and what’s important in the message. NLP powers things like voice-operated software, digital assistants, customer service chat bots, and many other academic, consumer and enterprise tools. The company B...

In software engineering, telemetry is the data that is collected about your applications. Unlike logging, which is used in the development of apps to pinpoint errors and code flows, telemetry data includes all operational data including logs, metrics, events, traces, usage, and other analytical data. Companies usually visualize this information to troubleshoot problems and understand patterns and...

Cloud computing provides tools, storage, servers, and software products through the internet. Securing these resources is a constant process for companies deploying new code to their cloud environments. It’s easy to overlook security flaws because company applications are very complex and many people work together to develop them. Wyze Labs, for example, had millions of users’ data stolen due to a...

Modern applications are increasingly built as large, distributed systems. A distributed system is a program where its components are located on different machines that communicate with one another to create a single cohesive app. Components may exist as multiple instances across “nodes,” the computers hosting them, which form clusters of nodes that span across geographic regions. The platform Uber...

Geospatial technology impacts every person who uses a smartphone, drives a car, or flies in airplanes. It refers to all of the technology used to acquire and interpret geographic information. In more advanced settings, geospatial technology is used for constructing dynamic maps, 3D visualizations, and scientific and governmental simulations.  The company Makepath specializes in geospatial technolo...

Large portions of software development budgets are dedicated for testing code. A new component may take weeks to thoroughly test, and even then mistakes happen. If you consider software defects as security issues then the concern goes well beyond an application temporarily crashing. Although even minor bugs can cost companies a lot of time to locate the bug, resolve it, retest it in lower environm...

Encryption algorithms provide the means to secure and transfer sensitive information by taking input and transforming it into an unreadable output. Usually a special key, or multiple keys, are needed to unscramble the information back to the original input. These algorithms power the security of everything from our cell phone lock screens to Fortune 500 company servers.  The company Skiff is prote...

A liquid market enables individuals or groups to quickly buy and sell assets. Decentralized platforms can struggle to execute trades when their platform does not have much liquidity for a specific token. Newer tokens or tokens with limited supply are most often the least liquid because there might be an imbalance of buyers and sellers. You can’t sell token A for price X without a consenting buyer...

Non-fungible tokens are proofs of authenticity that are stored on a blockchain. Unlike fungible tokens, such as cryptocurrencies which are interchangeable, non-fungible tokens aren’t inherently equivalent to any other token. Because they are unique, they can be used to represent any unique asset. Their presence on a blockchain enables an NFT owner to trade the asset and prove they are its owner. O...

Volatility is the degree of fluctuation of something’s price. Highly volatile assets may see rapid and large price changes, while less volatile assets will maintain a steady price. This concept is important in decentralized finance because cryptocurrencies tend to be volatile assets.  The company Synthetix provides assets called Synths that provide exposure to an asset without holding the underlyi...

A decentralized exchange, usually referred to as a DEX, is a platform for exchanging cryptocurrencies. Depending on trading volume for different coins, some DEXs are more liquid than others. On the one hand you can freely swap unlisted tokens and maintain full control over your private keys and wallet information. On the other hand, without the right supply and demand it’ll be difficult to swap a...