Kafka started as open-source software installed on a server. Complex and highly-configurable, early Kafka adopters learned first-hand how difficult, time-consuming and expensive managing Kafka clusters could be. Those staying with on-premises Kafka are adopting solutions such as data observability platforms to empower them with automated visibility and control over their environments.
Snowflake is one of the most popular cloud data warehouses today. In just one decade, the company has grown to more than 6,800 enterprise customers… Read More »Snowflake Users and Their Data: A Report on Snowflake Users and How They Optimize Their Data
In 1974, two distinct but interestingly similar milestones were achieved that would greatly affect the lives of data engineers: the Rubik’s Cube was invented, and IBM released the first relational database. Since its original rise in the 1980s, the Rubik’s Cube has become the world’s most popular puzzle toy.
IT administrators have used failure metrics for decades to track the reliability and performance of their infrastructure, whether it be PC hardware, networks, or servers.… Read More »The 12 Key Metrics Every Data Engineer Must Care About
What is the shortest distance between two points? A straight line of course. What if there are multiple points? Then, it depends. A job executed… Read More »Data Observability: Cracking the Code
You’ve all heard the saying “Garbage in, garbage out” and probably have your own horror story about wrestling data to the ground over missing values,… Read More »When Good Data Goes Bad
There’s no question that bad data hurts the bottom line. Bad customer data costs companies six percent of their total sales, according to a UK Royal Mail… Read More »Data Observability Goes Far Beyond Data Quality Monitoring and Alerts
The Acceldata Engineering team sought a way to identify Kafka Producer-Topic-Consumer relationship metrics, which resulted in us building our own Kafka utility, named Kapxy. The… Read More »Kapxy – A Kafka Utility for Topic Lineage
Data teams struggle to clean and validate incoming data streams using ETLvalidation scripts, which can be costly, time-consuming, and difficult to scale. This will only… Read More »How ETL Validation Scripts Automation Improves Data Validation
Enterprises collect and use more data than ever before. To make the most out of all this data, they build complex data pipelines, most of… Read More »Benefits of Using Kafka to Handle Real-time Data Streams
