Why Data Quality Teams Need More Than Automated Rules

Great Expectations Killed Automated Rule Discovery. Here’s Why the Industry Got Rules Discovery Wrong.

Auto-generating 100 rules in 60 seconds helped data quality vendors raise tens of millions in venture capital. In production, it created so much noise that the industry’s biggest open-source player just killed the feature entirely. 

Automated data quality rules can validate large datasets quickly, but rules based on recent data patterns can mistake normal business changes for data quality failures. 

Every data executive has sat through the same magic trick during a proof of concept. A vendor connects an automated profiler to a Snowflake or Databricks table, clicks a button, and generates 90 validation rules in under a minute. Null checks, quantile boundaries, string lengths, and volume thresholds populate the screen automatically. 

To the budget holder, it looks like an instant ROI. To the engineering lead, it looks like months of manual toil eliminated. 

Six weeks into production, the #data-quality-alerts Slack channel is permanently muted. 

Roughly 70% of those auto-guessed rules fire false positives at the moment the business experiences normal, real-world variance. Instead of saving time, data teams spend more hours triaging, tuning, and deleting noisy auto-rules than it would have taken to write 10 high-value rules by hand. In production, naive rule discovery doesn’t just fail to deliver value, it creates negative ROI through data quality false positives and unnecessary engineering work. 

Why Great Expectations Killed Auto-Discovery 

The Canary in the Coal Mine

Look no further than Great Expectations (GX). Backed by over $60M in venture funding from heavyweights like Index Ventures, Tiger Global, and CRV, GX became the darling of the modern data stack largely on the promise of automated profiling and expectation discovery. 

Yet in its recent 1.x architecture overhaul and following the June 2026 shutdown of GX Cloud Great Expectations completely stripped out its Rule-Based Profiler (RBP). 

Why Automated Rule Discovery Falls Short

Why would a market leader delete the exact capability that sold the dream? 

1. Negative User Value in Production: When RBP scanned a table, it emitted 80–100 rigid Expectations (expect_column_quantile_to_be_between, expect_column_values_to_not_be_null) learned from a narrow 30-day window. The moment a table hit seasonal variance or a pipeline backfill, alert fatigue buried the engineering team. Effective data quality monitoring should distinguish genuine data defects from expected changes in transaction volumes, distributions, and pipeline activity.

2. Crushing Engineering Overhead: Maintaining ~15,000 lines of complex profiling code (MetricMultiBatchParameterBuilder, custom config stores, multi-batch attribution) generated endless support tickets from frustrated users asking, “Why did the profiler generate this rule?” 

GX ultimately made a brutal calculation: Automated discovery makes a great demo, but enterprises only trust explicit contracts. Their new stance is simple: drop the magic profiler and write your rules manually in Python. 

Why Naive Rule Generators Break: “Snapshot Blindness” 

This problem isn’t unique to Great Expectations. Whether you look at legacy governance suites like Informatica and Collibra, YAML/code-first assertion tools like dbt and Soda, first-generation rule generators all hit the same wall. They scan a recent slice of data and mistake a static statistical snapshot for the permanent laws of your business: 

• Retail & E-Commerce (The Shopify / Target Black Friday Paradox): A profiler scans October checkout data and locks in daily order volume between 40,000 and 55,000 rows, with cart values averaging $82. On Black Friday, promotional bundling pushes cart values up 40% and volume triples. A static rule generator flags the company’s highest-revenue day of the year as a critical pipeline failure, halting executive dashboards right when leadership needs them most. 

• FinTech & Banking (The Stripe / JPMorgan Settlement Trap): Financial ledger data never flows at a uniform cadence. Institutional batch settlements, T+2 reconciliations, and historical ledger backfills routinely trigger massive, legitimate spikes in row counts and value distributions. When a generator sets tight quantile bounds based on a quiet Tuesday, every quarter-end close triggers a flood of 2:00 AM P1 alerts. 

• Healthcare & Payers (The UnitedHealth / Elevance Open Enrollment Shift): During Q4 Open Enrollment, member and claims datasets undergo sudden structural shifts. Rapid partner onboarding changes null rates in optional demographic columns, and age distributions skew toward new employer cohorts. Static profilers cannot tell the difference between a broken ingestion pipeline and a predictable annual business cycle. 

The “Manual Code” Overcorrection 

As first-generation profilers fail, the industry is swinging to the opposite extreme. Frameworks like GX 1.x, dbt, and Soda are telling enterprise teams to treat every data quality check like unit testing—hand-crafting thousands of assertions in Python, SQL, or YAML. 

For a startup with 20 core tables in Snowflake, manual coding works fine. For an enterprise managing 5,000 data assets across Databricks, Big Query, and legacy databases, writing and maintaining rules by hand is a non-starter. You simply trade Alert Fatigue for Configuration Debt. 

Where Enterprise Data Quality Is Moving Next: The Shift to Context 

Both first-generation profilers and manual coding fail at scale because they miss a fundamental truth: data does not exist in a vacuum. 

A 40% spike in transaction volume isn’t inherently an error, and a 0% null rate isn’t inherently a business requirement. Whether a data point represents a critical pipeline failure or a healthy business milestone depends entirely on one missing ingredient: Context. 

First-generation tools treated data quality as a narrow math problem – running isolated formulas over static rows and columns. The next era of enterprise data reliability is moving away from statistical guessing and toward context-aware discovery. 

A 2026 study by Precisely and Drexel University’s LeBow College of Business found that 43% of surveyed data and analytics leaders cited data readiness as a major barrier to aligning AI with business objectives. This finding reinforces the need for data quality practices that reflect real business requirements, not just statistical patterns. Read the 2026 State of Data Integrity and AI Readiness report. 

When a data quality engine understands context, it resolves the conflict between scale and precision across four dimensions: 

• Temporal & Historical Context (Knowing the Rhythm of the Business): Data has memory, cycles, and momentum. Instead of treating a 30-day window as absolute truth, context-aware discovery looks across the broader timeline of an organization. It understands how datasets breathe across days of the week, quarter-end closes, and annual cycles and it backtests its own candidate rules against historical reality before ever promoting them to production. 

• Semantic & Business Context (Knowing What Actually Matters): To a naive profiler, a cosmetic user-preference column and a regulatory general ledger look identical just strings and integers in a warehouse. Contextual intelligence understands the business meaning and criticality behind the schema. It knows where to enforce rigid, zero-tolerance contracts (like financial settlements or compliance reporting) and where to allow natural elasticity, preventing minor schema variations from paging an engineer at 2:00 AM. 

• Operational Context (Knowing the “Why” Behind the Shift): Data pipeline frameworks are living ecosystems. Volumes and distributions shift when engineering teams run historical backfills, migrate upstream schemas, or onboard new regional markets. Rather than evaluating a table in isolation and firing a hundred redundant alarms, a context-aware engine connects the dots across the data estate distinguishing between a coordinated operational change and genuine data corruption. 

• Institutional Context (Compounding Human Knowledge): In a traditional setup, business context lives exclusively inside the heads of senior engineers and domain leads, which is why tools like GX and dbt ask them to hardcode it into Python or YAML files. Modern platforms capture that institutional knowledge dynamically. As teams interact with the data, validate patterns, and resolve incidents, the system absorbs that business judgment and applies it automatically across thousands of connected assets. 

The New Metric for Data Leaders 

No Chief Data Officer buys a data quality platform to brag to the board about running 50,000 rules in production. You invest in data quality to buy trust in your numbers and focus for your engineering team. 

Effective data quality rule automation should reduce manual maintenance and unnecessary alerts without weakening checks on business-critical data. 

Great Expectations abandoning automated discovery wasn’t proof that automation is dead; it was proof that automation without context is dead. Next time you evaluate a data quality platform, ignore how many rules it can auto-generate in a five-minute demo. Ask whether the engine actually understands your business well enough to know when not to page your team. 

Move beyond static data quality rules with FirstEigen’s DataBuck. Discover how context-aware data quality can reduce false alerts and improve data reliability. Contact FirstEigen to learn more.