How to Be Nice to the Data Team

Tell the data team when something new exists

Data teams usually want all relevant data, not only what is already being reported. If a new feature, field, workflow, event, status, integration, or entity is added, let the data team know so they can decide whether it should be extracted, modeled, documented, or monitored.

A few examples of changes worth flagging:

• A new subscription_status value like paused is introduced.
• A new table referrals is added to support a growth experiment.
• A third-party integration starts writing rows into an existing table.
• A previously optional field becomes mandatory (or vice versa).

Make your data extractable

Extracting data is rarely a one-shot job. Pipelines run incrementally: instead of pulling the whole table every time, we pull only the rows that are new or have changed since the last successful run. This keeps extractions cheap, fast, and friendly to the source system.

For that to work, the source needs to expose a reliable signal of “what changed”. The minimum viable contract is:

• created_at: when the row was first inserted. Required for immutable records (events, logs, transactions).
• updated_at: when the row was last modified. Required for any entity that changes over time. Any change in a relevant field must update this timestamp.
• A primary key that is stable and unique (more on this below).

A typical incremental extraction looks like this:

SELECT *
FROM orders
WHERE updated_at >= :last_watermark - INTERVAL '1 minute'

The small margin (1 minute here) covers clock skew and late writes. You can read more about this pattern at Processing new data | Self-Healing Pipelines.

For immutable records (e.g. an events table where rows are never modified), created_at alone is enough. For everything else, expose both.

Never hard delete without a previous soft delete

Hard deletes (removing rows from a table) are invisible to incremental pipelines. The row was there yesterday, it is gone today, and the data team has no way to detect it without comparing full snapshots, which is a slow and costly process.

Prefer soft deletes: mark the row as deleted, keep it in the table, and let the consumer decide what to do with it.

-- Bad: hard delete
DELETE FROM users WHERE id = 42;

-- Good: soft delete
UPDATE users
SET deleted_at = NOW(),
    updated_at = NOW()
WHERE id = 42;

Common patterns:

• deleted_at TIMESTAMP NULL: null means active, a timestamp means deleted (and when).
• is_deleted BOOLEAN: simpler, but loses the “when”.
• A status enum with a deleted value.

Treat schemas as contracts

Tables, fields, types, enums, and meanings are dependencies for downstream data models, dashboards, and reports. A column rename that takes 5 minutes upstream can take days to fix downstream.

A few rules of thumb:

• Additive changes are safe: adding a new column or a new enum value rarely breaks anything.
• Renames and removals are breaking: announce them in advance, ideally with a deprecation window.
• Type changes are sneaky: changing amount from INT (cents) to DECIMAL (euros) silently breaks every downstream calculation.
• Meaning changes are the worst: if active_user starts meaning something different, dashboards keep working but tell a different story.

Use stable identifiers

Reliable IDs are essential for joining, deduplicating, tracking history, and reconciling data across systems. Bad ID hygiene is one of the hardest problems to fix downstream.

Good IDs are:

• Stable: a row’s ID never changes for its lifetime.
• Unique: no two rows ever share the same ID.
• Non-recyclable: when a row is deleted, its ID is never reused.
• Immutable in meaning: the ID itself doesn’t encode business logic that might change.

✅ Good:  user_id = 8f3a-... (UUID) or 12345 (auto-increment)
❌ Bad:   user_id = email           (changes when users update it)
❌ Bad:   user_id = "ACME-2024-01"  (changes if the company is renamed)
❌ Bad:   reused IDs after deletion

Think about history, not only current state

Operational systems often care about what is true now: what is this user’s current plan, what is this order’s current status. Analytics often needs to know how things changed: when did the user upgrade, how long was the order in pending, what was the price last quarter.

If the source only stores the current state, that history is lost forever. A few options to preserve it:

• Event log: every state change writes a row to an append-only table (subscription_changes, order_status_history).
• Slowly Changing Dimensions (SCD Type 2): keep multiple versions of the row with valid_from / valid_to columns.
• Audit log: a generic table that records every change to any row.

Ask for outcomes, not solutions

It is tempting to ask for a specific deliverable, like “I need this Excel every morning at 9am”, because it feels concrete. But the data team can usually offer a better solution if they understand the underlying need.

Compare:

❌ “I need an Excel with all orders from yesterday emailed to me every morning.”

✅ “I need to monitor conversion rate daily because we are testing a new onboarding flow. I want to spot drops within 24 hours.”

The second framing opens the door to a dashboard with alerts, a Slack notification, or a self-serve metric. Any of these might be cheaper and more useful than a daily email.

Bring context with every request

A good data request answers, at minimum:

• What is the question or metric?
• Why does it matter? What decision does it support?
• Who will use it?
• How often is it needed (one-off, daily, real-time)?
• By when do you need it?
• How critical is it (exploratory analysis vs. production dashboard vs. regulatory report)?

Explain urgency with business impact

ASAP is not a priority, it is a wish. Every requester thinks their request is urgent, and the data team has no way to triage between them.

Instead, explain why it is urgent, like:

This will help us improve churn, we estimate a recovered revenue of around 1 M€/year if we ship the fix this quarter.

We are deciding whether to roll back an experiment that is currently costing us ~20 k€/week in lost conversions.

Without this segmentation we cannot launch the campaign, and the media budget (300 k€) is already committed for next month.

A regulatory deadline on the 15th. Missing it exposes us to fines of up to 500 k€.

The CEO is presenting to the board on Thursday and needs the Q1 retention numbers.

Agree on definitions before comparing numbers

Before saying “the numbers don’t match”, check that everyone is computing the same thing. Common sources of confusion:

• Conversion rate: visitors to signups? signups to paid? in what time window?
• Active user: logged in this month? performed a key action? has an active subscription?
• Revenue: gross or net? booked or recognized? including refunds?
• Booking date vs. payment date: the same order can fall in different months depending on which one you use.
• Exclusions: do internal accounts, test users, and refunded orders count?

Report problems with reproducible examples

When something looks wrong, a vague report (“the numbers are broken”) forces the data team to play detective. A good bug report includes:

• The dashboard or query you were looking at.
• The filters applied (date range, segment, country, etc.).
• What you expected to see and why.
• What you actually saw.
• A concrete example: “Order #12345 should be in the paid bucket but appears as pending.”

Communicate incidents, fixes, and backfills

If historical data is corrected, reprocessed, duplicated, deleted, or changed retroactively, the data team needs to know before dashboards start showing weird numbers.

Examples of changes worth announcing:

• A bug in the checkout flow created duplicate orders for two days, and they were just cleaned up.
• A backfill is being run to recompute historical commissions.
• A migration moved data from one table to another, with possible row-count differences.
• A retroactive policy change is updating old records.

Do not use spreadsheets as hidden databases

Spreadsheets are great for ad-hoc analysis. They are terrible as operational systems, yet companies routinely let them become the source of truth for commissions, active clients, country-to-region mappings, or discount eligibility rules.

Warning signs a spreadsheet has crossed the line:

• No clear owner, but multiple editors.
• Pipelines, reports, or business processes depend on it.
• No schema, validation, or version control.
• “Don’t touch row 47, it has a special formula.”

Spreadsheets have no referential integrity, no audit log, no real access controls, and no review process. One accidental sort or hidden filter can silently corrupt everything, and you will find out weeks later, when the numbers stop adding up.

Be explicit about sensitive data

If a source contains personal, financial, health, or otherwise sensitive data, say it clearly and early. The data team can then apply the right controls from the start: access restrictions, masking, encryption at rest, retention policies, audit logs.

Categories worth flagging explicitly:

• PII: names, emails, phone numbers, addresses, IDs.
• Financial: card numbers, bank accounts, salary information.
• Health: medical records, diagnoses.
• Authentication: passwords (which should never be in analytics anyway), tokens, secrets.

Data quality is a shared responsibility

The data team can clean, model, test, and document data, but they cannot fully compensate for problems at the source. No amount of dbt tests will fix:

• Missing or unreliable timestamps.
• Unstable or recycled IDs.
• Silent hard deletions.
• Undocumented schema changes.
• Ambiguous business definitions.

If the habits in this post become part of how product, engineering, and business teams work, the data team can spend less time firefighting and more time doing what they are actually good at: turning data into decisions.