When the Right Answer Is Wrong Code

"We need to replicate the bug."

Everyone on the call went quiet for a second. Not because the idea was crazy. Because it was right, and it made people uncomfortable.

We were five weeks into a financial data migration at a cloud security company — moving five years of revenue analytics from a legacy BI platform to dbt + Snowflake. We had a validated pipeline, a working reconciliation methodology, and a fiscal quarter macro that was definitively, provably wrong. And the right move was to write that wrong code on purpose.

---

Why "just fix the bug" is the wrong instinct

The instinct is correct in isolation: you found a bug, you fix it. That's your job.

The problem is that you're not working in isolation. You're migrating a system that has been running in production — wrong, but consistently wrong — for some period of time. Downstream systems have adapted. Reports have been calibrated against the buggy output. Finance has been looking at those numbers for long enough that they've built intuitions around them. Comparisons, targets, board presentations — all developed relative to the wrong baseline.

When you correct the bug in the middle of a migration, you now have two things different between your new system and the old one: the migration itself, and the bug fix. Every downstream variance becomes ambiguous. You can't tell whether you made a migration error or your fix is working as intended. Your validation methodology breaks down.

The clean answer is also the counterintuitive one: replicate the bug first. Get to zero variance on the legacy output — including its mistakes. Then fix it, in a separate, explicit, documented step.

This principle has a name in our workflow: parity first, correctness second.

---

The decision framework

Not every legacy bug deserves replication. Here's how we decide.

Replicate when:

• The bug has been in production long enough for downstream systems to adapt to it
• Downstream reports, dashboards, or financial models reference the buggy output directly
• You need clean parity validation before you can confidently certify the migration
• The fix will produce a visible change in output that Finance or stakeholders need to approve explicitly

Fix immediately when:

• The bug is recent (days or weeks), with no accumulated downstream adaptation
• The bug is silent — produces wrong output that no downstream consumer has ever seen
• No validation comparison depends on the buggy behavior
• The fix is isolated enough that it won't create ambiguous variance

The timeline question is the most important one. A bug active for 15 months has probably generated a year of decisions against wrong numbers. A bug introduced last Tuesday has not. The calculus is different.

The audit trail requirement applies regardless. Whether you replicate or fix immediately, the decision needs to be documented. Not just "we replicated the bug" — but what the bug was, how long it was active, what it affected, and why replication was the right call. That documentation protects both you and your client when Finance asks why Q2FY25 numbers look different from historical comparisons.

---

---

The fiscal quarter macro: a three-way comparison

Here is the real example. A cloud security company's legacy BI system had a fiscal quarter conversion macro with a typo. Q2FY25 was defined as starting 2023-05-01 instead of 2024-05-01. The result: Q2FY25 spanned 15 months instead of 3. This was discovered on mid-December — the bug had been in production since sometime in 2023.

The error cascaded into ROW_NUMBER() window functions that used fiscal quarter partitioning to select the "last month" of each quarter for ARR calculations. With Q2FY25 spanning 15 months, the wrong month was selected as the quarter's representative value. Quarterly ARR figures for that period were incorrect across every dashboard that used them.

The legacy buggy version — what the legacy system had been computing for 15 months:

-- Version 1: Legacy system exact replication
-- File: convert_to_fiscal_quarter_legacy_exact.sql

{#
  WARNING: REPLICATES LEGACY SYSTEM BUG FOR 100% PARITY

  BUG DETAILS:
  - Q2FY25 is defined as: period >= '2023-05-01' and period <= '2024-07-01'
  - Should be: period >= '2024-05-01' and period <= '2024-07-01'
  - This causes Q2FY25 to span 15 months (May 2023 through July 2024)
  - This makes ROW_NUMBER() select wrong "last month" for Q2FY25

  This bug causes incorrect quarterly ARR values in the legacy system.
  We replicate it here for validation parity, but fix in a separate PR.
#}

{% macro convert_to_fiscal_quarter_legacy_exact(date_column) %}
  case
    -- ... (hardcoded cases through FY24) ...
    when {{ date_column }} >= '2024-02-01'
    and {{ date_column }} <= '2024-04-01'
      then 'Q1FY25'
    -- BUG: This should start from 2024-05-01, not 2023-05-01
    -- Replicating legacy system typo for validation parity
    when {{ date_column }} >= '2023-05-01'
    and {{ date_column }} <= '2024-07-01'
      then 'Q2FY25'
    -- ... (continues through Q4FY26) ...
  end
{% endmacro %}

The corrected dynamic version — what the fiscal calendar actually requires:

-- Version 2: Correct dynamic version
-- File: convert_to_fiscal_quarter.sql

{#
  Fiscal Year Definition:
  - Fiscal year starts February 1, ends January 31
  - Q1: February - April
  - Q2: May - July
  - Q3: August - October
  - Q4: November - January (spans calendar years)

  Dynamic logic works for any date, is fully maintainable.
  Results identical to legacy system hardcoded values — except the bug.
#}

{% macro convert_to_fiscal_quarter(date_column) %}
  case
    when extract(month from {{ date_column }}) in (2, 3, 4) then
      'Q1FY' || right(cast(extract(year from {{ date_column }}) + 1 as varchar), 2)
    when extract(month from {{ date_column }}) in (5, 6, 7) then
      'Q2FY' || right(cast(extract(year from {{ date_column }}) + 1 as varchar), 2)
    when extract(month from {{ date_column }}) in (8, 9, 10) then
      'Q3FY' || right(cast(extract(year from {{ date_column }}) + 1 as varchar), 2)
    when extract(month from {{ date_column }}) in (11, 12) then
      'Q4FY' || right(cast(extract(year from {{ date_column }}) + 1 as varchar), 2)
    when extract(month from {{ date_column }}) = 1 then
      'Q4FY' || right(cast(extract(year from {{ date_column }}) as varchar), 2)
    else null
  end
{% endmacro %}

Now here's the part people don't write about. We didn't just have these two versions. We had a third: the deliberately replicated bug version used in validation models. It looks like the buggy original — but the comments make explicit that this is intentional, documented, and temporary.

-- Version 3: Intentional replication with full audit trail
-- Used in: fct_arr_validation (validation model only — not production)

{#
  INTENTIONAL BUG REPLICATION — FOR VALIDATION PARITY ONLY

  Why this exists:
  - The legacy system has a typo in Q2FY25 fiscal quarter assignment
  - Q2FY25 starts 2023-05-01 in the legacy system; it should start 2024-05-01
  - This macro is used ONLY in validation/reconciliation models
  - Its purpose is to produce identical output to the legacy system so we can isolate
    migration errors from intentional fixes

  What happens next:
  - Once parity is validated at 0.00% variance using this macro,
    fct_arr_v2 is rebuilt using convert_to_fiscal_quarter (correct version)
  - The variance introduced by the fix is documented and presented to Finance
  - This macro is deprecated after that PR merges

  Decision made: [date]
  Approved by: [client lead]
  Tracked in: [the migration PR]
#}

{% macro convert_to_fiscal_quarter_legacy_exact(date_column) %}
  -- Identical logic to the buggy legacy version — intentional.
  -- Do not "fix" this macro. See comment above.
  case
    when {{ date_column }} >= '2024-02-01'
    and {{ date_column }} <= '2024-04-01'
      then 'Q1FY25'
    when {{ date_column }} >= '2023-05-01'
    and {{ date_column }} <= '2024-07-01'
      then 'Q2FY25'
    -- ...
  end
{% endmacro %}

The comment is the work. Anyone reading that file knows exactly what it is, why it exists, when it was created, and what makes it safe to deprecate. Without those comments, the next engineer to touch it either "fixes" the bug and breaks validation, or ships it to production thinking it's correct.

Three separate models used this validation macro. Each validated to 0.00% variance against the legacy system before we touched the fiscal quarter correction.

---

---

How this looks in a validation workflow

The parity-first methodology plays out as a two-PR sequence:

PR 1 — Parity (uses the replicated-bug macro):

-- Validation query: confirms we've matched legacy output exactly
-- Run against historical data before any fixes are applied

with legacy as (
  select
    fiscal_quarter,
    sum(arr) as legacy_arr
  from fct_arr_legacy  -- legacy system output
  group by fiscal_quarter
),
migrated as (
  select
    -- Using the deliberately buggy macro for parity validation
    convert_to_fiscal_quarter_legacy_exact(period_month) as fiscal_quarter,
    sum(arr) as migrated_arr
  from fct_arr_v2_parity
  group by fiscal_quarter
)
select
  coalesce(l.fiscal_quarter, m.fiscal_quarter) as fiscal_quarter,
  l.legacy_arr,
  m.migrated_arr,
  m.migrated_arr - l.legacy_arr as diff,
  round((m.migrated_arr - l.legacy_arr) / nullif(l.legacy_arr, 0) * 100, 4) as pct_diff
from legacy l
full outer join migrated m on l.fiscal_quarter = m.fiscal_quarter
order by fiscal_quarter
-- Expected: all rows show pct_diff = 0.0000

PR 2 — The fix (swaps in the correct macro, documents the expected variance):

-- After PR 1 is merged and validated at 0.00%:
-- Swap convert_to_fiscal_quarter_legacy_exact → convert_to_fiscal_quarter
-- Re-run the same validation query
-- Q2FY25 will now show a variance — this is expected and correct
-- That variance is the bug being fixed, not a migration error
-- Present to Finance with documentation of the Q2FY25 date boundary correction

The second PR produces a known, expected, documented variance. Finance sees it, understands it, approves it. That's the outcome. Not silence. Not an invisible fix. A decision.

---

Lessons learned

Bug replication is documentation of a decision, not a failure of engineering.

The engineer who writes the three-comment block above — "INTENTIONAL BUG REPLICATION — FOR VALIDATION PARITY ONLY" — is doing more useful work than the engineer who fixes the bug silently. They're capturing a decision that would otherwise be invisible. They're protecting every future engineer from an ambiguous variance. They're giving Finance a clean conversation instead of a confusing one.

The discomfort people feel when someone says "we need to replicate the bug" comes from conflating correctness with cleanliness. Correct code has no bugs. Clean code documents its decisions. A temporarily wrong macro with perfect comments is cleaner than a silently correct one that leaves behind unanswered questions.

The parity-first discipline makes validation meaningful. If you correct and validate simultaneously, you can't interpret your variance numbers. A 2% variance after a migration that also fixed three bugs could mean excellent work or it could mean a serious error — you can't tell. Separate them. Zero variance on the buggy output is a clear milestone. The fix variance is a separate, interpretable number.

Document the deprecation path before you merge. The replicated-bug macro has a job to do and a date when that job is done. Write the deprecation trigger into the comments at creation time — "deprecated after PR merges," "once parity is validated, remove this macro." Without that, replicated-bug code has a tendency to stick around past its expiration date.

---

This migration found 15 bugs in the legacy system — the fiscal quarter typo was one of them. For a full account of the other 14 and how systematic reconciliation found them, see 15 Silent Bugs We Found Migrating a Financial BI Platform. For the validation methodology that made parity comparisons possible at all, see How We Validated a Financial Data Migration to 0.002% Accuracy.

---

The right answer to "we need to replicate the bug" is not to stare. It's to ask one question: how long has this been in production?

Fifteen months. Then yes, you replicate it. You write the wrong code, you write the right comments, and you ship the fix in the PR where it belongs — with full visibility, full documentation, and a Finance team that understands exactly what changed and why.