A data engineering team enables change data feed on a Silver Delta table (delta.enableChangeDataFeed = true) to power a downstream audit pipeline. Six months later, they attempt to read CDF changes starting from version 1 (the day CDF was enabled). The read fails with an error indicating the version is no longer available. Separately, after a DBA runs ALTER TABLE silver RENAME COLUMN amount TO total_amount, an ongoing CDF streaming read on the table also fails. Which explanation correctly identifies the root cause for BOTH failures?
Answer & explanation
Correct answer: A
WHY A: Two separate root causes are identified. (1) Version availability: CDF records are documented as TRANSIENT. They follow the table's retention policy, and VACUUM removes old table versions' files at regular intervals; once version 1 is beyond the retention window, its change data can no longer be read. The documentation states: 'When a version is removed, you can no longer read the change data feed for that version.' CDF is not a permanent audit log and records are eventually purged. (2) Column rename: Delta column mapping permits renaming columns without rewriting data files, but CDF has documented limitations after non-additive schema changes (renaming, dropping, or changing types of columns). In DBR 12.2 LTS and below, CDF streaming reads on column-mapped tables that experienced non-additive changes fail outright. In DBR 13.x+, schema tracking helps ongoing streams, but a streaming read still does not span a version range that contains a non-additive schema change.
WHY NOT B: Column renames do not reset CDF enablement. The delta.enableChangeDataFeed = true property persists across schema changes, so nothing needs to be re-enabled.
WHY NOT C: Permissions are not the issue: the failures come from data retention and schema evolution constraints, not access control. SELECT WITH HISTORY is not a Delta Lake or Unity Catalog privilege.
WHY NOT D: It is true that CDF records changes from the point it is enabled, but that is NOT why reading version 1 fails SIX MONTHS later: the version was readable at first and became unavailable after retention/VACUUM removed it. The rename failure comes from the non-additive schema change, not from DDL versions being excluded from CDF output.
WHY NOT E: CDF reads are not managed or blocked by predictive optimization. There is no _change_data access restriction tied to the OPTIMIZE cycle.