A Silver layer Delta table is partitioned by transaction_date and a nightly maintenance job runs OPTIMIZE ... ZORDER BY (customer_id) on the latest partition. The team is evaluating a migration to liquid clustering. Which of the following accurately describes the KEY behavioral difference between Z-ordering and liquid clustering's OPTIMIZE that is most relevant to this decision?
Answer & explanation
Correct answer: C
WHY C: This is the documentation-backed key distinction. ZORDER is documented as NOT idempotent: 'Z-ordering is not idempotent but aims to be an incremental operation.' For a table with daily appends, each new batch of data is scattered across files and is re-incorporated into the Z-order layout on the next OPTIMIZE run, potentially causing wide rewrites of already-Z-ordered files in the same partition. Liquid clustering's OPTIMIZE is designed to be incremental: it rewrites the data files that need clustering (not yet processed, or with enough unclustered data to justify rewriting) and skips files already in a good clustering state. This dramatically reduces write amplification for tables with frequent incremental writes, making nightly maintenance jobs shorter and less resource-intensive.
WHY NOT A: Z-ordering in Delta Lake does not use Hilbert curves in the same way as liquid clustering. More importantly, the incremental vs. non-incremental nature of OPTIMIZE is a more fundamental and exam-relevant difference than the scheduling method.
WHY NOT B: Z-ordering is explicitly documented as NOT idempotent. It aims to be incremental but does not converge to a no-op on subsequent runs.
WHY NOT D: Liquid clustering supports a maximum of FOUR clustering keys, not 32. The 32-column figure is the default number of columns for which statistics are collected, not clustering key capacity.
WHY NOT E: ALTER TABLE REMOVE PARTITIONING is not a supported Delta Lake command. Migration from partitioned + ZORDER to liquid clustering uses a CTAS approach without PARTITION BY, not an ALTER TABLE modification.