Databricks Data Engineer Professional Practice Test

Correct answer: A

WHY A: For a multi-table pipeline with quality checks, dimension joins, and multiple output tables, Lakeflow SDP provides substantial advantages: automatic checkpoint management (no manual checkpoint path configuration), declarative data quality with CONSTRAINT clauses, automatic DAG dependency resolution between Silver and Gold tables, Unity Catalog lineage, and the AUTO CDC API for SCD patterns. Teams with mixed Spark expertise benefit from the higher-level declarative abstractions that reduce boilerplate and error-prone state management code. WHY NOT B: Lakeflow SDP fully supports Kafka as a streaming source via Spark Structured Streaming's Kafka connector under the hood. WHY NOT E: Lakeflow SDP supports both TRIGGERED and CONTINUOUS modes. CONTINUOUS mode runs as a persistent streaming job with sub-minute latency, comparable to direct Structured Streaming. There is no fixed 2–5 minute delay. WHY NOT D: Lakeflow SDP uses Python and SQL, not a proprietary YAML language. Pipeline definitions are written in SQL CREATE STREAMING TABLE / AUTO CDC INTO syntax or Python decorators — these are standard languages, not new abstractions. WHY NOT C: Lakeflow SDP has no table count limit. There is no 10-table restriction; production pipelines can contain dozens or hundreds of tables.

Correct answer: A

WHY A: In SQL Alerts, TRIGGERED indicates the alert condition was met (the condition evaluated true) on the most recent evaluation. WHY NOT B: Query failures are represented as ERROR, not TRIGGERED. WHY NOT D: A false condition maps to OK. WHY NOT C: Paused alerts generally do not execute; TRIGGERED is a result state from evaluation.

Correct answer: D

WHY D is correct: Unity Catalog enforces two strict rules for column masks: (1) each column can have at most ONE mask function assigned at any time — attempting to add a second mask to the same column is an error; (2) the SQL UDF used as the mask must return the same data type as the column being masked. This prevents type coercion issues at query time. WHY NOT B: Multiple masks per column are NOT allowed. Only a single mask function can be active on a column at any time. WHY NOT E: Column masks are defined as SQL UDFs, not Python UDFs. While the UDF can reside in any schema the user has permission to access, it does not have to be in the same schema as the table. WHY NOT C: Column masks cannot be applied to views. They are a table-level feature and apply to base tables only. WHY NOT A: On Databricks Runtime versions below 12.2 LTS, access to tables with row filters or column masks fails securely — the query returns NO data rather than exposing plain text. This is the opposite of what is described.

Correct answer: B

WHY B: The Databricks documentation explicitly lists features unique to Unity Catalog managed tables: (1) Predictive optimization — automatically runs OPTIMIZE, VACUUM, and ANALYZE; must be manually scheduled for external tables. (2) Automatic liquid clustering — intelligently selects clustering keys based on workload patterns, enabled via predictive optimization; not available for external tables. (3) Metadata caching — in-memory caching of Delta transaction metadata to reduce cloud storage requests; always enabled for managed tables. (4) Automatic file deletion after DROP TABLE — managed tables delete underlying cloud storage files after 8 days; external tables require manual storage cleanup. WHY NOT A: ACID transactions, time travel, and schema enforcement are Delta Lake features available to both managed and external Delta tables — not exclusive to managed tables. WHY NOT E: Delta Sharing and credential vending work with both managed and external tables through Unity Catalog; they are not managed-table-exclusive. WHY NOT C: Automatic liquid clustering (with predictive optimization) is exclusive to managed tables — external tables cannot benefit from automatic clustering key selection. WHY NOT D: Unity Catalog access controls, row/column security, and auditing apply to all tables registered in Unity Catalog, including external tables — they are not exclusive to managed tables.

Correct answer: B

WHY B is correct: The Repair job run dialog in the Databricks UI explicitly supports parameter overrides for the repair run. When initiating a repair, engineers can add or change task parameters and job-level parameters that will apply only to the repair execution — the original job definition remains unchanged. This is the designed workflow for remediating a failed run caused by a bad parameter value while preserving the job's original configuration and the results of tasks that already completed successfully. WHY NOT D: Editing the job definition changes the permanent default parameter, which would affect all future runs. It also requires triggering a completely new run from scratch, re-executing tasks that may have already completed successfully — which wastes compute resources and loses the efficiency benefit of the repair run mechanism. WHY NOT C: Cloning the job is an unnecessarily complex workaround. It requires infrastructure cleanup after the fact, does not leverage the built-in repair run functionality, and triggers a full new run that re-executes all tasks (including tasks that may have already succeeded) rather than just the failed ones. WHY NOT A: Submitting a new run via the API with the corrected parameter is a valid workaround but creates an entirely new run rather than repairing the original. This wastes compute by re-running successful tasks and creates an additional run record, complicating run history tracking. It also does not preserve the original run's context for audit purposes.

Correct answer: A

WHY A is correct: Liquid clustering is the Databricks-recommended approach for this scenario based on multiple matching criteria documented explicitly: (1) 'Tables that are often filtered by high cardinality columns' — customer_id with 10 million distinct values is a canonical high-cardinality case where partitioning would create millions of tiny partitions but liquid clustering handles efficiently. (2) 'Tables that have concurrent write requirements' — liquid clustering is specifically cited as benefiting tables with concurrent writes, unlike traditional partitioning which can cause more write conflicts. (3) If access patterns change (e.g., new filters emerge), clustering keys can be changed via ALTER TABLE CLUSTER BY without rewriting all existing data. (4) Databricks recommends liquid clustering for all new tables. WHY NOT B: Partitioning by customer_id with 10 million distinct values would create 10 million partitions — a classic over-partitioning problem. Databricks explicitly warns: 'Tables where a typical partition key could leave the table with too many or too few partitions' are a scenario that particularly benefits from liquid clustering instead. Over-partitioned tables suffer from severe metadata overhead, too many small files, and poor query performance. WHY NOT D: While pairing date partitioning with ZORDER BY (customer_id) is a valid legacy strategy, Databricks now recommends liquid clustering as the successor to this approach. Additionally, ZORDER is not idempotent, cannot cross partition boundaries, and requires full OPTIMIZE re-runs after each batch of writes. Databricks documentation states: 'Databricks recommends using liquid clustering for all new tables. You cannot use ZORDER in combination with liquid clustering.' WHY NOT C: While tables under 1 TB generally don't need partitioning, the specific access pattern here — primarily filtering by a high-cardinality column (customer_id) — means ingestion time clustering (which clusters by insertion order, not by customer_id) would NOT efficiently serve these queries. Liquid clustering on customer_id physically co-locates records for the same customer, enabling data skipping that ingestion time clustering cannot provide.

Correct answer: C

WHY C: Databricks-to-Databricks Delta Sharing is designed for sharing between Unity Catalog-enabled Databricks workspaces (often across accounts/clouds). It supports assets not available in open sharing (for example notebook files and Unity Catalog volumes) and avoids provider-managed bearer tokens by using the Databricks-managed identity/sharing identifier flow. WHY NOT B: Open sharing (D2O) is for recipients on any platform, but it typically uses bearer tokens or OIDC federation and does not support notebook/volume/model sharing. WHY NOT A: The open-source server is for customer-managed implementations and is not required for D2D on Databricks. WHY NOT D: If both workspaces are attached to the same Unity Catalog metastore, you can govern access with Unity Catalog directly, but the scenario describes sharing across organizations/metastores and explicitly includes Delta Sharing-specific assets.