Home / DE Professional practice test / Ensuring Data Security and Compliance

Free · 8 questions with explanations

Ensuring Data Security and Compliance: Databricks Data Engineer Professional Practice Questions

Exam-style questions on Ensuring Data Security and Compliance. Pick your answer, then open the explanation to see why it's right — and why the other options are wrong.

Updated November 16, 2026

1 Ensuring Data Security and Compliance Medium

A data engineering team implements GDPR compliance in a medallion architecture. After deleting user PII from the bronze layer tables, they need to propagate those deletions to the gold layer materialized views. What action must the team take to ensure the materialized views reflect the deletions?

  1. AManually re-run DELETE statements on each gold materialized view targeting the same user_id values.
  2. BDrop and recreate the gold materialized views from scratch after each deletion batch.
  3. CAdd skipChangeCommits to the materialized view definition to ignore deletions from the bronze source.
  4. DRefresh the materialized views and run maintenance; they will automatically handle the upstream deletions.
  5. ESet delta.enableChangeDataFeed = true on the bronze tables before running deletions.
Show answer & explanation

Correct answer: D

WHY D is correct: Per Databricks documentation, materialized views automatically handle deletions from their source tables. A materialized view always returns a correct result — it uses incremental computation unless full recomputation is required for correctness (e.g., after a deletion that changes aggregations). You do NOT need to manually delete records from materialized views. The team only needs to refresh the materialized view and run maintenance to ensure deletions are completely processed and the underlying data files are cleaned up. WHY NOT A: Materialized views are derived objects; you cannot run DML (DELETE) statements directly on a materialized view in Databricks. Materalized views are refreshed from their source definitions, not modified directly. WHY NOT B: Dropping and recreating materialized views is unnecessary and expensive. Materialized views are designed to handle source deletions via refresh — recreation would also lose any additional configuration or metadata associated with the view. WHY NOT C: skipChangeCommits is a streaming table option, not a materialized view option. Materialized views do not use this concept — they handle corrections automatically through full or incremental recomputation. WHY NOT E: Enabling change data feed on bronze tables helps with streaming CDC pipelines but it is not a requirement for materialized view refresh after deletions. Materialized views compute correctness independently of whether CDF is enabled on source tables.

2 Ensuring Data Security and Compliance Hard

A data engineer has a Lakeflow Spark Declarative Pipelines pipeline where a streaming table clicks_bronze reads from a Delta table source source_clicks. To comply with GDPR, the engineer deletes rows from source_clicks using DML and simultaneously deletes the same rows from clicks_bronze. When the pipeline runs again, it fails with an error stating that the streaming source contains non-append operations. What change must the engineer make to clicks_bronze to resolve this without doing a full refresh?

  1. ASwitch clicks_bronze from a streaming table to a materialized view that recomputes from the source.
  2. BAdd .option('skipChangeCommits', 'true') to the readStream call in the clicks_bronze definition.
  3. CEnable change data feed on source_clicks and read it with .option('readChangeFeed', 'true').
  4. DDelete the clicks_bronze streaming checkpoint and re-run the pipeline from the earliest offsets.
  5. ESet spark.databricks.delta.retentionDurationCheck.enabled to false on the streaming cluster.
Show answer & explanation

Correct answer: B

WHY B: Streaming tables in Lakeflow Spark Declarative Pipelines process append-only data when they stream from Delta sources. When DML operations (DELETE, UPDATE) are applied to the source Delta table, the streaming read encounters non-append transactions and fails. The documented GDPR compliance pattern is to add .option('skipChangeCommits', 'true') to the readStream definition: the streaming table then skips non-append commits (deletes/updates) and continues processing new inserts, so the pipeline proceeds after DML-based deletions. WHY NOT A: A materialized view does handle deletions from its source, but it is a different object type rather than a change to the streaming table, and it requires rewriting the pipeline definition. The question asks how to fix the streaming table without a full refresh. WHY NOT C: Reading the change data feed exposes the delete events, which then need extra logic in the pipeline code to handle them; it does not simply skip them. It is a more complex redesign and not the recommended fix for this scenario. WHY NOT D: Deleting the checkpoint and re-processing from the start is equivalent to a full refresh. It is expensive, can lose data if the upstream source has aged out data (e.g., Kafka with retention), and is not the targeted fix for the non-append commit problem. WHY NOT E: retentionDurationCheck.enabled controls how short a VACUUM retention period can be set. It is unrelated to streaming table behavior or non-append commit handling.

3 Ensuring Data Security and Compliance Easy

A data engineer needs to grant a group of analysts the ability to query a table named catalog1.schema1.sales in Unity Catalog. The analysts currently have no privileges on any securable objects. What is the MINIMUM set of privileges required?

  1. AGrant SELECT on catalog1.schema1.sales, with no privileges on its parent schema or catalog.
  2. BGrant MODIFY on catalog1.schema1.sales and USE CATALOG on catalog1, with no grant on the schema.
  3. CGrant ALL PRIVILEGES on catalog1.schema1.sales, which also covers its parent schema and catalog.
  4. DGrant SELECT on catalog1.schema1.sales, USE SCHEMA on catalog1.schema1, and USE CATALOG on catalog1.
  5. EGrant SELECT on catalog1.schema1.sales and USE CATALOG on catalog1, with no grant on the schema.
Show answer & explanation

Correct answer: D

WHY D: Unity Catalog enforces a hierarchical privilege model. To SELECT from a table, a user also needs USE SCHEMA on the parent schema and USE CATALOG on the parent catalog. The three privileges are required together; SELECT alone is insufficient even though it is granted directly on the table. WHY NOT A: SELECT on the table alone is not sufficient. Without USE SCHEMA and USE CATALOG, the query engine cannot navigate to the table and the request is denied. WHY NOT C: ALL PRIVILEGES grants a broad set of table capabilities (SELECT, MODIFY, APPLY TAG), which exceeds the minimum and violates least privilege. It also does not cover the parents: USE SCHEMA and USE CATALOG are not table-level privileges and still need to be granted separately. WHY NOT B: MODIFY implies write access (INSERT, UPDATE, DELETE) and is far beyond what read-only analysts need. USE SCHEMA on the schema is also missing, so the query would fail. WHY NOT E: USE SCHEMA on catalog1.schema1 is also required. Without it, navigation stops at the catalog level and the schema and its objects are inaccessible.

4 Ensuring Data Security and Compliance Easy

A healthcare data team is preparing a dataset for statistical analysis. The dataset contains exact patient ages (e.g., 34, 67, 22). The compliance officer requires that the published dataset cannot be used to re-identify individual patients from their age alone, but the distributed age values must still be useful for demographic analysis. Which technique should the data engineer apply?

  1. AHashing — apply SHA-256 to each age value to produce a fixed-length token.
  2. BSuppression — remove the age column from the published dataset before release.
  3. CTokenization — swap each age for a random token and keep a reverse mapping.
  4. DGeneralisation — replace exact ages with ranges such as '20-29' or '30-39'.
Show answer & explanation

Correct answer: D

WHY D: Generalisation replaces specific values with broader categories or ranges, reducing the granularity of the data so that individuals are not singled out while the aggregate/demographic signal is preserved. Replacing exact ages with 10-year buckets (e.g., 34 → '30-39') is the canonical application of generalisation and is widely used in k-anonymity implementations for healthcare and research data. WHY NOT A: Hashing converts exact ages to opaque fixed-length strings. This hides the age, but no statistical distribution or trend can be derived from a set of SHA-256 digests, so demographic utility is lost. WHY NOT B: Suppression removes the column. This satisfies the privacy requirement but eliminates demographic analysis utility, which contradicts the requirement. WHY NOT C: Tokenization replaces the age with a random token while retaining a lookup table. The tokens carry no demographic meaning and are useless for statistical analysis, and the retained reverse mapping means the data is pseudonymized rather than anonymized.

5 Ensuring Data Security and Compliance Medium

A security engineer is applying column masks to a Unity Catalog table. Which of the following statements correctly describes the constraints on column masks?

  1. AA column can have several masks applied at once, and they are evaluated in the order they were added.
  2. BA column mask is defined as a Python UDF that is stored in the same schema as the table it protects.
  3. CColumn masks can be applied to views as well as base tables, to keep the masking logic in one place.
  4. DEach column can have at most one mask, and the mask function returns the same data type as the column.
  5. EColumn masks work on runtimes below 12.2 LTS as well, where the data is returned in plain text.
Show answer & explanation

Correct answer: D

WHY D: Unity Catalog enforces two strict rules for column masks: (1) each column can have at most one mask function assigned at any time, and attempting to add a second mask to the same column is an error; (2) the SQL UDF used as the mask must return the same data type as the column being masked. This prevents type coercion issues at query time. WHY NOT A: Multiple masks per column are not allowed. A single mask function can be active on a column at any time, so there is no evaluation order to speak of. WHY NOT B: Column masks are defined as SQL UDFs, not Python UDFs. The UDF can reside in any schema the user has permission to access; it does not have to be in the same schema as the table. WHY NOT C: Column masks cannot be applied to views. They are a table-level feature and apply to base tables. WHY NOT E: On Databricks Runtime versions below 12.2 LTS, access to tables with row filters or column masks fails securely: the query returns no data rather than exposing plain text. This is the opposite of what the option describes.

6 Ensuring Data Security and Compliance Medium

A data governance team needs to store email addresses in a way that allows analysts to join datasets by user identity without ever seeing the raw email values. Which anonymization or pseudonymization technique is MOST appropriate for this requirement?

  1. ASuppression — remove the email column entirely from all datasets before storing.
  2. BGeneralisation — replace each email with a broader category such as the email domain.
  3. CTokenization — replace each email with a random token while maintaining a secure reverse-lookup mapping.
  4. DDeterministic hashing — replace each email with a consistent hash value using a function such as SHA-256.
  5. EData masking via a column mask UDF — show partial email values such as 'a***@example.com'.
Show answer & explanation

Correct answer: D

WHY D is correct: Deterministic (consistent) hashing maps each unique email to the same hash value every time. This means analysts can JOIN two hashed datasets on the hash column to correlate activity across tables — without ever accessing the raw email. Because SHA-256 (or similar) is a one-way function, analysts cannot reverse the hash to recover the original value, satisfying the anonymization requirement. WHY NOT A: Suppression removes the data entirely, which means there is no identifier left to join on. This would make correlation across datasets impossible, which contradicts the requirement. WHY NOT B: Generalisation replaces specific values with broader categories (e.g., exact age → age range, full email → domain). While it reduces identifiability, joining on a domain like 'gmail.com' is not meaningful for identifying individual users across datasets. WHY NOT C: Tokenization is a pseudonymization technique — it replaces PII with a token while retaining a reverse-mapping. Because re-identification is possible via the mapping table, it does not fully anonymize the data. Additionally, if the goal is solely to enable joins without re-identification risk, hashing is simpler and does not require a mapping store. WHY NOT E: Partial masking (showing part of the email) still exposes PII and would not prevent a determined analyst from identifying users. It is also reversible for common email patterns and does not constitute anonymization.

7 Ensuring Data Security and Compliance Medium

A data engineer is building a scheduled VACUUM job to comply with a strict data retention policy that requires deleted data to be permanently removed within 3 days. When they attempt to set the retention threshold to 3 days, the VACUUM command fails with a safety check error. What must the engineer do to allow VACUUM to run with a sub-7-day retention threshold, and what is the risk?

  1. ASet delta.deletedFileRetentionDuration = interval 3 days as a table property and run VACUUM; this carries no additional risk.
  2. BRun VACUUM table_name LITE with a 3-day threshold; LITE mode skips the safety check, at the risk of leaving stale files behind.
  3. CSet spark.databricks.delta.retentionDurationCheck.enabled = false; VACUUM can then delete uncommitted files of long-running jobs.
  4. DAsk Databricks support to enable sub-7-day retention for the workspace; the risk is losing time travel beyond the 3-day window.
  5. ERun VACUUM table_name FULL with a 3-day threshold; FULL mode accepts short thresholds, at the cost of a longer file listing.
Show answer & explanation

Correct answer: C

WHY C: Databricks enforces a safety check that prevents VACUUM from running with a retention threshold below 7 days by default. To override it, the engineer sets the Spark configuration spark.databricks.delta.retentionDurationCheck.enabled to false on the cluster or in the session. The documented risk is significant: if running jobs (e.g., long-running streaming or batch jobs) are writing data files that have not yet been committed to the Delta transaction log, VACUUM with a short threshold can permanently delete those uncommitted files before the job finishes, causing the job to fail or producing corrupted/incomplete tables. WHY NOT A: The delta.deletedFileRetentionDuration table property controls retention for tables on Databricks Runtime 18.0+ (or Unity Catalog managed tables on DBR 12.2+). The safety check that blocks VACUUM below 7 days is a separate, independent mechanism: even after setting the property, VACUUM still fails unless retentionDurationCheck.enabled is also set to false, and a short retention does carry risk. WHY NOT B: VACUUM LITE is a Public Preview mode (DBR 16.1+) that uses the transaction log to identify files to remove, avoiding full directory listings. It does NOT skip the safety check for short retention; the same minimum threshold applies in FULL and LITE modes. WHY NOT D: retentionDurationCheck.enabled is a standard, user-accessible Spark property. There is no workspace-level restriction that requires Databricks support. WHY NOT E: VACUUM FULL is the default mode of VACUUM and lists the files in the table directory to identify stale ones. It offers no bypass of the sub-7-day safety check; both FULL and LITE modes require disabling the check to use thresholds below 7 days.

8 Ensuring Data Security and Compliance Hard

A data engineer has a Delta table with deletion vectors enabled. After processing a GDPR 'right-to-be-forgotten' request, they run: ``sql DELETE FROM catalog1.schema1.users WHERE user_id = 42; VACUUM catalog1.schema1.users; `` After running these commands, a compliance auditor is concerned that the deleted user's data might still be physically readable in the Parquet files. Is the auditor's concern valid, and why?

  1. ANo. With deletion vectors enabled, DELETE rewrites the affected Parquet files at commit time, and VACUUM then removes the older copies.
  2. BNo. VACUUM with the default 7-day retention threshold removes deletion-vector-marked records from the current files in cloud storage.
  3. CYes. DELETE marks the rows in a deletion vector without rewriting the Parquet files; REORG TABLE ... APPLY (PURGE) is needed before VACUUM.
  4. DYes. VACUUM removes files older than the retention threshold, and no command exists to rewrite current files that hold soft-deleted rows.
  5. ENo. After VACUUM runs, the deleted records are encrypted at rest and are unreadable even if the Parquet files are accessed directly.
Show answer & explanation

Correct answer: C

WHY C: When deletion vectors are enabled on a Delta table, DELETE operations are 'soft deletes': they write a deletion vector (a metadata bitmap file) marking which rows are logically deleted, but the Parquet data files containing the original rows are NOT rewritten or removed, so the raw user data physically remains in the current files. VACUUM removes older data files that are no longer referenced by the transaction log, but it does not rewrite current files to remove soft-deleted data. The GDPR purge workflow is: (1) DELETE the records, (2) run REORG TABLE catalog1.schema1.users APPLY (PURGE) to physically rewrite the current Parquet files without the soft-deleted rows, and then (3) run VACUUM to remove the older (pre-REORG) files once they are outside the retention window. WHY NOT A: Deletion vectors exist to avoid rewriting Parquet files for performance, so DELETE does NOT rewrite files at commit time when they are enabled. The auditor's concern is valid. WHY NOT B: VACUUM removes older files that are no longer referenced by the transaction log beyond the retention threshold. It does not rewrite current, actively referenced Parquet files, so the user's rows (soft-deleted via deletion vector) remain in them after VACUUM alone. WHY NOT D: The concern is valid, but the reason given is inaccurate: a command to rewrite the current files does exist. After REORG TABLE ... APPLY (PURGE) the data is physically removed from the current files, and VACUUM then removes the now-old pre-REORG files, so the data does not remain forever. WHY NOT E: VACUUM does not encrypt data; it deletes files. Encryption at rest is a storage-layer protection that is unrelated to whether raw data bytes are readable by someone with direct cloud storage access.

Take the full DE Professional practice test →