Home / DE Professional practice test / Developing Code for Data Processing using Python and SQL / Question

Databricks Data Engineer Professional · free practice question

A data engineer has built an internal Python library and packaged it as a wheel file (`mylib-1.0.0-py3-none-any.whl`). The library contains custom data…

Exam-style question on Developing Code for Data Processing using Python and SQL from the free DE Professional practice test, with the answer and a full explanation below.

Updated November 16, 2026

Developing Code for Data Processing using Python and SQL Hard

A data engineer has built an internal Python library and packaged it as a wheel file (mylib-1.0.0-py3-none-any.whl). The library contains custom data validation logic used by multiple notebooks in a Databricks Job. The engineer needs to make this wheel available to the job cluster. During testing, after uploading the wheel to a Unity Catalog Volume path (/Volumes/dev/libs/wheels/mylib-1.0.0-py3-none-any.whl) and configuring the job cluster to use it, one notebook successfully imports mylib but a second notebook in the same job raises ModuleNotFoundError: No module named 'mylib'. The cluster logs show no installation errors. What is the MOST likely root cause, and what is the correct remediation?

  1. AThe second notebook is using %pip install to install a conflicting version of mylib that shadows the cluster-level wheel installation. The kernel restart triggered by %pip install in any notebook on the cluster invalidates the cluster-level library state for all notebooks sharing that Python process. The remediation is to remove the %pip install call from the conflicting notebook or ensure all notebooks use a consistent install method.
  2. BThe wheel file was uploaded to a Unity Catalog Volume, which is not a supported source for cluster library installation. Wheels must be uploaded to DBFS (dbfs:/FileStore/jars/) or stored in Workspace Files to be referenced as a cluster library. The remediation is to move the wheel to DBFS and update the library path reference.
  3. CUnity Catalog Volumes are supported as a wheel source for job clusters in Databricks Runtime 13.3 LTS and later. The issue is that the second notebook is attached to a different Python process isolation context (the cluster is in Shared Access Mode), which runs each notebook in a separate Python process where cluster-level libraries must be explicitly re-imported. The remediation is to add import importlib; importlib.reload(mylib) at the top of the second notebook.
  4. DThe wheel file was incorrectly packaged without a METADATA file, making it a malformed wheel that installs silently but cannot be imported. The first notebook succeeds only because it has mylib in its local workspace cache from a previous manual install. The remediation is to repackage the wheel using python -m build and validate it with twine check before uploading.
  5. EThe cluster is configured with Photon enabled, and Photon's vectorized execution engine uses a separate Python runtime that does not share the system Python library path where wheel installations are placed. The remediation is to disable Photon on the job cluster when using custom wheel dependencies that do not have Photon-compatible native bindings.
Answer & explanation

Correct answer: A

WHY A: Using %pip install in any notebook on a shared cluster triggers a Python kernel restart for that notebook's process. In some configurations this can conflict with or shadow cluster-level library installations. More critically, in Shared Access Mode clusters, each notebook runs in an isolated Python process — cluster-level libraries are installed into the shared driver/worker environment, but %pip install installs into the per-notebook virtual environment, which takes precedence and can produce inconsistent import behavior across notebooks. The remediation is to ensure a consistent installation method across all notebooks. WHY NOT B: Unity Catalog Volumes (path: /Volumes/...) ARE a supported source for cluster library installation in modern Databricks Runtimes. The wheel path format for Volumes is correctly specified. WHY NOT C: importlib.reload() reloads an already-imported module; it does not fix a ModuleNotFoundError where the module was never found in the first place. WHY NOT D: A malformed wheel would fail installation with an error in the cluster init logs, not install silently. The cluster logs show no installation errors, ruling this out. WHY NOT E: Photon is a vectorized query execution engine for SQL and DataFrame operations. It does not create a separate Python runtime or affect Python library installation paths.

Take the full DE Professional practice test free →

Would you pass today?

This question is one of 60 in the free, timed Databricks Data Engineer Professional practice test. Take it and get a section-by-section score.

Start the free test →