An Auto Loader stream ingests from a bucket that accumulates millions of files. Directory-listing mode has become slow and expensive because each trigger re-lists the whole path. Which change most directly reduces the per-trigger discovery cost?
Answer & explanation
Correct answer: A
WHY A: File notification mode subscribes to cloud storage event notifications, so Auto Loader learns about new files from a queue rather than re-listing a huge directory each trigger — the standard fix for high-file-count paths. WHY NOT B: A longer interval reduces how often listing runs, but each listing is still a full-directory scan and stays costly; it does not eliminate listing. WHY NOT C: Deleting the checkpoint causes already-processed files to be reprocessed and duplicated, not cheaper discovery. WHY NOT D: maxFilesPerTrigger only caps files per batch; it does not change the discovery mechanism. WHY NOT E: Converting to Delta changes the source format, but the cost here is listing files on the landing path.