Databricks Databricks-Certified-Data-Engineer-Professional 問題集 : Databricks Certified Data Engineer Professional Exam

A data engineer is brining an existing production Databricks job under asset bundle management and wants to ensure that:
- The job's current configuration is captured as YAML, and all
referenced files are included in their bundle project.
- Future changes to the bundle's YAML will update the existing job in-
place (not create a new job)
How should the data engineer successfully move the production job under asset bundle management?

• A. Run databricks bundle generate job --existing-job-id to generate the YAML and download referenced files. Then, run Databricks bundle deployment, bind to link the bundle's job resource to the existing job in Databricks.
• B. Manually create the YAML configuration for the job in your bundle project, ensuring all settings match the existing job. Then, run Databricks bundle deploy the bundle, which will update the existing job in your workspace.
• C. Run Databricks bundle generate job --existing-job-id to generate the YAML and download referenced files. Then, run Databricks bundle deploy to deploy the bundle, which will always update the existing job automatically.
• D. Export the job definition as JSON, convert it to YAML, and place it in your bundle. Then, run Databricks bundle deploy to update the existing job.

A data engineer wants to join a stream of advertisement impressions (when an ad was shown) with another stream of user clicks on advertisements to correlate when impressions led to monetizable clicks.
In the code below, Impressions is a streaming DataFrame with a watermark ("event_time", "10 minutes")

The data engineer notices the query slowing down significantly.
Which solution would improve the performance?

• A. Joining on event time constraint: clickTime >= impressionTime - interval 3 hours and removing watermarks
• B. Joining on event time constraint: clickTime + 3 hours < impressionTime - 2 hours
• C. Joining on event time constraint: clickTime == impressionTime using a leftOuter join
• D. Joining on event time constraint: clickTime >= impressionTime AND clickTime <= impressionTime interval 1 hour

A data engineer, User A, has promoted a new pipeline to production by using the REST API to programmatically create several jobs. A DevOps engineer, User B, has configured an external orchestration tool to trigger job runs through the REST API. Both users authorized the REST API calls using their personal access tokens.
Which statement describes the contents of the workspace audit logs concerning these events?

• A. Because the REST API was used for job creation and triggering runs, a Service Principal will be automatically used to identity these events.
• B. Because these events are managed separately, User A will have their identity associated with the job creation events and User B will have their identity associated with the job run events.
• C. Because User A created the jobs, their identity will be associated with both the job creation events and the job run events.
• D. Because User B last configured the jobs, their identity will be associated with both the job creation events and the job run events.
• E. Because the REST API was used for job creation and triggering runs, user identity will not be captured in the audit logs.

Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM.
Given a job with at least one wide transformation, which of the following cluster configurations will result in maximum performance?

• A. Total VMs: 1
  400 GB per Executor
  160 Cores / Executor
• B. Total VMs: 2
  200 GB per Executor
  80 Cores / Executor
• C. Total VMs: 4
  100 GB per Executor
  40 Cores/Executor
• D. Total VMs: 8
  50 GB per Executor
  20 Cores / Executor

A junior data engineer has been asked to develop a streaming data pipeline with a grouped aggregation using DataFrame df. The pipeline needs to calculate the average humidity and average temperature for each non-overlapping five-minute interval. Incremental state information should be maintained for 10 minutes for late-arriving data.
Streaming DataFrame df has the following schema:
"device_id INT, event_time TIMESTAMP, temp FLOAT, humidity FLOAT"
Code block:

Choose the response that correctly fills in the blank within the code block to complete this task.

• A. withWatermark("event_time", "10 minutes")
• B. awaitArrival("event_time", "10 minutes")
• C. delayWrite("event_time", "10 minutes")
• D. await("event_time + `10 minutes'")
• E. slidingWindow("event_time", "10 minutes")