Databricks Certified Associate Developer for Apache Spark Practice Exam

Correct: B. select and filter are transformations and are lazy — Spark builds a logical plan but does no computation until an action like count() forces execution of the whole chain.

Why not the others: transformations do not run eagerly (A); caching affects reuse, not whether lazy transformations execute (C); creating the SparkSession does not run any job (D). Lazy evaluation plus an action is the core execution model.

Correct: C — collect(). collect() is an action: it materializes the result and returns it to the driver, triggering the job. (Be cautious with it on large data, since it pulls everything to the driver.)

Why not the others: select (A), filter (B), and withColumn (D) are all lazy transformations that only extend the plan. Only collect() forces execution and returns data.

Correct: A — withColumn. withColumn(name, expression) returns a new DataFrame with the derived column added (DataFrames are immutable). df.price * 1.08 is a valid column expression.

Why not the others: there is no addColumn method (B); price alone is undefined and select needs a column object (C); DataFrames are immutable, so attribute assignment (D) does not add a column. withColumn is the idiom.

Correct: B — dropna(how="all"). With how="all", a row is dropped only if all of its columns are null, which keeps any row with at least one value — exactly the requirement.

Why not the others: how="any" (A) drops a row if any column is null, which is too aggressive; fillna (C) replaces nulls instead of dropping rows; distinct (D) removes duplicates, unrelated to nulls. how="all" matches.

Correct: B. Supplying an explicit StructType schema avoids an extra inference pass over the data, is faster, and guarantees consistent typing across runs — the production best practice.

Why not the others: relying on inference (A) costs an extra scan and can drift; stringifying everything (C) loses types and pushes work downstream; reading one file at a time (D) cripples parallelism. An explicit schema is correct.

Correct: B. createOrReplaceTempView("sales") registers the DataFrame under a name in the session catalog so Spark SQL can reference it as a table in spark.sql(...).

Why not the others: a Python variable is not automatically a SQL-visible table (A); exporting to CSV (C) does not create a queryable view; renaming the Python variable (D) has no effect on SQL name resolution. A temp view is required.

Correct: B — broadcast join. Broadcasting the small table to all executors lets each one join locally without shuffling the large table, which is the standard fix for a large-table/small-table join. Adaptive Query Execution can also choose this automatically.

Why not the others: bumping shuffle partitions (A) does not remove the shuffle; caching the large table (C) does not address the join-shuffle cost; converting to RDDs (D) discards Catalyst optimizations. Broadcast is the right tool.

Correct: A — a watermark. A watermark on the event-time column tells Spark how late data may arrive; events later than the watermark are dropped and the associated state can be cleaned up, bounding state in stateful streaming aggregations.

Why not the others: a once trigger (B) controls batch cadence, not late-data state; shuffle partitions (C) affect parallelism, not watermarking; disabling checkpointing (D) breaks fault tolerance. The watermark bounds late-data state.

Correct: A — Spark Connect. Spark Connect introduces a client-server architecture where a thin client sends unresolved logical plans to a remote Spark server for execution, enabling remote development from IDEs and lightweight clients.

Why not the others: copying the runtime locally (B) is not how remote execution works; local mode (C) does not use the remote cluster; emailing a JAR (D) is not a deployment mechanism. Spark Connect is the client-server feature being described.

Correct: B — the Pandas API on Spark. pyspark.pandas offers pandas-compatible syntax that executes on Spark, so familiar pandas code scales across the cluster rather than being bound to one node’s memory.

Why not the others: plain pandas on the driver (A) is single-node and will run out of memory on large data; Excel (C) and row-by-row Python loops (D) do not scale at all. The Pandas API on Spark is purpose-built for this.