Skip to main content

Module 8 Test: 20 MCQs

This is your progress tracker : 20 questions, 5 Easy, 10 Moderate, 5 Hard . First solve the questions yourself (for code questions, think of the output before running it in Colab), then check against the Answer Key.

ScoreMeaning
18 to 20Excellent! Module 8 is your strong foundation.
12 to 17Good. Re-read the chapters where you made mistakes.
Below 12Don't worry. Re-read Chapters 41 to 45, practise with the projects, then take the test again.
EASY​
Q1. Which of the 5 Vs of Big Data tells how fast the data is arriving?​

A) Volume B) Variety

C) Veracity

D) Velocity

Q2. In HDFS, how many copies (replicas) of each block are kept by default?​

A) 3

B) 1

C) 2

D) 5

Q3. Which is the correct order of the three steps of MapReduce?​

A) Reduce, Map, Shuffle

B) Shuffle, Map, Reduce

C) Map, Shuffle, Reduce

D) Map, Reduce, Shuffle

Q4. Which of the following is a Spark action?​

A) filter()

B) count()

C) select()

D) withColumn()

Q5. In the cloud, what does pay-as-you-go mean?​

A) Paying the whole year's bill in advance

B) The cloud is always free

C) Only big companies can use it

D) You pay only for what you use

​
MODERATE​

Q6. 2 TB of data, each disk reads at 200 MB/second, and the data is spread evenly over 20 disks (1 TB = 1024 x 1024 MB). Roughly how long will it take to read all the data?

A) About 52 seconds

B) About 524 seconds

C) About 5243 seconds

D) About 10486 seconds

Q7. A 1000 MB file, block size 128 MB, replication 3. How many blocks will be made and how many block copies will be stored in total?

A) 8 blocks, 24 copies B) 7 blocks, 21 copies C) 8 blocks, 8 copies D) 8 blocks, 3 copies

Q8. What will be the output of this code?​
from collections import defaultdict
g = defaultdict(list)
for line in ["a b a", "b a"]:
for w in line.split():
g[w].append(1)
print({k: sum(v) for k, v in g.items()})

A) {"a": 2, "b": 3} B) {"a": 5, "b": 5} C) {"a": 1, "b": 1} D) {"a": 3, "b": 2}

Q9. If the Spark explain() plan shows Exchange, what does this mean?

A) Data is being kept in the cache B) The file is being written as Parquet C) Data is being sent between partitions (shuffle)

D) The schema is being changed

Q10. In this Spark code, up to **

print

, the actual calculation (processing of data) was done on how many rows?**

df2 = df.filter(F.col("qty") > 2)
df3 = df2.withColumn("x", F.col("qty") * 2)
print("ready")

A) On all rows, because the filter ran

B) On none, because no action ran

C) Only on the rows where qty > 2 is true

D) Only on the first 20 rows

Q11. The partition rule is key % 4 . If the keys are 10 to 19 (10 keys), how many keys go to partition 2? A) 3 B) 2 C) 4 D) 1

Q12. A table has 1 million rows and 20 columns. The query needs only 2 columns. Compared with a row-store, roughly how many fewer values does a column-store (Parquet) have to read?

A) 2 times fewer B) 20 times fewer C) 10 times fewer (2M compared with 20M) D) No difference

Q13. A cluster has an imaginary price of $0.50 per node per hour. 3 nodes run 10 hours a day for 20 days. What is the total compute cost?

A) $30 B) $150 C) $3,000 D) $300

Q14. A user's IAM policy allows only s3:GetObject (on the objects of one bucket). What can that user do?

A) Can both read and upload files

B) Can read files in that bucket, but cannot upload new files or delete

C) Can only delete

D) Can do nothing

Q15. The sales table has 5000 rows. In the regions table, Quetta is missing out of the 5 cities (only 4 cities). After sales.join(regions, "city", "left") , how many rows will there be, and what will be the region of the Quetta rows? (Quetta has 975 rows.)

A) 4025 rows, the Quetta rows disappear

B) 5001 rows

C) 5000 rows, Quetta's region is null

D) An error

​
HARD​

Q16. A join is very slow because one customer_id has 90% of the records and the other partitions sit idle. Which is the most suitable solution?

A) Salting, or handle the skewed key separately (or broadcast if one table is small)

B) Increase the RAM of all executors in the cluster

C) Call cache() on the result

D) Use coalesce(1) to put everything in one partition

Q17. The times (in seconds) of the streaming events are below, and there is a 5-second tumbling window. Which window has the most events and how many?

events = [1, 3, 4, 7, 8, 11, 12, 13, 14, 14] # (second)
window = 5

A) [0s, 5s): 3 events

B) [5s, 10s): 5 events

C) [10s, 15s): 4 events

D) [10s, 15s): 5 events

Q18. In a data lake, why is it wrong to overwrite/modify the Raw zone?​

A) Because the files in the Raw zone must be Parquet

B) Because the original data is lost and you cannot reprocess if there is a cleaning mistake

C) Because the Raw zone holds only aggregates

D) Because overwriting reduces the bill

Q19. A Spark job runs only 2 hours every night, but the cluster stays on 24x7 and the bill is very high. What is the best change?

A) Add more nodes to the cluster

B) Convert the data to CSV

C) Transient/auto-terminate cluster or serverless: runs only while the work is done

D) Make the bucket public

Q20. What will be the result (k, v) of this Spark code? ( row_number counts within each k in descending order of v.)

data = [("A", 10), ("A", 30), ("B", 5), ("B", 50), ("B", 20)]
df = spark.createDataFrame(data, ["k", "v"])
w = Window.partitionBy("k").orderBy(F.desc("v"))
(df.withColumn("r", F.row_number().over(w))
.filter("r = 2").select("k", "v").show())

A) ("A", 10) and ("B", 20) B) ("A", 30) and ("B", 50) C) ("A", 10) and ("B", 5) D) ("A", 30) and ("B", 20)

Module 8 Answer Key (Answers and explanations)​

#LevelAnswerExplanation
1EasyD) VelocityVelocity is the speed of data (events arriving every second). Volume
is the amount, Variety is the kinds, Veracity is about trust.
2EasyA) 3The default replication factor is 3. That is why data survives even
when one or two machines fail.
3EasyC) Map, Shuffle, ReduceFirst Map (make key, value), then Shuffle (same keys in one place),
and finally Reduce (sum/count).
4EasyB)count()count()is an action: it runs the work immediately. filter, select,
withColumn are transformations, which only build a plan.
5EasyD) You pay only for what you useIn pay-as-you-go you pay only for what you actually used (time,
storage, data).
6ModerateB) About 524 secondsTotal MB = 2 x 1024 x 1024 = 2,097,152. Speed = 200 x 20 = 4000
MB/s. Time = 2,097,152 / 4000 = 524.3 seconds (about 8.7 minutes).
7ModerateA) 8 blocks, 24 copies1000 / 128 = 7.8, i.e. 8 blocks (the last one is smaller). Each block has
3 copies: 8 x 3 = 24.
8ModerateD) {"a": 3, "b": 2}"a b a" and "b a" contain a three times (2 + 1) and b twice (1 + 1).
Shuffle gathers the same key, Reduce sums them.
9ModerateC) Data is being sent between
partitions (shuffle)
Exchange indicates a shuffle (wide transformations such as groupBy,
join, orderBy). It is expensive, so the aim in performance work is to
reduce shuffles.
10ModerateB) On none, because no action ranfilter and withColumn are transformations (lazy). Until an action like
count/show/write runs, Spark only builds a plan.
11ModerateA) 3The keys with remainder 2: 10, 14, 18, i.e. 3. The rest: partition 0: 12,
16 (2); partition 1: 13, 17 (2); partition 3: 11, 15, 19 (3).
12ModerateC) 10 times fewer (2M compared
with 20M)
In a row-store every row has to be read in full: 20 x 1M = 20M values.
In a column-store only 2 columns: 2M. That is, 10 times fewer.
13ModerateD) $3000.50 x 3 x 10 x 20 = 300 dollars.
14ModerateB) Can read files in that bucket, but
cannot upload new files or delete
GetObject is permission only to read objects. PutObject (upload) or
DeleteObject are separate permissions that were not given here
(least privilege).
15ModerateC) 5000 rows, Quetta's region is
null
A left join keeps all rows of the left table, so 5000. Where the key is
not found on the right, the right-hand columns are null. An inner join
would leave 4025 rows.
16HardA) Salting, or handle the skewed
key separately (or broadcast if one
table is small)
The problem is data skew: the load is on one partition. More RAM or
cache does not spread the load; coalesce(1) makes it worse. Salting
splits the key up.
17HardD) [10s, 15s): 5 eventsWindows: [0,5): 1, 3, 4 = 3 events; [5,10): 7, 8 = 2; [10,15): 11, 12, 13,
14, 14 = 5. The most is 5, in [10s, 15s).
18HardB) Because the original data is lost
and you cannot reprocess if there is
a cleaning mistake
Raw should stay exactly as it arrived. If a mistake is found in the
cleaning or calculation, Clean and Curated can be rebuilt from Raw.
Parquet/aggregates belong to Clean/Curated.
19HardC) Transient/auto-terminate cluster
or serverless: runs only while the
work is done
The cost depends on how long the cluster runs. Creating/stopping
the cluster only when the work happens (or using serverless) lowers
the bill. Adding nodes raises the bill; a public bucket is dangerous.
20HardA) ("A", 10) and ("B", 20)In each k, v is in descending order: A: 30, 10; B: 50, 20, 5.
r = 2

is
the second row: 10 for A and 20 for B.|