Explanation
Correct code block:
transactionsDf.filter(col("productId") >= 2).limit(2)
The filter and where operators in gap 1 are just aliases of one another, so you cannot use them to pick the right answer.
The column definition in gap 2 is more helpful. The DataFrame.filter() method takes an argument of type Column or str. From all possible answers, only the one including col("productId") >= 2 fits this profile, since it returns a Column type.
The answer option using "productId" > 2 is invalid, since Spark does not understand that "productId" refers to column productId. The answer option using transactionsDf[productId] >= 2 is wrong because you cannot refer to a column using square bracket notation in Spark (if you are coming from Python using Pandas, this is something to watch out for). In all other options, productId is being referred to as a Python variable, so they are relatively easy to eliminate.
Also note that the question asks for the value in column productId being at least 2. This translates to a
"greater or equal" sign (>= 2), but not a "greater" sign (> 2).
Another thing worth noting is that there is no DataFrame.max() method. If you picked any option including this, you may be confusing it with the pyspark.sql.functions.max method. The correct method to limit the amount of rows is the DataFrame.limit() method.
More info:
- pyspark.sql.DataFrame.filter - PySpark 3.1.2 documentation
- pyspark.sql.DataFrame.limit - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3