Comprehensive and Detailed Explanation From Exact Extract:
* Exact extract: "select() expects column names or Column expressions."
* Exact extract: "When using strings directly, Spark SQL interprets them as literal column names."
* Exact extract: "Python string operations, such as "colname"*3, return repeated strings, not column expressions." The expression 3*"heartrate" is Python string multiplication, which evaluates to "heartrateheartrateheartrate".
The select() method interprets this as a literal column name. Since there is no column with that name in the DataFrame schema, Spark raises AnalysisException saying it cannot resolve that column. To correctly multiply a column by a scalar, one must use the column expression form:
from pyspark.sql.functions import col
df.select((col("heartrate") * 3).alias("heartrate_x3"))
This ensures Spark evaluates the arithmetic operation on the column instead of misinterpreting the string.
References: PySpark DataFrame select; PySpark Column expressions with col().