I'm new in Scala programming and this is my question: How to count the number of string for each row? My Dataframe is composed of a single column of Array[String] type.
friendsDF: org.apache.spark.sql.DataFrame = [friends: array<string>]
You can use the size
function:
val df = Seq((Array("a","b","c"), 2), (Array("a"), 4)).toDF("friends", "id")
// df: org.apache.spark.sql.DataFrame = [friends: array<string>, id: int]
df.select(size($"friends").as("no_of_friends")).show
+-------------+
|no_of_friends|
+-------------+
| 3|
| 1|
+-------------+
To add as a new column:
df.withColumn("no_of_friends", size($"friends")).show
+---------+---+-------------+
| friends| id|no_of_friends|
+---------+---+-------------+
|[a, b, c]| 2| 3|
| [a]| 4| 1|
+---------+---+-------------+