adding two columns from a data frame in scala

Nirmal picture Nirmal · Oct 14, 2016 · Viewed 13.7k times · Source

I have two columns age and salary stored in DF. I just want to write a scala code to add these values column wise. i tried

val age_1 = df.select("age")
val salary_1=df.select("salary")
val add = age_1+salary_1

gives me error. please help

Answer

mrsrinivas picture mrsrinivas · Oct 14, 2016

In the following spark is an instance of SparkSession, so the import has to come after the instantiation of spark.

$-notation can be used here by importing spark implicits with

import spark.implicits._ 

then use $-notation

val add = df.select($"age" + $"salary")

final scala code:

import spark.implicits._
val add = df.select($"age" + $"salary")

Apache doc