How to add header and column to dataframe spark?

user3637823 picture user3637823 · Mar 31, 2017 · Viewed 13.5k times · Source

I have got a dataframe, on which I want to add a header and a first column manually. Here is the dataframe :

import org.apache.spark.sql.SparkSession 

val spark = SparkSession.builder.master("local").appName("my-spark-app").getOrCreate()
val df = spark.read.option("header",true).option("inferSchema",true).csv("C:\\gg.csv").cache()

the content of the dataframe

12,13,14
11,10,5
3,2,45

The expected output is

define,col1,col2,col3
c1,12,13,14
c2,11,10,5
c3,3,2,45

Answer

Adonis picture Adonis · Mar 31, 2017

What you want to do is:

df.withColumn("columnName", column) //here "columnName" should be "define" for you

Now you just need to create the said column (this might help)