Top "Sparkr" questions

SparkR is an R package that provides a light-weight frontend to use Apache Spark from R.

Summing multiple columns in Spark

How can I sum multiple columns in Spark? For example, in SparkR the following code works to get the sum …

apache-spark pyspark sparkr
Installing of SparkR

I have the last version of R - 3.2.1. Now I want to install SparkR on R. After I execute: > …

r apache-spark sparkr
How do I read a Parquet in R and convert it to an R DataFrame?

I'd like to process Apache Parquet files (in my case, generated in Spark) in the R programming language. Is an …

r apache-spark parquet sparkr
SparkR vs sparklyr

Does someone have an overview with respect to advantages/disadvantages of SparkR vs sparklyr? Google does not yield any satisfactory …

r apache-spark sparkr sparklyr
How to check for intersection of two DataFrame columns in Spark

Using either pyspark or sparkr (preferably both), how can I get the intersection of two DataFrame columns? For example, in …

apache-spark pyspark sparkr
Difference between createOrReplaceTempView and registerTempTable

I am new to spark and was trying out a few commands in sparkSql using python when I came across …

apache-spark pyspark apache-spark-sql pyspark-sql sparkr
Duplicate columns in Spark Dataframe

I have a 10GB csv file in hadoop cluster with duplicate columns. I try to analyse it in SparkR so …

r csv hadoop apache-spark sparkr
Convert date to end of month in Spark

I have a Spark DataFrame as shown below: #Create DataFrame df <- data.frame(name = c("Thomas", "William", "Bill", "…

pyspark spark-dataframe sparkr
Getting last value of group in Spark

I have a SparkR DataFrame as shown below: #Create R data.frame custId <- c(rep(1001, 5), rep(1002, 3), 1003) date <…

apache-spark pyspark spark-dataframe sparkr
sparkR: Error while instantiating 'org.apache.spark.sql.hive.HiveSessionState':

I am struggling solving this problem when I try to use sparkR. sparkR.session(master = "local[*]", sparkConfig = list(spark.driver.…

java scala apache-spark sparkr