Top "Apache-spark-1.5" questions

Use for questions specific to Apache Spark 1.5. For general questions related to Apache Spark use the tag [apache-spark].

"INSERT INTO ..." with SparkSQL HiveContext

I'm trying to run an insert statement with my HiveContext, like this: hiveContext.sql('insert into my_table (id, score) …

apache-spark apache-spark-sql pyspark apache-spark-1.5 hivecontext
Convert null values to empty array in Spark DataFrame

I have a Spark data frame where one column is an array of integers. The column is nullable because it …

apache-spark dataframe apache-spark-sql apache-spark-1.5
How to limit decimal values to 2 digits before applying agg function?

I am following this solution from one of the stack overflow post, my only requirement here is how can I …

scala apache-spark apache-spark-sql apache-spark-1.5
Spark job execution time

This might be a very simple question. But is there any simple way to measure the execution time of a …

apache-spark apache-spark-mllib apache-spark-1.5
Save Spark Dataframe into Elasticsearch - Can’t handle type exception

I have designed a simple job to read data from MySQL and save it in Elasticsearch with Spark. Here is …

elasticsearch apache-spark elasticsearch-hadoop apache-spark-1.5