Top "Parquet" questions

How to save a partitioned parquet file in Spark 2.1?

I am trying to test how to write data in HDFS 2.7 using Spark 2.1. My data is a simple sequence of …

scala apache-spark apache-spark-sql parquet

How to append data to an existing parquet file

I'm using the following code to create ParquetWriter and to write records to it. ParquetWriter<GenericRecord> parquetWriter = new …

java hadoop parquet

Python: save pandas data frame to parquet file

Is it possible to save a pandas data frame directly to a parquet file? If not, what would be the …

python-3.x hdfs parquet

What are the differences between feather and parquet?

Both are columnar (disk-)storage formats for use in data analysis systems. Both are integrated within Apache Arrow (pyarrow package …

python pandas parquet feather pyarrow

how to merge multiple parquet files to single parquet file using linux or hdfs command?

I have multiple small parquet files generated as output of hive ql job, i would like to merge the output …

hdfs parquet

Spark SQL - difference between gzip vs snappy vs lzo compression formats

I am trying to use Spark SQL to write parquet file. By default Spark SQL supports gzip, but it also …

apache-spark gzip parquet snappy lzo

Methods for writing Parquet files using Python?

I'm having trouble finding a library that allows Parquet files to be written using Python. Bonus points if I can …

python apache-spark apache-spark-sql parquet snappy

how to read a parquet file, in a standalone java code?

the parquet docs from cloudera shows examples of integration with pig/hive/impala. but in many cases I want to …

java parquet

Updating values in apache parquet file

I have a quite hefty parquet file where I need to change values for one of the column. One way …

apache-spark parquet

How to split parquet files into many partitions in Spark?

So I have just 1 parquet file I'm reading with Spark (using the SQL stuff) and I'd like it to be …

scala apache-spark parquet