Top "Large-data" questions

Large data is data that is difficult to process and manage because its size is usually beyond the limits of the software being used to perform the analysis.

"Large data" work flows using pandas

I have tried to puzzle out an answer to this question for many months while learning pandas. I use SAS …

python mongodb pandas hdf5 large-data
What causes a Python segmentation fault?

I am implementing Kosaraju's Strong Connected Component(SCC) graph search algorithm in Python. The program runs great on small data …

python segmentation-fault large-data
Shared memory in multiprocessing

I have three large lists. First contains bitarrays (module bitarray 0.8.0) and the other two contain arrays of integers. l1=[bitarray 1, …

python multiprocessing shared-memory large-data
Add lines to a file

I'm new using R. I'm trying to add new lines to a file with my existing data in R. The …

r file large-data
How to store extremely large numbers?

For example I have a factorial program that needs to save really huge integers that can be 50+ digits long. The …

c++ data-structures int large-data
Is there any JSON viewer to open large json files (windows)?

I have very large JSON file which is of several GB. I am looking for any efficient JSON viewer. In …

json viewer json-view large-data
Read large data from csv file in php

I am reading csv & checking with mysql that records are present in my table or not in php. csv …

php csv large-data
How much data can R handle?

By "handle" I mean manipulate multi-columnar rows of data. How does R stack up against tools like Excel, SPSS, SAS, …

r large-data
Writing large Pandas Dataframes to CSV file in chunks

How do I write out a large data file to a CSV file in chunks? I have a set of …

python pandas dataframe export-to-csv large-data
How to read only lines that fulfil a condition from a csv into R?

I am trying to read a large csv file into R. I only want to read and work with some …

r large-data read.csv