Fastest way to perform bulk add/insert in Neo4j with Python?

python neo4j py2neo
wodow picture wodow · Sep 28, 2012 · Viewed 11.6k times · Source

I am finding Neo4j slow to add nodes and relationships/arcs/edges when using the REST API via py2neo for Python. I understand that this is due to each REST API call executing as a single self-contained transaction.

Specifically, adding a few hundred pairs of nodes with relationships between them takes a number of seconds, running on localhost.

What is the best approach to significantly improve performance whilst staying with Python?

Would using bulbflow and Gremlin be a way of constructing a bulk insert transaction?

Thanks!

Answer

Nigel Small picture Nigel Small · Sep 29, 2012

There are several ways to do a bulk create with py2neo, each making only a single call to the server.

  1. Use the create method to build a number of nodes and relationships in a single batch.
  2. Use a cypher CREATE statement.
  3. Use the new WriteBatch class (just released this week) to manually make a batch of nodes and relationships (this is really just a manual version of 1).

If you have some code, I'm happy to look at it and make suggestions on performance tweaks. There are also quite a few tests you may be able to get inspiration from.

Cheers, Nige

Related questions

AttributeError: 'Graph' object has no attribute 'cypher' in migration of data from Postgress to Neo4j(Graph Database)

I am working on migration of data from postgres to Graph Database manually. I have wrote script below: import psycopg2 from py2neo import authenticate, Graph authenticate("localhost:7474", "neo4j", "password") n4j_graph = Graph("http://localhost:7474/db/data/") try: …

Read More
Error connecting python to neo4j using py2neo

I wrote the following python code to neo4j using py2neo from py2neo import Graph from py2neo import neo4j,Node,Relationship sgraph = Graph() alice = Node("person",name="alice") bob = Node("person",name="bob") alice_knows_bob = …

Read More
py2neo, neo4j: How to create relation between two existing node

I am following this tutorial to access neo4j db using python. According to this tutorial I have created 2 relations among 4 nodes. The code is given below alice, bob, rel = graph_db.create( {"name": "Alice"}, {"name": "Bob"}, (0, "KNOWS", 1)) dev, carol, …

Read More