How to import JSON file to ElasticSearch 6.x?

http curl elasticsearch elasticsearch-6
Victor Ferreira picture Victor Ferreira · Dec 22, 2017 · Viewed 9.9k times · Source

I got a brand new empty installation of Elastic Search 6.1.1 running on port 9200 of my CentOS7 host. I have this "es.json" file with data sample ready to be inserted in the ES.

I also have a log file in which each line is a json chunk. I could use both to populate my ES database.

How do I insert this and other data to ES?

I didnt find good explanations on the documentation and on the internet. for some reason the docs does not make it very clear

Answer

Bless picture Bless · Dec 22, 2017

Use the --data-binary flag in curl to bulk import from a JSON file.

curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/{index}/{type}/_bulk?pretty' --data-binary @es.json

Data can be posted to one of the endpoints - /_bulk, /{index}/_bulk or {index}/{type}/_bulk. When {index} or {index}/{type} is provided, they will be used by default on bulk items that don’t provide them explicitly.

Content-Type: application/x-ndjson stands for Newline delimited JSON.

Before importing the JSON file, you might want to define mappings yourself or let Elasticsearch generate mappings dynamically during import. If you don't want Elasticsearch to generate mappings dynamically during import, refer to this doc to define mappings yourself.

References:

Related questions

Representing a Kibana query in a REST, curl form

I have a Kibana server in a classic ELK configuration, querying an Elasticsearch instance. I use the Kibana console to execute sophisticated queries on elasticsearch. I would like to use some of these queries in the command linem using cURL …

Read More
PHP + curl, HTTP POST sample code?

Can anyone show me how to do a php curl with an HTTP POST? I want to send data like this: username=user1, password=passuser1, gender=1 To www.domain.com I expect the curl to return a response like result=…

Read More
How to do a PUT request with curl?

How do I test a RESTful PUT (or DELETE) method using curl?

Read More