Python elasticsearch.helpers.scan example

python python-2.7 elasticsearch elasticsearch-5
Ankam picture Ankam · Dec 8, 2017 · Viewed 17.2k times · Source

Can someone provide scan API example of python elasticsearch helpers client?

res = elasticsearch.helpers.scan(....)

How can i get all results from elasticsearch with res object?

Answer

Norrius picture Norrius · Dec 9, 2017

The documentation includes an example, although if I'm reading it right, helpers.scan by default sets search_type=scan, which was removed in ES 5.1. This causes the example code to fail with ES returning No search type for [scan]. We can amend this with preserve_order=True (I am however not sure about the performance implications here):

import elasticsearch
import elasticsearch.helpers
es = elasticsearch.Elasticsearch()
results = elasticsearch.helpers.scan(es,
    index="test_index",
    doc_type="my_document",
    preserve_order=True,
    query={"query": {"match_all": {}}},
)

for item in results:
    print(item['_id'], item['_source']['name'])

This helper returns an object which you can iterate to obtain the actual results from the query.

item is of form 

{'_index': <str>, '_type': <str>, '_id': <str>, '_score': <float or None>, '_source': {'key': val}, 'sort': [<int>]}

Related questions

Connection Timeout with Elasticsearch

from datetime import datetime from elasticsearch import Elasticsearch es = Elasticsearch() doc = { 'author': 'kimchy', 'text': 'Elasticsearch: cool. bonsai cool.', 'timestamp': datetime(2010, 10, 10, 10, 10, 10) } res = es.index(index="test-index", doc_type='tweet', id=1, body=doc) print(res['created']) This simples code is returning …

Read More
Elasticsearch : How to delete an Index using python

Forgive me if this is quite basic but I have Python 2.7 and Elasticsearch 2.1.1 and I am just trying to delete an index using es.delete(index='researchtest', doc_type='test') but this gives me return func(*args, params=params, **kwargs) …

Read More
User authentication in Elasticsearch query using python

I'm using elastisearch using Python. My code looks somewhat like this:- from elasticsearch import Elasticsearch if __name__ == '__main__': index="IndexPosition" es=Elasticsearch(['https://localhost:8080']) res = es.search(index='{0}'.format(index), doc_type="log",size=1000, …

Read More