Retrieve analyzed tokens from ElasticSearch documents

text elasticsearch tokenize
Clay Wardell picture Clay Wardell · Nov 15, 2012 · Viewed 22.6k times · Source

Trying to access the analyzed/tokenized text in my ElasticSearch documents.

I know you can use the Analyze API to analyze arbitrary text according your analysis modules. So I could copy and paste data from my documents into the Analyze API to see how it was tokenized.

This seems unnecessarily time consuming, though. Is there any way to instruct ElasticSearch to returned the tokenized text in search results? I've looked through the docs and haven't found anything.

Answer

Torsten Engelbrecht picture Torsten Engelbrecht · Jun 24, 2014

This question is a litte old, but maybe I think an additional answer is necessary.

With ElasticSearch 1.0.0 the Term Vector API was added which gives you direct access to the tokens ElasticSearch stores under the hood on per document basis. The API docs are not very clear on this (only mentioned in the example), but in order to use the API you have to first indicate in your mapping definition that you want to store term vectors with the term_vector property on each field.

Related questions

How do I find all files containing specific text on Linux?

I'm trying to find a way to scan my entire Linux system for all files containing a specific string of text. Just to clarify, I'm looking for text within the file, not in the file name. When I was looking …

Read More
Print string to text file

I'm using Python to open a text document: text_file = open("Output.txt", "w") text_file.write("Purchase Amount: " 'TotalAmount') text_file.close() I want to substitute the value of a string variable TotalAmount into the text document. Can someone …

Read More
How to read a text file into a list or an array with Python

I am trying to read the lines of a text file into a list or array in python. I just need to be able to individually access any item in the list or array after it is created. The text …

Read More