I want to delete documents in my elasticsearch index which are older than 30 days.
Any ideas?
EDIT:
I want this to happen automatically - no document in my index shoudl be older than 30 days. So, in my opinion there are 2 options: either using curator or DELETE requests.
I have tried both, but i failed. Somehow i have to create a filter which filters all documents older than 30 days and deletes them, when i am using DELETE http statement.
I tried with curator, but curator (as far as i understood this) deletes only whole indices. When attempting to delete indices older than 30 days with curator, my timestamp causing errors.My moment.js
pattern looks like this"MMMM Do YYYY, HH:mm:ss.SSS"
.
EDIT 2: I added the following to my logstash configuration:
elasticsearch
{
hosts => ["http://localhost:9200"]
index => "logstash-%{type}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
So logstash creates for every type and every day a particular index. Now i can use curator to delete the indices older than a specific date.
Problem solved imho.
You can use DELETE
query for that:
https://www.elastic.co/guide/en/elasticsearch/reference/1.6/docs-delete-by-query.html
in example the query will delete everything older than: 2016-02-29
DELETE index_name/_query
{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "*"
}
},
"filter": {
"range": {
"@timestamp": {
"lte": "2016-02-29"
}
}
}
}
}
}
Update >6.4
According to the official documentation, this function has been deprecated and replaced by _delete_by_query
POST index_name/_delete_by_query
{
"query": {
"match": {
"message": "some message"
}
}
}
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html