Delete multiple indices in one Elasticsearch HTTP request (cURL)

oHo picture oHo · Jul 4, 2017 · Viewed 24.5k times · Source

I was using this curl command line to clean my indices:

curl -XDELETE http://example.com/my_index-*

But, now, I want to delete my_index-.*[.][0-3][0-9]:

  • to delete only my_index-YYYY.MM.dd
  • to keep my_index-YYYY.MM.dd-*

The relevant Elasticsearch documentation I have found:

  • Delete index API does say nothing on regex.

  • Multiple indices says:

    It also support wildcards, for example: test* or *test or te*t or *test*, and the ability to "add" (+) and "remove" (-), for example: +test*,-test3.

  • Date math support in index names says:

    Almost all APIs that have an index parameter, support date math in the index parameter value.
    [...]
    date_format is the optional format in which the computed date should be rendered. Defaults to YYYY.MM.dd.


My Questions:

  • Is it possible to send a DELETE request method to Elasticsearch HTTP server to delete indices only formatted my_index-YYYY.MM.dd?
  • Or the inverse, to delete all my_index-* but keeping my_index-*-*?

For example, regex can sometimes be provided within the POST data:

curl -XPOST http://example.com/my_index-2017.07.14/_search?pretty' -H 'Content-Type: application/json' -d'
{
    "suggest": {
        "song-suggest" : {
            "regex" : "n[ever|i]r",
            "completion" : {
                "field" : "suggest"
            }
        }
    }
}'

Answer

oHo picture oHo · Jul 7, 2017

Short answer

Delete all indices my_index-* except indices my_index-*-*

curl -X DELETE http://es.example.com/my_index-*,-my_index-*-*

No regex

Elasticsearch 5.x does not accept regex or filename patterns ?[a-z] to select multiple indices.

However, the multiple indices documentation allows + and - to include and exclude indices.

Script to prevent accidental deletion of indices my_index-*-*:

#!/bin/bash -xe
pattern="${1:-*}"
curl -X DELETE https://es.example.com/my_index-"$pattern",-my_index-*-*?pretty

Explanation

  • The parameter index can contain a comma separated list of index patterns, for example my_index_1,my_index_2,my_index_3.
  • Index pattern is based on wildcards, for example my_index*.
  • To include and exclude indices, use + and - as index prefix, for example my_index_*,-my_index_2017*,+my_index_2017-01*,-my_index_2017-01-31.
  • Do not need to use + on first index

Described example

This DELETE request deletes all indices my_index_* before my_index_2017-01-31

index_list='my_index_*,-my_index_2017*,+my_index_2017-01*,-my_index_2017-01-31'
curl -X DELETE http://es.example.com/"$index_list"
  • Delete all my_index_*
  • Except my_index_2017*
  • Delete my_index_2017-01*
  • Except my_index_2017-01-31