"Fossies" - the Fresh Open Source Software Archive

Member "elasticsearch-6.8.5/docs/reference/api-conventions.asciidoc" (13 Nov 2019, 23218 Bytes) of package /linux/www/elasticsearch-6.8.5-src.tar.gz:


As a special service "Fossies" has tried to format the requested source page into HTML format (assuming AsciiDoc format). Alternatively you can here view or download the uninterpreted source code file. A member file download can also be achieved by clicking within a package contents listing on the according byte size field. See also the last Fossies "Diffs" side-by-side code changes report for "api-conventions.asciidoc": 7.4.2_vs_7.5.0.

Multiple Indices

Most APIs that refer to an index parameter support execution across multiple indices, using simple test1,test2,test3 notation (or _all for all indices). It also supports wildcards, for example: test* or test or te*t or *test, and the ability to "exclude" (-), for example: test*,-test3.

All multi indices APIs support the following url query string parameters:

ignore_unavailable

Controls whether to ignore if any specified indices are unavailable, including indices that don’t exist or closed indices. Either true or false can be specified.

allow_no_indices

Controls whether to fail if a wildcard indices expression results in no concrete indices. Either true or false can be specified. For example if the wildcard expression foo* is specified and no indices are available that start with foo, then depending on this setting the request will fail. This setting is also applicable when _all, *, or no index has been specified. This settings also applies for aliases, in case an alias points to a closed index.

expand_wildcards

Controls what kind of concrete indices that wildcard indices expressions can expand to. If open is specified then the wildcard expression is expanded to only open indices. If closed is specified then the wildcard expression is expanded only to closed indices. Also both values (open,closed) can be specified to expand to all indices.

If none is specified then wildcard expansion will be disabled. If all is specified, wildcard expressions will expand to all indices (this is equivalent to specifying open,closed).

The defaults settings for the above parameters depend on the API being used.

Note
Single index APIs such as the [docs] and the single-index alias APIs do not support multiple indices.

Date math support in index names

Date math index name resolution enables you to search a range of time-series indices, rather than searching all of your time-series indices and filtering the results or maintaining aliases. Limiting the number of indices that are searched reduces the load on the cluster and improves execution performance. For example, if you are searching for errors in your daily logs, you can use a date math name template to restrict the search to the past two days.

Almost all APIs that have an index parameter support date math in the index parameter value.

A date math index name takes the following form:

<static_name{date_math_expr{date_format|time_zone}}>

Where:

static_name

is the static text part of the name

date_math_expr

is a dynamic date math expression that computes the date dynamically

date_format

is the optional format in which the computed date should be rendered. Defaults to YYYY.MM.dd.

time_zone

is the optional time zone. Defaults to utc.

Note
Pay attention to the usage of small vs capital letters used in the date_format. For example: mm denotes minute of hour, while MM denotes month of year. Similarly hh denotes the hour in the 1-12 range in combination with AM/PM, while HH denotes the hour in the 0-23 24-hour range.

Date math expressions are resolved locale-independent. Consequently, it is not possible to use any other calendars than the Gregorian calendar.

You must enclose date math index name expressions within angle brackets, and all special characters should be URI encoded. For example:

# GET /<logstash-{now/d}>/_search
GET /%3Clogstash-%7Bnow%2Fd%7D%3E/_search
{
  "query" : {
    "match": {
      "test": "data"
    }
  }
}
Note
Percent encoding of date math characters

The special characters used for date rounding must be URI encoded as follows:

<

%3C

>

%3E

/

%2F

{

%7B

}

%7D

|

%7C

+

%2B

:

%3A

,

%2C

The following example shows different forms of date math index names and the final index names they resolve to given the current time is 22nd March 2024 noon utc.

Expression Resolves to

<logstash-{now/d}>

logstash-2024.03.22

<logstash-{now/M}>

logstash-2024.03.01

<logstash-{now/M{YYYY.MM}}>

logstash-2024.03

<logstash-{now/M-1M{YYYY.MM}}>

logstash-2024.02

<logstash-{now/d{YYYY.MM.dd|+12:00}}>

logstash-2024.03.23

To use the characters { and } in the static part of an index name template, escape them with a backslash \, for example:

  • <elastic\\{ON\\}-{now/M}> resolves to elastic{ON}-2024.03.01

The following example shows a search request that searches the Logstash indices for the past three days, assuming the indices use the default Logstash index name format, logstash-YYYY.MM.dd.

# GET /<logstash-{now/d-2d}>,<logstash-{now/d-1d}>,<logstash-{now/d}>/_search
GET /%3Clogstash-%7Bnow%2Fd-2d%7D%3E%2C%3Clogstash-%7Bnow%2Fd-1d%7D%3E%2C%3Clogstash-%7Bnow%2Fd%7D%3E/_search
{
  "query" : {
    "match": {
      "test": "data"
    }
  }
}

Common options

The following options can be applied to all of the REST APIs.

Pretty Results

When appending ?pretty=true to any request made, the JSON returned will be pretty formatted (use it for debugging only!). Another option is to set ?format=yaml which will cause the result to be returned in the (sometimes) more readable yaml format.

Human readable output

Statistics are returned in a format suitable for humans (e.g. "exists_time": "1h" or "size": "1kb") and for computers (e.g. "exists_time_in_millis": 3600000 or "size_in_bytes": 1024). The human readable values can be turned off by adding ?human=false to the query string. This makes sense when the stats results are being consumed by a monitoring tool, rather than intended for human consumption. The default for the human flag is false.

Date Math

Most parameters which accept a formatted date value — such as gt and lt in range queries, or from and to in daterange aggregations — understand date maths.

The expression starts with an anchor date, which can either be now, or a date string ending with ||. This anchor date can optionally be followed by one or more maths expressions:

  • +1h: Add one hour

  • -1d: Subtract one day

  • /d: Round down to the nearest day

The supported time units differ from those supported by time units for durations. The supported units are:

y

Years

M

Months

w

Weeks

d

Days

h

Hours

H

Hours

m

Minutes

s

Seconds

Assuming now is 2001-01-01 12:00:00, some examples are:

now+1h

now in milliseconds plus one hour. Resolves to: 2001-01-01 13:00:00

now-1h

now in milliseconds minus one hour. Resolves to: 2001-01-01 11:00:00

now-1h/d

now in milliseconds minus one hour, rounded down to UTC 00:00. Resolves to: 2001-01-01 00:00:00

2001.02.01\|\|+1M/d

2001-02-01 in milliseconds plus one month. Resolves to: 2001-03-01 00:00:00

Response Filtering

All REST APIs accept a filter_path parameter that can be used to reduce the response returned by Elasticsearch. This parameter takes a comma separated list of filters expressed with the dot notation:

GET /_search?q=elasticsearch&filter_path=took,hits.hits._id,hits.hits._score

Responds:

{
  "took" : 3,
  "hits" : {
    "hits" : [
      {
        "_id" : "0",
        "_score" : 1.6375021
      }
    ]
  }
}

It also supports the * wildcard character to match any field or part of a field’s name:

GET /_cluster/state?filter_path=metadata.indices.*.stat*

Responds:

{
  "metadata" : {
    "indices" : {
      "twitter": {"state": "open"}
    }
  }
}

And the ** wildcard can be used to include fields without knowing the exact path of the field. For example, we can return the Lucene version of every segment with this request:

GET /_cluster/state?filter_path=routing_table.indices.**.state

Responds:

{
  "routing_table": {
    "indices": {
      "twitter": {
        "shards": {
          "0": [{"state": "STARTED"}, {"state": "UNASSIGNED"}],
          "1": [{"state": "STARTED"}, {"state": "UNASSIGNED"}],
          "2": [{"state": "STARTED"}, {"state": "UNASSIGNED"}],
          "3": [{"state": "STARTED"}, {"state": "UNASSIGNED"}],
          "4": [{"state": "STARTED"}, {"state": "UNASSIGNED"}]
        }
      }
    }
  }
}

It is also possible to exclude one or more fields by prefixing the filter with the char -:

GET /_count?filter_path=-_shards

Responds:

{
  "count" : 5
}

And for more control, both inclusive and exclusive filters can be combined in the same expression. In this case, the exclusive filters will be applied first and the result will be filtered again using the inclusive filters:

GET /_cluster/state?filter_path=metadata.indices.*.state,-metadata.indices.logstash-*

Responds:

{
  "metadata" : {
    "indices" : {
      "index-1" : {"state" : "open"},
      "index-2" : {"state" : "open"},
      "index-3" : {"state" : "open"}
    }
  }
}

Note that Elasticsearch sometimes returns directly the raw value of a field, like the _source field. If you want to filter _source fields, you should consider combining the already existing _source parameter (see Get API for more details) with the filter_path parameter like this:

POST /library/book?refresh
{"title": "Book #1", "rating": 200.1}
POST /library/book?refresh
{"title": "Book #2", "rating": 1.7}
POST /library/book?refresh
{"title": "Book #3", "rating": 0.1}
GET /_search?filter_path=hits.hits._source&_source=title&sort=rating:desc
{
  "hits" : {
    "hits" : [ {
      "_source":{"title":"Book #1"}
    }, {
      "_source":{"title":"Book #2"}
    }, {
      "_source":{"title":"Book #3"}
    } ]
  }
}

Flat Settings

The flat_settings flag affects rendering of the lists of settings. When the flat_settings flag is true, settings are returned in a flat format:

GET twitter/_settings?flat_settings=true

Returns:

{
  "twitter" : {
    "settings": {
      "index.number_of_replicas": "1",
      "index.number_of_shards": "1",
      "index.creation_date": "1474389951325",
      "index.uuid": "n6gzFZTgS664GUfx0Xrpjw",
      "index.version.created": ...,
      "index.provided_name" : "twitter"
    }
  }
}

When the flat_settings flag is false, settings are returned in a more human readable structured format:

GET twitter/_settings?flat_settings=false

Returns:

{
  "twitter" : {
    "settings" : {
      "index" : {
        "number_of_replicas": "1",
        "number_of_shards": "1",
        "creation_date": "1474389951325",
        "uuid": "n6gzFZTgS664GUfx0Xrpjw",
        "version": {
          "created": ...
        },
        "provided_name" : "twitter"
      }
    }
  }
}

By default flat_settings is set to false.

Parameters

Rest parameters (when using HTTP, map to HTTP URL parameters) follow the convention of using underscore casing.

Boolean Values

All REST API parameters (both request parameters and JSON body) support providing boolean "false" as the value false and boolean "true" as the value true. All other values will raise an error.

Number Values

All REST APIs support providing numbered parameters as string on top of supporting the native JSON number types.

Time units

Whenever durations need to be specified, e.g. for a timeout parameter, the duration must specify the unit, like 2d for 2 days. The supported units are:

d

Days

h

Hours

m

Minutes

s

Seconds

ms

Milliseconds

micros

Microseconds

nanos

Nanoseconds

Byte size units

Whenever the byte size of data needs to be specified, e.g. when setting a buffer size parameter, the value must specify the unit, like 10kb for 10 kilobytes. Note that these units use powers of 1024, so 1kb means 1024 bytes. The supported units are:

b

Bytes

kb

Kilobytes

mb

Megabytes

gb

Gigabytes

tb

Terabytes

pb

Petabytes

Unit-less quantities

Unit-less quantities means that they don’t have a "unit" like "bytes" or "Hertz" or "meter" or "long tonne".

If one of these quantities is large we’ll print it out like 10m for 10,000,000 or 7k for 7,000. We’ll still print 87 when we mean 87 though. These are the supported multipliers:

k

Kilo

m

Mega

g

Giga

t

Tera

p

Peta

Distance Units

Wherever distances need to be specified, such as the distance parameter in the [query-dsl-geo-distance-query]), the default unit is meters if none is specified. Distances can be specified in other units, such as "1km" or "2mi" (2 miles).

The full list of units is listed below:

Mile

mi or miles

Yard

yd or yards

Feet

ft or feet

Inch

in or inch

Kilometer

km or kilometers

Meter

m or meters

Centimeter

cm or centimeters

Millimeter

mm or millimeters

Nautical mile

NM, nmi, or nauticalmiles

Fuzziness

Some queries and APIs support parameters to allow inexact fuzzy matching, using the fuzziness parameter.

When querying text or keyword fields, fuzziness is interpreted as a Levenshtein Edit Distance — the number of one character changes that need to be made to one string to make it the same as another string.

The fuzziness parameter can be specified as:

0, 1, 2

The maximum allowed Levenshtein Edit Distance (or number of edits)

AUTO

Generates an edit distance based on the length of the term. Low and high distance arguments may be optionally provided AUTO:[low],[high]. If not specified, the default values are 3 and 6, equivalent to AUTO:3,6 that make for lengths:

0..2

Must match exactly

3..5

One edit allowed

>5

Two edits allowed

AUTO should generally be the preferred value for fuzziness.

Enabling stack traces

By default when a request returns an error Elasticsearch doesn’t include the stack trace of the error. You can enable that behavior by setting the error_trace url parameter to true. For example, by default when you send an invalid size parameter to the _search API:

POST /twitter/_search?size=surprise_me

The response looks like:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "Failed to parse int parameter [size] with value [surprise_me]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "Failed to parse int parameter [size] with value [surprise_me]",
    "caused_by" : {
      "type" : "number_format_exception",
      "reason" : "For input string: \"surprise_me\""
    }
  },
  "status" : 400
}

But if you set error_trace=true:

POST /twitter/_search?size=surprise_me&error_trace=true

The response looks like:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Failed to parse int parameter [size] with value [surprise_me]",
        "stack_trace": "Failed to parse int parameter [size] with value [surprise_me]]; nested: IllegalArgumentException..."
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Failed to parse int parameter [size] with value [surprise_me]",
    "stack_trace": "java.lang.IllegalArgumentException: Failed to parse int parameter [size] with value [surprise_me]\n    at org.elasticsearch.rest.RestRequest.paramAsInt(RestRequest.java:175)...",
    "caused_by": {
      "type": "number_format_exception",
      "reason": "For input string: \"surprise_me\"",
      "stack_trace": "java.lang.NumberFormatException: For input string: \"surprise_me\"\n    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)..."
    }
  },
  "status": 400
}

Request body in query string

For libraries that don’t accept a request body for non-POST requests, you can pass the request body as the source query string parameter instead. When using this method, the source_content_type parameter should also be passed with a media type value that indicates the format of the source, such as application/json.

Content-Type Requirements

The type of the content sent in a request body must be specified using the Content-Type header. The value of this header must map to one of the supported formats that the API supports. Most APIs support JSON, YAML, CBOR, and SMILE. The bulk and multi-search APIs support NDJSON, JSON, and SMILE; other types will result in an error response.

Additionally, when using the source query string parameter, the content type must be specified using the source_content_type query string parameter.

URL-based access control

Many users use a proxy with URL-based access control to secure access to Elasticsearch indices. For multi-search, multi-get, and bulk requests, the user has the choice of specifying an index in the URL and on each individual request within the request body. This can make URL-based access control challenging.

To prevent the user from overriding the index which has been specified in the URL, add this setting to the elasticsearch.yml file:

rest.action.multi.allow_explicit_index: false

The default value is true, but when set to false, Elasticsearch will reject requests that have an explicit index specified in the request body.