0%

Elasticsearch基准测试

Bash Shell只支持一维数组(不支持多维数组),初始化时不需要定义数组大小(与PHP类似)。与大部分编程语言类似,数组元素的下标由0开始。Shell数组用括号来表示,元素用空格符号分割开。

一、基础

二、使用

  1. 自己写代码做压测

  2. 使用Jmeter等工具做压测

  3. 使用官方工具rally(网上大部分的文章都是几年前的,最新的rally已经不支持比如configure命令了)

    • 安装pip3 install esrally

    • esrally -h

      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      21
      22
      23
      24
      25
      26
      27
      28
      29
      30
      31
      usage: esrally [-h] [--version]
      {race,list,info,create-track,generate,compare,download,install,start,stop}
      ...

      ____ ____
      / __ \____ _/ / /_ __
      / /_/ / __ `/ / / / / /
      / _, _/ /_/ / / / /_/ /
      /_/ |_|\__,_/_/_/\__, /
      /____/

      You Know, for Benchmarking Elasticsearch.

      optional arguments:
      -h, --help show this help message and exit
      --version show program's version number and exit

      subcommands:
      {race,list,info,create-track,generate,compare,download,install,start,stop}
      race Run a benchmark
      list List configuration options
      info Show info about a track
      create-track Create a Rally track from existing data
      generate Generate artifacts
      compare Compare two races
      download Downloads an artifact
      install Installs an Elasticsearch node locally
      start Starts an Elasticsearch node locally
      stop Stops an Elasticsearch node locally

      Find out more about Rally at https://esrally.readthedocs.io/en/2.3.0/
    • 查看版本esrally --version,博主最新的esrally 2.3.0

      • 配置java环境变量esrally configure,对于最新版本的rally已经不支持了
      • 指定es版本esrally --distribution-version=5.0.0,也不支持了
    • 根据官方仓库README来操作,或官方博客

      • esrally race --distribution-version=6.5.3 --track=geonames,报错

        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
            ____        ____
        / __ \____ _/ / /_ __
        / /_/ / __ `/ / / / / /
        / _, _/ /_/ / / / /_/ /
        /_/ |_|\__,_/_/_/\__, /
        /____/

        [WARNING] No Internet connection detected. Automatic download of track data sets etc. is disabled.
        [INFO] Race id is [52257024-97e9-49bb-94f4-dafd8a1dfe0f]
        [ERROR] Cannot race. Traceback (most recent call last):
        ...
        ...
        ...
        Please run:
        git -C /Users/liuyulong/.rally/benchmarks/tracks/default init

        Getting further help:
        *********************
        * Check the log files in /Users/liuyulong/.rally/logs for errors.
        * Read the documentation at https://esrally.readthedocs.io/en/2.3.0/.
        * Ask a question on the forum at https://discuss.elastic.co/tags/c/elastic-stack/elasticsearch/rally.
        * Raise an issue at https://github.com/elastic/rally/issues and include the log files in /Users/liuyulong/.rally/logs.

        --------------------------------
        [INFO] FAILURE (took 15 seconds)
    • 手动下载资源

    • sudo esrally list tracks

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
    ____        ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/

[WARNING] Could not update tracks. Continuing with your locally available state.
Available tracks:

Name Description Documents Compressed Size Uncompressed Size Default Challenge All Challenges
------------- ----------------------------------------------------------------------- ----------- ----------------- ------------------- ----------------------- ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
geonames POIs from Geonames 11,396,503 252.9 MB 3.3 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-fast-with-conflicts,significant-text
percolator Percolator benchmark based on AOL queries 2,000,000 121.1 kB 104.9 MB append-no-conflicts append-no-conflicts
http_logs HTTP server log data 247,249,096 1.2 GB 31.1 GB append-no-conflicts append-no-conflicts,runtime-fields,append-no-conflicts-index-only,append-sorted-no-conflicts,append-index-only-with-ingest-pipeline,update,append-no-conflicts-index-reindex-only
geoshape Shapes from PlanetOSM 60,523,283 13.4 GB 45.4 GB append-no-conflicts append-no-conflicts
metricbeat Metricbeat data 1,079,600 87.7 MB 1.2 GB append-no-conflicts append-no-conflicts
geopoint Point coordinates from PlanetOSM 60,844,404 482.1 MB 2.3 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-fast-with-conflicts
nyc_taxis Taxi rides in New York in 2015 165,346,692 4.5 GB 74.3 GB append-no-conflicts indexing-querying,append-no-conflicts,append-no-conflicts-index-only,append-sorted-no-conflicts-index-only,update,append-ml,date-histogram
geopointshape Point coordinates from PlanetOSM indexed as geoshapes 60,844,404 470.8 MB 2.6 GB append-no-conflicts append-no-conflicts,append-no-conflicts-index-only,append-fast-with-conflicts
so Indexing benchmark using up to questions and answers from StackOverflow 36,062,278 8.9 GB 33.1 GB append-no-conflicts append-no-conflicts
eql EQL benchmarks based on endgame index of SIEM demo cluster 60,782,211 4.5 GB 109.2 GB default default
nested StackOverflow Q&A stored as nested docs 11,203,029 663.3 MB 3.4 GB nested-search-challenge nested-search-challenge,index-only
noaa Global daily weather measurements from NOAA 33,659,481 949.4 MB 9.0 GB append-no-conflicts sql,append-no-conflicts,append-no-conflicts-index-only,top_metrics,aggs
pmc Full text benchmark with academic papers from PMC 574,199 5.5 GB 21.7 GB append-no-conflicts indexing-querying,append-no-conflicts,append-no-conflicts-index-only,append-sorted-no-conflicts,append-fast-with-conflicts

--------------------------------
[INFO] SUCCESS (took 90 seconds)
--------------------------------
  • sudo esrally --distribution-version=6.0.0 --track=geopoint
    • cd .rally/benchmarks
    • git clone https://github.com/elastic/rally-teams.git
    • mkdir teams
    • cp -r rally-teams ./teams/
    • mv rally-teams default
    • 重试sudo esrally --distribution-version=6.0.0 --track=geopoint

三、参考

  1. 参考一
  2. 参考二
  3. 参考三
  4. 参考四
  5. 参考五
  6. 参考六
  7. 参考七
  8. 参考八
  9. 参考九