[フレックス検索]フレックス検索学習

設定

JDK

をインストールする必要があります.
インストール

フレックスサーチ

Download DEB file from https://www.elastic.co/downloads/elasticsearch

dpkg -i elasticsearch-5.1.1.deb

install at :/usr/share/elasticsearh

config file at:/etc/elasticsearch

init script at:/etc/init.d/elasticsearch

sudo systemctl enable elasticsearch

ESにおけるDB概念

index->リレーショナル・データベース

type -> table

document -> row

field -> column

mapping -> schema

ES上のDML

GET -> SELECT

curl -XGET localhost:9200/classes/class/1
// select * from class where id = 1

POST -> INSERT

curl -XPOST localhost:9200/classes/class/1 -d '{xxx}'
// curl -XPOST localhost:9200/classes/class/1 -d @xxx.json
// insert into class values()

PUT->UPDATE(PUT,

)
作成できます.)

curl -XPUT localhost:9220/classes/class/1 -d '{xxx}'
// update class set xxx where id = 1

DELETE -> DELETE

curl -XDELETE localhost:9220/classes/class/1 -d '{xxx}'
// delete from class where id = 1

RESTコマンドの末尾に?prettyを入れるときれいに撮れた結果が回ってきます

curl -XGET localhost:9200/classes?pretty

データCRUD

UPDATE

// docment의 property 추가
http://localhost:9200/classes/class/1/_update
{"doc": {"unit": 123}}

http://localhost:9200/classes/class/1/_update
{"script": "ctx._source.unit += 10"}

BULK POST

http://localhost:9200/_bulk -d @파일이름
// bulk 파일에는 메타데이터를 입력해야함.

マッピング(PUT)

の作成

http://localhost:9200/인덱스/_mapping -d @파일이름
{
    "properties": {
      "title": {"type": "text"},
      "date": {"type": "date", "format": "yyyy-MM-dd"}
    }
}

Search

http://localhost:9200/인덱스/_doc/_search?q=date:2019?pretty

// GET 요청에 request body에 쿼리 요청을 넣어서 날림.
http://localhost:9200/인덱스/_doc/_search -d
{
    "query": {
        "term": {"points": 30}
    }
}

Aggregation

{
    "aggs": {
        "{aggregation
        name}": {
            "{aggregation_type}": {
                // aggregation body
            }
            [, "meta": { 
                // meta body
            }]
            [, "aggs": { 
                // sub aggregation
            }]
        },
        "{aggregation_name2}": {
            "{aggregation_type}": {
                //aggregation body
            }
        },
    }
}

- ES 안의 document들을 조합을 도출
- metric은 산수 조합 (평균, 최소값 등)

// GET 요청
http://localhost:9200/_search -d
// 평균값
{
    "size": 0,
    "aggs": {
        "avg_score": {
            "avg": {
                "field": "points"
            }
        }
    }
}
// 최대값
{
    "size": 0,
    "aggs": {
        "max_score": {
            "max": {
                "field": "points"
            }
        }
    }
}
// 최소값
{
    "size": 0,
    "aggs": {
        "min_score": {
            "min": {
                "field": "points"
            }
        }
    }
}
// 덧샘
{
    "size": 0,
    "aggs": {
        "sum_score": {
            "sum": {
                "field": "points"
            }
        }
    }
}
// 한꺼번에 도출
{
    "size": 0,
    "aggs": {
        "stats_score": {
            "stats": {
                "field": "points"
            }
        }
    }
}

Bucket Aggregation
動作は

SQLのgroupbyに似ています

// mapping에서 fielddata로 선언해야 group으로 묶을 수 있음.
{
    "properties": {
      "team": {"type": "string", "fielddata": true},
      "points": {"type": "long"}
    }
}

// GET요청 localhost:9200/인덱스/_search
{
    "size":0,
    "aggs": {
        "team_stats": {
                "terms": {
                    "field": "team"
                },
                "aggs": {
                    "stats_bucket": {
                        "stats": {
                            "field": "points"
                        }
                    }
                }
        }
    }
}

Logstash

Rolling

https://stackoverflow.com/questions/31016101/elasticsearch-monthly-rolling-indices

ディレクトリ設定、プロファイル設定

./bin logstash --path.settings /data/logstash-7.8.1/config/gw12/ -f /data/logstash-7.8.1/config/gw12/logstash-gw12.conf

Kibana

取付

1. Downalod DEB file from https://www.elastic.co/downloads/kibana
2. sudo dpkg -i kibana-5.0.2-amd64.deb

運転

sudo /usr/share/kibana/bin/kibana
// kibana 기본 port = 5601

参考資料

Esインフラストラクチャコース

Esクラスタdoker-合成ファイルの作成