Elasticsearch初探(一)

时间:2022-01-16 17:39:21

一、安装(Linux篇)

现在的开源软件越来越成熟,都打好包了,下载后直接解压就可以测试了。

 curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz
tar -zxvf elasticsearch-6.3..tar.gz
cd elasticsearch-6.3./bin/
./elasticsearch

注意:elasticsearch不允许使用root权限启动,所以使用root下载解压的同学,要chown改变一下文件夹的用户权限。

看到如下信息就说明启动成功了。

[--12T11::,][INFO ][o.e.n.Node               ] [] initializing ...
[--12T11::,][INFO ][o.e.e.NodeEnvironment ] [4LdtI3q] using [] data paths, mounts [[/home (/dev/mapper/cl-home)]], net usable_space [.9gb], net total_space [.9gb], types [xfs]
[--12T11::,][INFO ][o.e.e.NodeEnvironment ] [4LdtI3q] heap size [.7mb], compressed ordinary object pointers [true]
[--12T11::,][INFO ][o.e.n.Node ] [4LdtI3q] node name derived from node ID [4LdtI3q-T2ehl0Z6Xv6cOA]; set [node.name] to override
[--12T11::,][INFO ][o.e.n.Node ] [4LdtI3q] version[6.3.], pid[], build[default/tar/eb782d0/--29T21::.107521Z], OS[Linux/3.10.-.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) -Bit Server VM/1.8.0_112/25.112-b15]
[--12T11::,][INFO ][o.e.n.Node ] [4LdtI3q] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.af3cuwkJ, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=, -XX:GCLogFileSize=64m, -Des.path.home=/home/lings/elastic/elasticsearch-6.3., -Des.path.conf=/home/lings/elastic/elasticsearch-6.3./config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [aggs-matrix-stats]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [analysis-common]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [ingest-common]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [lang-expression]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [lang-mustache]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [lang-painless]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [mapper-extras]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [parent-join]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [percolator]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [rank-eval]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [reindex]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [repository-url]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [transport-netty4]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [tribe]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-core]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-deprecation]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-graph]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-logstash]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-ml]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-monitoring]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-rollup]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-security]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-sql]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-upgrade]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-watcher]
[--12T11::,][INFO ][o.e.p.PluginsService ] [4LdtI3q] no plugins loaded
[--12T11::,][INFO ][o.e.x.s.a.s.FileRolesStore] [4LdtI3q] parsed [] roles from file [/home/lings/elastic/elasticsearch-6.3./config/roles.yml]
[--12T11::,][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/] [Main.cc@] controller ( bit): Version 6.3. (Build 4d0b8f0a0ef401) Copyright (c) Elasticsearch BV
[--12T11::,][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[--12T11::,][INFO ][o.e.d.DiscoveryModule ] [4LdtI3q] using discovery type [zen]
[--12T11::,][INFO ][o.e.n.Node ] [4LdtI3q] initialized
[--12T11::,][INFO ][o.e.n.Node ] [4LdtI3q] starting ...
[--12T11::,][INFO ][o.e.t.TransportService ] [4LdtI3q] publish_address {127.0.0.1:}, bound_addresses {[::]:}, {127.0.0.1:}
[--12T11::,][WARN ][o.e.b.BootstrapChecks ] [4LdtI3q] max file descriptors [] for elasticsearch process is too low, increase to at least []
[--12T11::,][WARN ][o.e.b.BootstrapChecks ] [4LdtI3q] max virtual memory areas vm.max_map_count [] is too low, increase to at least []
[--12T11::,][INFO ][o.e.c.s.MasterService ] [4LdtI3q] zen-disco-elected-as-master ([] nodes joined)[, ], reason: new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:}{ml.machine_memory=, xpack.installed=true, ml.max_open_jobs=, ml.enabled=true}
[--12T11::,][INFO ][o.e.c.s.ClusterApplierService] [4LdtI3q] new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:}{ml.machine_memory=, xpack.installed=true, ml.max_open_jobs=, ml.enabled=true}, reason: apply cluster state (from master [master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:}{ml.machine_memory=, xpack.installed=true, ml.max_open_jobs=, ml.enabled=true} committed version [] source [zen-disco-elected-as-master ([] nodes joined)[, ]]])
[--12T11::,][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [4LdtI3q] publish_address {127.0.0.1:}, bound_addresses {[::]:}, {127.0.0.1:}
[--12T11::,][INFO ][o.e.n.Node ] [4LdtI3q] started
[--12T11::,][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [4LdtI3q] Failed to clear cache for realms [[]]
[--12T11::,][INFO ][o.e.l.LicenseService ] [4LdtI3q] license [d717e767-fe3f-4b36-bc85-cad6dbde3efc] mode [basic] - valid
[--12T11::,][INFO ][o.e.g.GatewayService ] [4LdtI3q] recovered [] indices into cluster_state
[--12T11::,][INFO ][o.e.c.r.a.AllocationService] [4LdtI3q] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[customer][], [customer][]] ...]).

二、REST使用示例

1. 查看集群的状态
curl -X GET "localhost:9200/_cat/health?v"
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1531365002 11:10:02 elasticsearch yellow 1 1 10 10 0 0 10 0 - 50.0%
注意这个status的状态是yellow,理想的状态应该是green,原因是没有配置集群。 2. 查看节点的状态
curl -X GET "localhost:9200/_cat/nodes?v"
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1 20 57 3 0.07 0.06 0.06 mdi * 4LdtI3q 3. 查看所有索引(类比mysql的库)的状态
curl -X GET 'http://localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open bank BO1IYzoXSpGGyYP3XKo4bA 5 1 1000 0 475.1kb 475.1kb
yellow open customer -8lF3GAFT1WI43gR9DiYpw 5 1 5 0 17.7kb 17.7kb
有两个索引bank和customer,一个1000条数据,一个有5条数据。 4. 查看索引(customer)的状态
curl -X PUT "localhost:9200/customer?pretty"
{
"error" : {
"root_cause" : [
{
"type" : "resource_already_exists_exception",
"reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists",
"index_uuid" : "-8lF3GAFT1WI43gR9DiYpw",
"index" : "customer"
}
],
"type" : "resource_already_exists_exception",
"reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists",
"index_uuid" : "-8lF3GAFT1WI43gR9DiYpw",
"index" : "customer"
},
"status" : 400
} 5. 更新(不存在时则增加)索引的document(类比Mysql的行)
curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
'
响应如下:
{
"_index" : "customer",
"_type" : "_doc",
"_id" : "1",
"_version" : 8, //每更新一次版本记录就+1
"result" : "updated",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 7,
"_primary_term" : 4
} 6. 增加索引的document(和PUT增加的区别就是不用指定document的id,会随机生成一个)
curl -X POST "localhost:9200/customer/_doc?pretty" -H 'Content-Type: application/json' -d'
{
"name": "Jane Doe"
}
'
响应如下:
{
"_index" : "customer",
"_type" : "_doc",
"_id" : "Bu1tjWQBZa8_pSJjP5AF", //随机生成的id
"_version" : 1,
"result" : "created",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 8,
"_primary_term" : 4
} 7. 查看指定的document
curl -X GET "localhost:9200/customer/_doc/1?pretty"
响应结果
{
"_index" : "customer",
"_type" : "_doc",
"_id" : "1",
"_version" : 8,
"found" : true,
"_source" : {
"name" : "John Doe"
}
} 8. 删除指定的document
curl -X DELETE "localhost:9200/customer/_doc/B-1ujWQBZa8_pSJjH5CC?pretty"
响应结果:
{
"_index" : "customer",
"_type" : "_doc",
"_id" : "B-1ujWQBZa8_pSJjH5CC",
"_version" : 3,
"result" : "not_found",
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 7,
"_primary_term" : 4
} 9. 批量插入文本数据
curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@/home/lings/accounts.json"
/home/lings/accounts.json为json文件的存放路径 10. 按照account_number升幂排序
curl -X GET "localhost:9200/bank/_search?q=*&sort=account_number:asc&pretty"
完全等效于这个命令
curl -X GET "localhost:9200/bank/_search" -H 'Content-Type: application/json' -d'
{
"query": { "match_all": {} },
"sort": [
{ "account_number": "asc" }
]
}
' 响应结果:
{
"took" : 3, //查询花费了3ms
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1000, //一共有1000条数据
"max_score" : null,
"hits" : [
{
"_index" : "bank",
"_type" : "_doc",
"_id" : "0",
"_score" : null,
"_source" : {
"account_number" : 0,
"balance" : 16623,
"firstname" : "Bradshaw",
"lastname" : "Mckenzie",
"age" : 29,
"gender" : "F",
"address" : "244 Columbus Place",
"employer" : "Euron",
"email" : "bradshawmckenzie@euron.com",
"city" : "Hobucken",
"state" : "CO"
},
"sort" : [
0
]
},
......省略中间数据
{
"_index" : "bank",
"_type" : "_doc",
"_id" : "9",
"_score" : null,
"_source" : {
"account_number" : 9,
"balance" : 24776,
"firstname" : "Opal",
"lastname" : "Meadows",
"age" : 39,
"gender" : "M",
"address" : "963 Neptune Avenue",
"employer" : "Cedward",
"email" : "opalmeadows@cedward.com",
"city" : "Olney",
"state" : "OH"
},
"sort" : [
9 //一共只返回了10个document
]
}
]
}
} 11. 查询住址包含Street单词的所有document
curl -X GET "localhost:9200/bank/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": { "match": {"address":"Street"} },
"sort": [
{ "account_number": "desc" }
]
}
'
响应如下:
{
"took" : 9,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 385, //满足条件的385个,总共还是返回10个document
"max_score" : null,
"hits" : [
{
"_index" : "bank",
"_type" : "_doc",
"_id" : "996",
"_score" : null,
"_source" : {
"account_number" : 996,
"balance" : 17541,
"firstname" : "Andrews",
"lastname" : "Herrera",
"age" : 30,
...... 12. 查询住址包含Street单词的降幂排序后的2个document
curl -X GET "localhost:9200/bank/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": { "match": {"address":"Street"} },
"sort": [
{ "account_number": "desc" }
],
"size":2
}
'
响应如下:
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 385,
"max_score" : null,
"hits" : [
{
"_index" : "bank",
"_type" : "_doc",
"_id" : "996",
"_score" : null,
"_source" : {
"account_number" : 996,
"balance" : 17541,
"firstname" : "Andrews",
"lastname" : "Herrera",
"age" : 30,
"gender" : "F",
"address" : "570 Vandam Street",
"employer" : "Klugger",
"email" : "andrewsherrera@klugger.com",
"city" : "Whitehaven",
"state" : "MN"
},
"sort" : [
996
]
},
{
"_index" : "bank",
"_type" : "_doc",
"_id" : "994",
"_score" : null,
"_source" : {
"account_number" : 994,
"balance" : 33298,
"firstname" : "Madge",
"lastname" : "Holcomb",
"age" : 31,
"gender" : "M",
"address" : "612 Hawthorne Street",
"employer" : "Escenta",
"email" : "madgeholcomb@escenta.com",
"city" : "Alafaya",
"state" : "OR"
},
"sort" : [
994
]
}
]
}
}