We are going to deploy one master node, one hot data node, warm data node, cold data node and one kibana node in same local machine. Then we play with Index Life Cycle Management UI.
Steps:
1. Clone below github project and go to cloned directory
https://github.com/dhanuka84/es-hot-warm-cold.git
2. You need to change my local machine configurations according to your machine
Local IP: 192.168.0.114
Project Folder: /home/dhanuka/research/elastic/docker2/es-hot-warm-cold/
How to change: Use below linux command to replace all
find ./ -type f -exec sed -i 's/192.168.0.114/new_ip/g' {} \;
find ./ -type f -exec sed -i 's#home/dhanuka/research/elastic/docker2#new_folder#g' {} \;
Install Docker :
Install Docker :
http://dhanuka84.blogspot.com/2019/02/install-docker-in-ubuntu.html
Change system settings: Run below command in terminal
sudo echo "vm.max_map_count=262144" >> /etc/sysctl.conf
Change system settings: Run below command in terminal
sudo echo "vm.max_map_count=262144" >> /etc/sysctl.conf
3. Run run-all.sh script as below
sh run-all.sh
You can check terminal output with nohup.out file
4. Then login to kibana as below.
5. Then go to monitoring tab
You can see number of Elasticsearch and Kibana nodes with their status.
6. Now let's create ILCM policy
Go to management tab and click index lifecycle policies as below.
7. Let's create policy as shown above and below.
In here we will create hot, warm , cold and delete phases of an index life cycle
8. Then you need to create a index template as below.
PUT _template/my_template
{
"index_patterns": ["kibana_sample_data*"],
"settings": {
"index.routing.allocation.require.my_node_type": "hot"
}
}
9. Finally you need to add previously created ILCM policy to index template as below
10. Now when ever you create new index based on that template , ILCM policywill be applied to that index. You can verify template as below.
Request :
GET _template/my_template
Response:
{
"my_template" : {
"order" : 0,
"index_patterns" : [
"kibana_sample_data*"
],
"settings" : {
"index" : {
"lifecycle" : {
"name" : "hotwarm-policy",
"rollover_alias" : "my_index_alias"
},
"routing" : {
"allocation" : {
"require" : {
"my_node_type" : "hot"
}
}
}
}
},
"mappings" : { },
"aliases" : { }
}
}
