{"id":310,"date":"2019-09-20T22:36:20","date_gmt":"2019-09-20T15:36:20","guid":{"rendered":"http:\/\/geeksops.com\/?p=310"},"modified":"2020-07-04T05:59:24","modified_gmt":"2020-07-03T22:59:24","slug":"monitoring-mtr-using-influxdb-grafana","status":"publish","type":"post","link":"https:\/\/rotreein.com\/?p=310","title":{"rendered":"Monitoring MTR Statistic with Influxdb &#038; Grafana"},"content":{"rendered":"\n<p>The sometimes network guy confused to determine root cause related network problem &amp; no data statistic to report as evidence of the root cause at the previously time. This time we will create a simple script to collect data and stored into influxdb. Take a cup of coffee and smoke ( not on the water ), Let&#8217;s Go! \ud83d\ude09<\/p>\n\n\n\n<p>Install requirement package. we tested it in our environment using debian 9<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">apg-get install mtr docker-ce python3-pip -y<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">pip3 install influxdb<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">docker pull influxdb<\/p>\n\n\n\n<p>The result of docker image have you download<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">influxdb                        latest              <strong>234234234<\/strong>        2 weeks ago         264MB<\/p>\n\n\n\n<p><br>in this case above, image id docker is <strong>234234234<\/strong>. Run container &amp; make sure the container already up &amp; have exposed to external<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">#docker run -d -p 8086:8086  234234234<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">#docker ps -a | grep  23423423<br>842501af3012        <strong>234234234<\/strong>        &#8220;\/entrypoint.sh infl\u2026&#8221;   20 hours ago        Up 20 hours         0.0.0.0:8086-&gt;8086\/tcp<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">#netstat -ntap | grep 8086<br>tcp6       0      0 :::8086                 :::*                    LISTEN      27816\/docker-proxy<\/p>\n\n\n\n<p>login into docker and execute some of command in influxdb <\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\"># docker exec -it  234234234 \/usr\/bin\/influx -precision rfc3339<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">Connected to http:\/\/localhost:8086 version 1.7.8<br>InfluxDB shell version: 1.7.8<br>&gt;CREATE DATABASE mtr<br>&gt;USE mtr<br>&gt;CREATE USER mtruser with PASSWORD &#8216;123456&#8217; WITH ALL PRIVILEGES<br>&gt;GRANT ALL ON mtr TO mtruser<\/p>\n\n\n\n<p><br>Create a bash script for collect data mtr, for example the name of file is <strong>mtr-exporter.sh<\/strong> in folder<strong> \/opt\/mtr-exporter-influx <\/strong>and put these script into those file have you created. in this scenario all files will put in folder <strong>\/opt\/mtr-exporter-influx.<\/strong><\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\"># mkdir -p  \/opt\/mtr-exporter-influx<\/p>\n\n\n\n<p style=\"color:#f3f5f7\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\"># vi \/opt\/mtr-exporter-influx\/mtr-exporter.sh<\/p>\n\n\n\n<p>Put the script below &amp; save<\/p>\n\n\n\n<pre data-mode=\"typescript\" data-theme=\"monokai\" data-fontsize=\"14\" data-lines=\"Infinity\" class=\"wp-block-simple-code-block-ace\">#!\/bin\/bash\n\nINTERVAL=60\nINFLUXDB_HOST=\"192.168.38.60\"\nINFLUXDB_PORT=8086\n\nfunction monitor_mtr() {\n  for MTR_HOST in $(cat \/opt\/mtr-exporter-influx\/list-ip-dest); do\n    ( mtr --report --tcp --port=443 --json --report-cycles $CYCLES $MTR_HOST | \/usr\/bin\/python3 \/opt\/mtr-exporter-influx\/save_data.py --host $INFLUXDB_HOST --port $INFLUXDB_PORT ) &amp;\n  done\n}\n\nwhich mtr &amp;>\/dev\/null\nif [ $? -eq 1 ]; then\n  echo \"mtr not found, please install mtr \"\n  exit 1\nelse\necho \"collecting data...\"\nfi \nwhile true; do\n  monitor_mtr\n  sleep $INTERVAL\ndone<\/pre>\n\n\n\n<p style=\"color:#f6f9fa\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\"># chmod +x \/opt\/mtr-exporter-influx\/mtr_exporter.sh<\/p>\n\n\n\n<p>Create List of IP Destination that you will monitor. For example 1.1.1.1 and 8.8.8.8<\/p>\n\n\n\n<p style=\"color:#f6f9fa\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\"># vi \/opt\/mtr-exporter-influx\/list-ip-dest<\/p>\n\n\n\n<p>The following format file after file created<\/p>\n\n\n\n<p style=\"color:#f6f9fa\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\"># cat \/opt\/mtr-exporter-influx\/list-ip-dest<br>1.1.1.1<br>8.8.8.8<br>#<\/p>\n\n\n\n<p>Create a python script for export data mtr into Influxdb, for example the name of file save_data.py and put these script into your have you created<\/p>\n\n\n\n<pre data-mode=\"python\" data-theme=\"monokai\" data-fontsize=\"14\" data-lines=\"Infinity\" class=\"wp-block-simple-code-block-ace\">#!\/usr\/bin\/env python3\nimport argparse\nimport json\nimport sys\nimport datetime as dt\nimport logging\n\nfrom influxdb import InfluxDBClient, SeriesHelper\n\nlogging.basicConfig(level=logging.INFO)\ndb_name = 'mtr'\nuser = 'mtruser'\npassword = '123456'\n\nclass HubEntry(SeriesHelper):\n     class Meta:\n         series_name = '{destination}'\n         fields = ['time', 'loss', 'snt', 'last', 'avg', 'best', 'wrst', 'stdev']\n         tags = ['destination', 'hop']\n         \ndef get_cmd_arguments():\n     parser = argparse.ArgumentParser(description='JSON parser')\n     parser.add_argument('--host', default='192.168.38.60', help='influxdb host')\n     parser.add_argument('--port', default=8086, help='influxdb port')\nreturn parser.parse_args()\n\nmtr_result = json.load(sys.stdin)\n# ping destination\ndestination = mtr_result['report']['mtr']['dst']\nreport_time = dt.datetime.utcnow()\nfor hub in mtr_result['report']['hubs']:\n    # persist the hub entry\n    # Modifying the data if needed so that is can be easily sorted in the event of more than 9 hops.\n    if len(hub['count']) &lt; 2:\n        hop = \"0\" + hub['count'] + \"-\" + hub['host']\n    else:\n        hop = hub['count'] + \"-\" + hub['host']\n    HubEntry(\n        time=report_time,\n        destination=destination,\n        hop=hop,\n        loss=hub['Loss%'],\n        snt=hub['Snt'],\n        last=hub['Last'],\n        avg=hub['Avg'],\n        best=hub['Best'],\n        wrst=hub['Wrst'],\n        stdev=hub['StDev']\n    )\nHubEntry.commit()\n\nif name == 'main':\n     main()<\/pre>\n\n\n\n<p>Assuming we have 3 files that have been created<\/p>\n\n\n\n<p style=\"color:#f5f7f8\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">#ls -l<br>mtr-exporter.sh<br>save_data.py<br>list-ip-dest<\/p>\n\n\n\n<p>Create a linux service mtr-exporter, the example service name is mtr-exporter, create a file <strong>mtr-exporter.service<\/strong> in folder <strong>\/lib\/systemd\/system<\/strong><\/p>\n\n\n\n<p style=\"color:#f5f7f8\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">[Unit]<br> Description=mtr exporter<br> After=multi-user.target<br><br>[Service]<br> Type=idle<br> ExecStart=\/bin\/bash \/opt\/mtr-exporter-influx\/mtr-export.sh<br><br>[Install]<br> WantedBy=multi-user.target<\/p>\n\n\n\n<p style=\"color:#f5f7f8\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">systemctl daemon-reload<\/p>\n\n\n\n<p style=\"color:#f5f7f8\" class=\"has-text-color has-background has-small-font-size has-very-dark-gray-background-color\">systemctl start mtr-exporter.service<\/p>\n\n\n\n<p><br>Add datasource in Grafana<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"770\" height=\"805\" src=\"https:\/\/geeksops.com\/wp-content\/uploads\/2019\/09\/image-12.png\" alt=\"\" class=\"wp-image-364\" srcset=\"https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-12.png 770w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-12-287x300.png 287w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-12-768x803.png 768w\" sizes=\"(max-width: 770px) 100vw, 770px\" \/><\/figure>\n\n\n\n<p><br><br>Download the<a download=\"\" href=\"https:\/\/grafana.com\/api\/dashboards\/3288\/revisions\/1\/download\"> dashboard<\/a> and then import from grafana<br><br><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"551\" src=\"https:\/\/geeksops.com\/wp-content\/uploads\/2019\/09\/image-13-1024x551.png\" alt=\"\" class=\"wp-image-369\" srcset=\"https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-13-1024x551.png 1024w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-13-300x161.png 300w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-13-768x413.png 768w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-13.png 1084w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><figcaption><br><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"923\" height=\"601\" src=\"https:\/\/geeksops.com\/wp-content\/uploads\/2019\/09\/image-15.png\" alt=\"\" class=\"wp-image-373\" srcset=\"https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-15.png 923w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-15-300x195.png 300w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-15-768x500.png 768w\" sizes=\"(max-width: 923px) 100vw, 923px\" \/><figcaption><br><br><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"919\" height=\"577\" src=\"https:\/\/geeksops.com\/wp-content\/uploads\/2019\/09\/image-16.png\" alt=\"\" class=\"wp-image-374\" srcset=\"https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-16.png 919w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-16-300x188.png 300w, https:\/\/rotreein.com\/wp-content\/uploads\/2019\/09\/image-16-768x482.png 768w\" sizes=\"(max-width: 919px) 100vw, 919px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Reference <a href=\"https:\/\/grafana.com\/grafana\/dashboards\/3288\">https:\/\/grafana.com\/grafana\/dashboards\/3288<\/a><br><br><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The sometimes network guy confused to determine root cause related network problem &amp; no data statistic to report as evidence of the root cause at the previously time. This time we will create a simple script to collect data and stored into influxdb. Take a cup of coffee and smoke ( not on the water&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,6],"tags":[],"_links":{"self":[{"href":"https:\/\/rotreein.com\/index.php?rest_route=\/wp\/v2\/posts\/310"}],"collection":[{"href":"https:\/\/rotreein.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rotreein.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rotreein.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rotreein.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=310"}],"version-history":[{"count":113,"href":"https:\/\/rotreein.com\/index.php?rest_route=\/wp\/v2\/posts\/310\/revisions"}],"predecessor-version":[{"id":525,"href":"https:\/\/rotreein.com\/index.php?rest_route=\/wp\/v2\/posts\/310\/revisions\/525"}],"wp:attachment":[{"href":"https:\/\/rotreein.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=310"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rotreein.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=310"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rotreein.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=310"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}