This is a monitoring system give us the ability of monitoring latency between Kubernetes Nodes. Metrics then is exported to Prometheus with metrics latency_between_nodes
Let's take a look in manifest/monlat.yaml
, adjust the field spec.template.spec.affinity
to choose the node which deploy monlat
...
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- node1
...
Install RBAC, monlat-agent
and monlat
$ kubectl apply -f manifest/rbac.yaml
$ kubectl apply -f manifest/agents.yaml
$ kubectl apply -f manifest/monlat.yaml
View log of all monlat-agent Pods. It is expected to be looked like this:
latency_between_nodes{from="node2",to="node3"} 0.387
latency_between_nodes{from="node2",to="node1"} 0.463
latency_between_nodes{from="node2",to="node3"} 0.318
latency_between_nodes{from="node2",to="node1"} 0.440
Check if monlat is ready for scraping from Prometheus: view log of monlat-... Pod:
____ __
/ __/___/ / ___
/ _// __/ _ \/ _ \
/___/\__/_//_/\___/ v4.12.0
High performance, minimalist Go web framework
https://echo.labstack.com
____________________________________O/_______
O\
⇨ http server started on [::]:9090
Make a http request to get metrics from monlat service:
ubuntu@ubuntu:~/monlat$ curl monlat.default.svc.cluster.local:9090/metrics
latency_between_nodes{from="node1",to="node2"} 0.436
latency_between_nodes{from="node1",to="node3"} 0.314
latency_between_nodes{from="node2",to="node1"} 0.439
latency_between_nodes{from="node2",to="node3"} 0.379
latency_between_nodes{from="node3",to="node1"} 0.474
latency_between_nodes{from="node3",to="node2"} 0.323
If the system does not have any signs of stable operation as above, let's remove monlat and monlat-agent Pods.
If the problem still exists, please create an issue for this repo (Thanks).
Rebuild image of monlat
component if you need. Let's take a quick view in src/build.sh. Then, rebuild image if you need.
cd agent
vi build.sh
...
#####CONFIG HERE#####
DOCKERHUB="bonavadeur"
TAG="latest"
NAME="monlat-agent" # docker.io/$HUB/$NAME:$TAG
#####################
...
chmod +x build.sh
./build.sh image # build from Bash script to Docker image
./build.sh push # push Docker image to Docker Hub
cd ..
cd src
vi build.sh
...
#####CONFIG HERE#####
DOCKERHUB="bonavadeur"
TAG="latest"
NAME="monlat" # docker.io/$HUB/$NAME:$TAG
#####################
...
chmod +x build.sh
./build.sh image # build from Golang code to Docker image
./build.sh push # push Docker image to Docker Hub
cd ..
Specify image you use to run monlat
and specified Nodes you want to deploy it in Deployment
in file manifest/monlat.yaml and manifest/agents.yaml
Assume that Your Kubernetes run Prometheus before. Metrics are updated to prometheus immediately with metrics latency_between_nodes
.
Đào Hiệp - Bonavadeur - ボナちゃん
The Future Internet Laboratory, Room E711 C7 Building, Hanoi University of Science and Technology, Vietnam.
未来のインターネット研究室, C7 の E 711、ハノイ百科大学、ベトナム。