📡Granfana Dashboard - Monitor node

1. Install Prometheus

cd $HOME
wget https://github.com/prometheus/prometheus/releases/download/v2.42.0/prometheus-2.42.0.linux-amd64.tar.gz
tar xvf prometheus-2.42.0.linux-amd64.tar.gz
sudo cp prometheus-2.42.0.linux-amd64 /opt/prometheus

# Create Prometheus user

sudo useradd --no-create-home --shell /bin/false prometheus

# Create directory

sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo touch /etc/default/prometheus
sudo chown prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus
sudo chown -R prometheus:prometheus /opt/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus
sudo chown -R prometheus:prometheus /var/lib/prometheus
sudo chmod 755 /opt/prometheus/prometheus

# Create Prometheus config

sudo nano /etc/prometheus/prometheus.yml

Sample:

# Sample config for Prometheus.

global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).



# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).


# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets: ['localhost:9093']

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    scrape_timeout: 5s

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['localhost:9090']

  - job_name: node-exporter
    # If prometheus-node-exporter is installed, grab stats about the local
    # machine by default.
    static_configs:
      - targets: ['IP_VPS_1:9200']
      - targets: ['IP_VPS_2:9200']
      - targets: ['IP_VPS_3:9200']
      - targets: ['IP_VPS_4:9200']
  
  - job_name: '0gchain'
    static_configs:
      - targets: ['localhost:51660']

  - job_name: '0gchain-Contabo'
    static_configs:
      - targets: ['IP_VPS_1:47660']

  - job_name: 'Airchain'
    static_configs:
      - targets: ['IP_VPS_2:19660']

  - job_name: 'Initia'
    static_configs:
      - targets: ['IP_VPS_3:46660']

Explain config:

job_name: '0gchain'
    static_configs:
      - targets: ['localhost:51660']

For config this port IP_VPS:51660 you have to edit 2 files at your node app.toml & config.tomland Restart node

Set permissions for the configuration file

sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

# Create service file

sudo nano /lib/systemd/system/prometheus.service

Content:

[Unit]
Description=Monitoring system and time series database
Documentation=https://prometheus.io/docs/introduction/overview/ man:prometheus(1)
After=time-sync.target

[Service]
Restart=on-failure
User=prometheus
EnvironmentFile=/etc/default/prometheus
ExecStart=/opt/prometheus/prometheus $ARGS --web.listen-address=:9099 --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus
ExecReload=/bin/kill -HUP $MAINPID
TimeoutStopSec=20s
SendSIGKILL=no

# systemd hardening-options
AmbientCapabilities=
CapabilityBoundingSet=
DeviceAllow=/dev/null rw
DevicePolicy=strict
LimitMEMLOCK=0
LimitNOFILE=8192
LockPersonality=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
PrivateDevices=true
PrivateTmp=true
PrivateUsers=true
ProtectControlGroups=true
ProtectHome=true
ProtectKernelModules=true
ProtectKernelTunables=true
ProtectSystem=full
RemoveIPC=true
RestrictNamespaces=true
RestrictRealtime=true
SystemCallArchitectures=native

[Install]
WantedBy=multi-user.target

Reload systemd & Start

sudo systemctl daemon-reload
sudo systemctl enable prometheus

sudo systemctl restart prometheus && \
sudo systemctl status prometheus

2. Install Node Exporter:

Node Exporter is a tool that collects hardware and OS-level metrics from Linux systems. It exposes these metrics in a format that Prometheus can scrape, providing insights into system performance and resource usage.

cd ~
wget https://github.com/prometheus/node_exporter/releases/download/v1.5.0/node_exporter-1.5.0.linux-amd64.tar.gz
tar xvf node_exporter-1.5.0.linux-amd64.tar.gz
sudo cp node_exporter-1.5.0.linux-amd64/node_exporter /usr/local/bin/

# Create service file

sudo nano /etc/systemd/system/node_exporter.service

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=root
ExecStart=/root/node_exporter-1.5.0.linux-amd64/node_exporter --web.listen-address=:9200
Restart=always

[Install]
WantedBy=default.target

*Port listen: 9200 (you can custom) with flag `--web.listen-address=:9200`

chmod +x /root/node_exporter-1.5.0.linux-amd64/node_exporter

# Reload systemd and start

systemctl daemon-reload
sudo systemctl enable node_exporter

systemctl restart node_exporter && \
sudo systemctl status node_exporter

3. Install Grafana

sudo apt-get install -y software-properties-common
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
sudo apt-get update
sudo apt-get install grafana
sudo systemctl enable grafana-server

Start & check status

sudo systemctl restart grafana-server && \
sudo systemctl status grafana-server

Web interface:

http://your_server_ip:3000
use:admin
pass:admin
Login & change

# Grafana config

Default port is 3000 if you need custom find this line to change to 3002and restart

sudo nano /etc/grafana/grafana.ini

# The http port  to use
http_port = 3002

4. Set up Dashboard and Alerting

# Access your Grafana

Launch browser and go to http://<YOUR-SERVER-GLOBAL-IP>:3000/

# Click Add new data source and select Prometheus

Enter http://localhost:9099 into Prometheus server URL

Scroll to the bottom and click Save & test

This port the same port Prometheus service config

# Go to Dashboards to create dashboard

Click Upload JSON file:

Please download the 0G-grafana-json-file and upload: https://josephtran.co/0g_validator.json

# Select prometheus datasource

# Finish