1) Install node_exporter on each host

Here’s a typical systemd-based installation on an Ubuntu host (replace version as needed):

# Download
wget https://github.com/prometheus/node_exporter/releases/download/v1.7.1/node_exporter-1.7.1.linux-amd64.tar.gz
tar xvf node_exporter-1.7.1.linux-amd64.tar.gz
sudo mv node_exporter-1.7.1.linux-amd64/node_exporter /usr/local/bin/
# Create systemd unit
sudo tee /etc/systemd/system/node_exporter.service >/dev/null <<'EOF'
[Unit]
Description=Prometheus Node Exporter
After=network.target

[Service]
User=nobody
ExecStart=/usr/local/bin/node_exporter --collector.textfile.directory=/var/lib/node_exporter/textfile_collector
Restart=always

[Install]
WantedBy=multi-user.target
EOF
# Start service
sudo mkdir -p /var/lib/node_exporter/textfile_collector
sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter
# Verify
curl http://localhost:9100/metrics | head -n 20

The --collector.textfile.directory allows adding custom metrics as text files (useful for scripts). Use User=nobody or a dedicated user for security.

2) Configure Prometheus to scrape node_exporter

Prometheus configuration example (prometheus.yml). Add a job that scrapes all your nodes:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node_exporter'
    metrics_path: /metrics
    scrape_interval: 15s
    static_configs:
      - targets:
        - 'host1.example.com:9100'
        - 'host2.example.com:9100'
    # Optional relabeling to simplify instance label
    relabel_configs:
      - source_labels: [__address__]
        regex: '([^:]+):.*'
        target_label: instance
        replacement: '$1'

After updating prometheus.yml, restart Prometheus and check the Targets page (http://PROM_SERVER:9090/targets) to ensure node exporters are UP.

3) Configure Grafana & add Prometheus datasource

Two ways to add the Prometheus datasource:

Option A — UI (Quick)

1. Open Grafana web UI (e.g., http://GRAFANA_HOST:3000), log in (default admin/admin).
2. Go to Configuration → Data Sources → Add data source
3. Select Prometheus, set URL to http://PROMETHEUS_HOST:9090, and click Save & Test.

Option B — Provisioning (recommended for automation)

Create a YAML file at /etc/grafana/provisioning/datasources/prometheus.yaml:

apiVersion: 1
datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true
    editable: false

Restart Grafana — the datasource will be automatically created. This is ideal for repeatable infra-as-code deployments.

4) Dashboards & example PromQL queries

You can import community dashboards (search “Node Exporter Full” on grafana.com) or create custom panels. Below are useful PromQL queries for common panels:

100 - (avg by (instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)

node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes

(node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes) / node_memory_MemTotal_bytes * 100

100 - (node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"} * 100)

node_load1{instance="$instance"}

rate(node_network_receive_bytes_total{device!="lo"}[5m])
rate(node_network_transmit_bytes_total{device!="lo"}[5m])

When creating dashboards, group panels by purpose (CPU, Memory, Disk, Network, Processes) and include host selector variables for easy filtering.

5) Alerts (Prometheus Alertmanager or Grafana)

Two common approaches:

1. Prometheus rules + Alertmanager: Define Prometheus alerting rules and route alerts via Alertmanager to email/Slack/PagerDuty.
2. Grafana alerting: Grafana’s unified alerting can evaluate Prometheus queries and send notifications. This centralizes alert management in Grafana.

Example Prometheus alert rule (high CPU)

groups:
- name: node_alerts
  rules:
  - alert: HighCPUUsage
    expr: (100 - (avg by (instance) (irate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)) > 85
    for: 2m
    labels:
      severity: critical
    annotations:
      summary: "High CPU on {{ $labels.instance }}"
      description: "CPU usage is > 85% for more than 2 minutes."

Place rules in a file (e.g., node_rules.yml) and reference it in prometheus.yml under rule_files. Configure Alertmanager endpoints in prometheus.yml under alerting.