Monitoring

This guide explains how to monitor the performance and health of PCH-SIG.

Monitoring architecture

┌─────────────────────────────────────────────────────────────┐
│                    Local Machine                             │
│  ┌─────────────┐    ┌─────────────┐                         │
│  │   Grafana   │◀───│  Prometheus │                         │
│  │   :3001     │    │   :9090     │                         │
│  └─────────────┘    └──────┬──────┘                         │
└────────────────────────────┼────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────┐
│                 Server serveur-production                         │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐         │
│  │  cAdvisor   │  │ NodeExporter│  │ PG Exporter │         │
│  │   :8080     │  │   :9100     │  │   :9187     │         │
│  └─────────────┘  └─────────────┘  └─────────────┘         │
│                                                              │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐         │
│  │Redis Export │  │    Loki     │  │  Promtail   │         │
│  │   :9121     │  │   :3100     │  │             │         │
│  └─────────────┘  └─────────────┘  └─────────────┘         │
└─────────────────────────────────────────────────────────────┘

---

Start monitoring

Prometheus (metrics collection)

cd monitoring/prometheus
.\prometheus.exe --config.file=prometheus.yml

Grafana (visualization)

& "C:\Program Files\GrafanaLabs\grafana\bin\grafana-server.exe" `
  --homepath="C:\Program Files\GrafanaLabs\grafana" `
  --config="C:\Users\clave\WebstormProjects\pch-sig\monitoring\grafana\grafana-custom.ini"

Grafana access

• URL: http://localhost:3001
• Login: admin
• Password: Passer@123

---

Available dashboards

Docker Overview

URL: http://localhost:3001/d/pch-docker-overview

Displayed metrics:

• CPU per container
• Memory used
• Network (bytes in/out)
• Disk I/O

PostgreSQL

URL: http://localhost:3001/d/pch-postgresql

Displayed metrics:

• Active connections
• Transactions per second
• Slow queries
• Table sizes
• Cache hit ratio

Docker Logs

URL: http://localhost:3001/d/pch-docker-logs

Features:

• Real-time logs
• Container filtering
• Log search
• History

---

Key metrics

Application

Metric	Description	Alert threshold
API response time	Average request latency	> 500ms
5xx errors	Number of server errors	> 10/min
Requests/second	API load	> 100/s

Database

Metric	Description	Alert threshold
Active connections	PostgreSQL connections	> 80% max
Cache hit ratio	Cache efficiency	< 95%
Transactions/second	DB activity	Variable
Slow queries	Queries > 1s	> 5/min

Containers

Metric	Description	Alert threshold
CPU	CPU usage	> 80%
Memory	RAM usage	> 90%
Restarts	Number of restarts	> 0 in 1h

System

Metric	Description	Alert threshold
Disk space	Disk usage	> 85%
Load average	System load	> CPU count
System memory	RAM used	> 90%

---

Prometheus configuration

File: monitoring/prometheus/prometheus.yml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'cadvisor'
    static_configs:
      - targets: ['serveur-production:8080']

  - job_name: 'node'
    static_configs:
      - targets: ['serveur-production:9100']

  - job_name: 'postgresql'
    static_configs:
      - targets: ['serveur-production:9187']

  - job_name: 'redis'
    static_configs:
      - targets: ['serveur-production:9121']

---

Alerts

Configure a Grafana alert

1. Open a dashboard
2. Click on the panel to monitor
3. Edit > Alert
4. Configure:
   • Condition (e.g., avg > 80)
   • Duration (e.g., for 5m)
   • Notification (email, Slack, etc.)

CPU alert example

alert: HighCPUUsage
expr: container_cpu_usage_seconds_total{name="pch_backend"} > 0.8
for: 5m
labels:
  severity: warning
annotations:
  summary: "High CPU on backend"
  description: "Backend is using more than 80% CPU for 5 minutes"

Notification channels

Channel	Configuration
Email	SMTP server
Slack	Webhook URL
Teams	Webhook URL
PagerDuty	Integration key

---

Check monitoring status

Verification script

File: monitoring/check-ports.ps1

# Check monitoring ports
$Ports = @{
    "cAdvisor" = "serveur-production:8080"
    "Node Exporter" = "serveur-production:9100"
    "PostgreSQL Exporter" = "serveur-production:9187"
    "Redis Exporter" = "serveur-production:9121"
    "Loki" = "serveur-production:3100"
}

foreach ($Service in $Ports.Keys) {
    $Target = $Ports[$Service]
    try {
        $Response = Invoke-WebRequest -Uri "http://$Target" -TimeoutSec 5 -UseBasicParsing
        Write-Host "$Service ($Target): OK" -ForegroundColor Green
    } catch {
        Write-Host "$Service ($Target): ERROR" -ForegroundColor Red
    }
}

Check Prometheus targets

1. Open http://localhost:9090/targets
2. All targets should be in "UP" state

---

Health endpoint

API Health Check

curl http://localhost:8000/api/health

Expected response:

{
    "status": "ok",
    "database": "connected",
    "redis": "connected",
    "timestamp": "2024-01-15T10:30:00Z"
}

Implement a health check

// src/Controller/HealthController.php
#[Route('/api/health', methods: ['GET'])]
public function health(): JsonResponse
{
    $checks = [];

    // PostgreSQL
    try {
        $this->entityManager->getConnection()->executeQuery('SELECT 1');
        $checks['database'] = 'connected';
    } catch (\Exception $e) {
        $checks['database'] = 'error';
    }

    // Redis
    try {
        $this->redis->ping();
        $checks['redis'] = 'connected';
    } catch (\Exception $e) {
        $checks['redis'] = 'error';
    }

    return new JsonResponse([
        'status' => 'ok',
        ...$checks,
        'timestamp' => (new \DateTime())->format('c')
    ]);
}

---

Server exporters

Start cAdvisor

docker run -d \
  --name=pch_cadvisor \
  --restart=always \
  --privileged \
  -p 8080:8080 \
  -v /:/rootfs:ro \
  -v /var/run:/var/run:ro \
  -v /sys:/sys:ro \
  -v /var/lib/docker/:/var/lib/docker:ro \
  gcr.io/cadvisor/cadvisor:latest

Start Node Exporter

docker run -d \
  --name=pch_node_exporter \
  --restart=always \
  --net=host \
  -v /:/host:ro \
  quay.io/prometheus/node-exporter:latest \
  --path.rootfs=/host

Start PostgreSQL Exporter

docker run -d \
  --name=pch_postgres_exporter \
  --restart=always \
  -p 9187:9187 \
  -e DATA_SOURCE_NAME="postgresql://pch_admin:pch_secure_2025@pch_postgres:5432/pch_sig?sslmode=disable" \
  quay.io/prometheuscommunity/postgres-exporter:latest

Start Redis Exporter

docker run -d \
  --name=pch_redis_exporter \
  --restart=always \
  -p 9121:9121 \
  -e REDIS_ADDR=pch_redis:6379 \
  -e REDIS_PASSWORD=redis_secure_2025 \
  oliver006/redis_exporter:latest

---

Useful PromQL queries

CPU

# CPU per container
rate(container_cpu_usage_seconds_total{name=~"pch_.*"}[5m]) * 100

# Top 5 CPU consumers
topk(5, rate(container_cpu_usage_seconds_total[5m]))

Memory

# Memory per container
container_memory_usage_bytes{name=~"pch_.*"} / 1024 / 1024

# Memory percentage
container_memory_usage_bytes / container_spec_memory_limit_bytes * 100

PostgreSQL

# Active connections
pg_stat_activity_count

# Transactions per second
rate(pg_stat_database_xact_commit{datname="pch_sig"}[5m])

---

Best practices

Metrics retention

• Detailed metrics: 15 days
• Aggregated metrics: 90 days
• Trend metrics: 1 year

Proactive monitoring

• Configure alerts BEFORE problems occur
• Set realistic thresholds
• Regularly review alerts

Documentation

• Document thresholds and their justification
• Keep an incident log
• Create runbooks for each alert

---

Next steps

• Logs
• Troubleshooting