Étiquette : Computers

  • Socle V004 – Métriques

    Socle V004 – Métriques

    15 – Metrics

    Version : 4.0.0 Date : 2025-12-09

    1. Introduction

    Le Socle V4 expose des métriques au format Prometheus pour le monitoring et l’alerting.

    Types de métriques

    • Counter : Valeur qui ne fait qu’augmenter (requêtes, erreurs)
    • Gauge : Valeur qui peut monter et descendre (connexions actives)
    • Histogram : Distribution de valeurs (latences)
    • Summary : Similaire à histogram avec percentiles pré-calculés

    2. Configuration

    2.1 application.yml

    management:
      endpoints:
        web:
          exposure:
            include: prometheus,health,info,metrics
          base-path: /actuator
      endpoint:
        prometheus:
          enabled: true
      metrics:
        export:
          prometheus:
            enabled: true
        tags:
          application: ${socle.app_name}
          environment: ${socle.env_name}
          region: ${socle.region}
    

    2.2 Dépendances Maven

    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-registry-prometheus</artifactId>
    </dependency>
    

    3. Métriques Socle

    3.1 Métriques Workers

    # Nombre de workers
    socle_workers_total{application="socle-v4"} 5
    
    # Workers healthy
    socle_workers_healthy{application="socle-v4"} 5
    
    # Workers unhealthy
    socle_workers_unhealthy{application="socle-v4"} 0
    
    # État par worker
    socle_worker_status{worker="kafka-consumer",status="RUNNING"} 1
    socle_worker_status{worker="order-processor",status="RUNNING"} 1
    
    # Heartbeats par worker
    socle_worker_heartbeats_total{worker="kafka-consumer"} 1234
    socle_worker_missed_heartbeats{worker="kafka-consumer"} 0
    

    3.2 Métriques KvBus

    # Opérations
    socle_kvbus_operations_total{operation="get"} 12345
    socle_kvbus_operations_total{operation="put"} 6789
    socle_kvbus_operations_total{operation="delete"} 234
    
    # Latence
    socle_kvbus_operation_duration_seconds{operation="get",quantile="0.5"} 0.001
    socle_kvbus_operation_duration_seconds{operation="get",quantile="0.95"} 0.005
    socle_kvbus_operation_duration_seconds{operation="get",quantile="0.99"} 0.01
    
    # Nombre de clés
    socle_kvbus_keys_count 456
    

    3.3 Métriques Pipeline

    # Exécutions
    socle_pipeline_executions_total{pipeline="order-processing",status="SUCCESS"} 1234
    socle_pipeline_executions_total{pipeline="order-processing",status="FAILURE"} 12
    
    # Durée
    socle_pipeline_duration_seconds{pipeline="order-processing",quantile="0.5"} 0.5
    socle_pipeline_duration_seconds{pipeline="order-processing",quantile="0.95"} 2.0
    socle_pipeline_duration_seconds{pipeline="order-processing",quantile="0.99"} 5.0
    
    # Steps
    socle_pipeline_step_duration_seconds{step="validation",quantile="0.5"} 0.01
    socle_pipeline_step_duration_seconds{step="processing",quantile="0.5"} 0.3
    

    3.4 Métriques Resilience

    # Circuit breaker état (0=CLOSED, 1=HALF_OPEN, 2=OPEN)
    socle_circuit_breaker_state{name="payment-gateway"} 0
    
    # Tentatives de retry
    socle_retry_attempts_total{operation="external-api",attempt="1",success="true"} 1000
    socle_retry_attempts_total{operation="external-api",attempt="2",success="true"} 50
    socle_retry_attempts_total{operation="external-api",attempt="3",success="false"} 5
    

    3.5 Métriques TechDB (V4)

    # Opérations
    socle_techdb_operations_total{operation="saveOffset"} 5678
    socle_techdb_operations_total{operation="getOffset"} 12345
    
    # Taille des tables
    socle_techdb_rows_count{table="socle_offsets"} 23
    socle_techdb_rows_count{table="socle_events"} 456
    socle_techdb_rows_count{table="socle_log_fallback"} 0
    

    3.6 Métriques LogForwarder (V4)

    # Queue
    socle_logforwarder_queue_size 45
    socle_logforwarder_queue_capacity 10000
    
    # Logs envoyés
    socle_logforwarder_logs_sent_total 123456
    socle_logforwarder_logs_failed_total 23
    socle_logforwarder_logs_fallback_total 0
    
    # Batches
    socle_logforwarder_batches_sent_total 1234
    socle_logforwarder_batch_size{quantile="0.5"} 100
    

    4. Implémentation

    4.1 Enregistrement des métriques

    package eu.lmvi.socle.metrics;
    
    @Component
    public class SocleMetrics {
    
        private final MeterRegistry registry;
    
        // Counters
        private final Counter requestsTotal;
        private final Counter errorsTotal;
    
        // Gauges
        private final AtomicInteger activeConnections = new AtomicInteger(0);
    
        // Timers
        private final Timer requestDuration;
    
        public SocleMetrics(MeterRegistry registry) {
            this.registry = registry;
    
            // Counter
            this.requestsTotal = Counter.builder("socle_requests_total")
                .description("Total number of requests")
                .register(registry);
    
            this.errorsTotal = Counter.builder("socle_errors_total")
                .description("Total number of errors")
                .register(registry);
    
            // Gauge
            Gauge.builder("socle_active_connections", activeConnections, AtomicInteger::get)
                .description("Number of active connections")
                .register(registry);
    
            // Timer
            this.requestDuration = Timer.builder("socle_request_duration_seconds")
                .description("Request duration in seconds")
                .publishPercentiles(0.5, 0.95, 0.99)
                .register(registry);
        }
    
        public void recordRequest() {
            requestsTotal.increment();
        }
    
        public void recordError() {
            errorsTotal.increment();
        }
    
        public void connectionOpened() {
            activeConnections.incrementAndGet();
        }
    
        public void connectionClosed() {
            activeConnections.decrementAndGet();
        }
    
        public Timer.Sample startTimer() {
            return Timer.start(registry);
        }
    
        public void stopTimer(Timer.Sample sample) {
            sample.stop(requestDuration);
        }
    }
    

    4.2 Utilisation dans le code

    @Service
    public class OrderService {
    
        @Autowired
        private SocleMetrics metrics;
    
        public Order processOrder(Order order) {
            Timer.Sample sample = metrics.startTimer();
            metrics.recordRequest();
    
            try {
                Order result = doProcess(order);
                return result;
            } catch (Exception e) {
                metrics.recordError();
                throw e;
            } finally {
                metrics.stopTimer(sample);
            }
        }
    }
    

    4.3 Métriques avec tags

    @Component
    public class WorkerMetrics {
    
        private final MeterRegistry registry;
    
        public void recordWorkerStatus(String workerName, String status) {
            Gauge.builder("socle_worker_status", () -> 1)
                .tag("worker", workerName)
                .tag("status", status)
                .register(registry);
        }
    
        public void recordProcessed(String workerName, String type) {
            Counter.builder("socle_worker_processed_total")
                .tag("worker", workerName)
                .tag("type", type)
                .register(registry)
                .increment();
        }
    }
    

    5. Endpoint Prometheus

    5.1 Accès

    curl http://localhost:8080/actuator/prometheus
    

    5.2 Sortie

    # HELP socle_workers_total Number of workers
    # TYPE socle_workers_total gauge
    socle_workers_total{application="socle-v4",environment="PROD",region="MTQ"} 5
    
    # HELP socle_workers_healthy Number of healthy workers
    # TYPE socle_workers_healthy gauge
    socle_workers_healthy{application="socle-v4",environment="PROD",region="MTQ"} 5
    
    # HELP socle_requests_total Total number of requests
    # TYPE socle_requests_total counter
    socle_requests_total{application="socle-v4",environment="PROD",region="MTQ"} 12345
    
    # HELP socle_request_duration_seconds Request duration in seconds
    # TYPE socle_request_duration_seconds summary
    socle_request_duration_seconds{application="socle-v4",quantile="0.5"} 0.05
    socle_request_duration_seconds{application="socle-v4",quantile="0.95"} 0.2
    socle_request_duration_seconds{application="socle-v4",quantile="0.99"} 0.5
    socle_request_duration_seconds_count{application="socle-v4"} 12345
    socle_request_duration_seconds_sum{application="socle-v4"} 617.25
    

    6. Prometheus Configuration

    6.1 prometheus.yml

    global:
      scrape_interval: 15s
    
    scrape_configs:
      - job_name: 'socle-v4'
        metrics_path: '/actuator/prometheus'
        static_configs:
          - targets: ['socle-app:8080']
            labels:
              app: 'socle-v4'
              env: 'prod'
    
      - job_name: 'socle-v4-kubernetes'
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
            action: keep
            regex: true
          - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
            action: replace
            target_label: __metrics_path__
            regex: (.+)
    

    6.2 Kubernetes annotations

    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/path: "/actuator/prometheus"
        prometheus.io/port: "8080"
    

    7. Grafana Dashboards

    7.1 Exemple de requêtes

    # Taux de requêtes par seconde
    rate(socle_requests_total[5m])
    
    # Taux d'erreurs
    rate(socle_errors_total[5m]) / rate(socle_requests_total[5m]) * 100
    
    # Latence P95
    histogram_quantile(0.95, rate(socle_request_duration_seconds_bucket[5m]))
    
    # Workers unhealthy
    socle_workers_unhealthy
    
    # Circuit breakers ouverts
    socle_circuit_breaker_state == 2
    
    # Queue LogForwarder
    socle_logforwarder_queue_size / socle_logforwarder_queue_capacity * 100
    

    7.2 Dashboard JSON

    {
      "title": "Socle V4 Dashboard",
      "panels": [
        {
          "title": "Request Rate",
          "type": "graph",
          "targets": [
            {
              "expr": "rate(socle_requests_total[5m])",
              "legendFormat": "{{application}}"
            }
          ]
        },
        {
          "title": "Error Rate",
          "type": "graph",
          "targets": [
            {
              "expr": "rate(socle_errors_total[5m]) / rate(socle_requests_total[5m]) * 100",
              "legendFormat": "Error %"
            }
          ]
        },
        {
          "title": "P95 Latency",
          "type": "graph",
          "targets": [
            {
              "expr": "histogram_quantile(0.95, rate(socle_request_duration_seconds_bucket[5m]))",
              "legendFormat": "P95"
            }
          ]
        },
        {
          "title": "Workers Status",
          "type": "stat",
          "targets": [
            {
              "expr": "socle_workers_healthy",
              "legendFormat": "Healthy"
            }
          ]
        }
      ]
    }
    

    8. Alerting

    8.1 Prometheus Alertmanager rules

    groups:
      - name: socle-alerts
        rules:
          - alert: SocleHighErrorRate
            expr: rate(socle_errors_total[5m]) / rate(socle_requests_total[5m]) > 0.05
            for: 5m
            labels:
              severity: warning
            annotations:
              summary: "High error rate on {{ $labels.application }}"
              description: "Error rate is {{ $value | humanizePercentage }}"
    
          - alert: SocleWorkerUnhealthy
            expr: socle_workers_unhealthy > 0
            for: 2m
            labels:
              severity: critical
            annotations:
              summary: "Unhealthy workers on {{ $labels.application }}"
              description: "{{ $value }} workers are unhealthy"
    
          - alert: SocleCircuitBreakerOpen
            expr: socle_circuit_breaker_state == 2
            for: 5m
            labels:
              severity: warning
            annotations:
              summary: "Circuit breaker {{ $labels.name }} is OPEN"
    
          - alert: SocleLogForwarderQueueHigh
            expr: socle_logforwarder_queue_size / socle_logforwarder_queue_capacity > 0.8
            for: 5m
            labels:
              severity: warning
            annotations:
              summary: "LogForwarder queue is {{ $value | humanizePercentage }} full"
    

    9. Bonnes pratiques

    DO

    • Utiliser des noms de métriques cohérents (socle_*)
    • Ajouter des tags pertinents (application, environment, region)
    • Utiliser des histogrammes pour les latences
    • Définir des alertes sur les métriques critiques
    • Documenter les métriques

    DON’T

    • Ne pas créer trop de métriques (cardinalité)
    • Ne pas utiliser de valeurs à haute cardinalité dans les tags
    • Ne pas oublier les métriques d’erreur
    • Ne pas ignorer les métriques de queue/buffer

    10. Références

  • Socle V004 – gRPC Inter-Socles

    Socle V004 – gRPC Inter-Socles

    31 – Communication gRPC Inter-Socles

    Vue d’ensemble

    Le module gRPC permet aux instances Socle V4 de communiquer entre elles via streaming bidirectionnel. Il offre :

    • Sessions : Gestion de sessions multi-participants avec TTL
    • Streaming bidirectionnel : Communication temps reel entre Socles
    • Pipeline de traitement : Transformation des messages via Janino et JDM
    • Fan-out : Routage broadcast ou cible vers les participants
    • Pool de connexions : Connexions persistantes vers les peers

    Architecture

    ┌─────────────────────────────────────────────────────────────────┐
    │                         Socle A                                  │
    │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
    │  │ SessionMgr  │  │  Pipeline   │  │    GrpcServerWorker     │  │
    │  │             │  │  Executor   │  │    (port 9400)          │  │
    │  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
    │         │               │                      │                 │
    │         └───────────────┼──────────────────────┘                 │
    │                         │                                        │
    │              ┌──────────┴──────────┐                            │
    │              │   gRPC Services     │                            │
    │              │  - SocleComm        │                            │
    │              │  - SessionService   │                            │
    │              │  - DiscoveryService │                            │
    │              └──────────┬──────────┘                            │
    └─────────────────────────┼───────────────────────────────────────┘
                              │ gRPC/HTTP2
                              ▼
    ┌─────────────────────────────────────────────────────────────────┐
    │                         Socle B                                  │
    │              ┌──────────────────────┐                           │
    │              │  PeerConnectionPool  │                           │
    │              └──────────────────────┘                           │
    └─────────────────────────────────────────────────────────────────┘
    

    Configuration

    application.yml

    socle:
      grpc:
        # Activation du module
        enabled: ${GRPC_ENABLED:false}
    
        # Port du serveur gRPC
        port: ${GRPC_PORT:9400}
    
        # Identification du Socle
        socle-id: ${SOCLE_ID:${socle.app_name}}
        socle-version: ${socle.version}
    
        # Limites serveur
        max-inbound-message-size: ${GRPC_MAX_MESSAGE_SIZE:4194304}  # 4MB
        max-concurrent-calls-per-connection: ${GRPC_MAX_CONCURRENT_CALLS:100}
    
        # Sessions
        session:
          ttl-seconds: ${GRPC_SESSION_TTL:1800}           # 30 min
          max-participants: ${GRPC_MAX_PARTICIPANTS:100}
          persist-to-tech-db: ${GRPC_PERSIST_SESSIONS:true}
          cache-in-redis: ${GRPC_CACHE_REDIS:true}
    
        # Pipeline de traitement
        pipeline:
          enabled: ${GRPC_PIPELINE_ENABLED:true}
          config-cache-ttl-seconds: ${GRPC_PIPELINE_CACHE_TTL:300}
    
        # Connexions peer
        peer:
          max-channels-per-peer: ${GRPC_PEER_MAX_CHANNELS:4}
          connection-timeout-ms: ${GRPC_PEER_CONNECT_TIMEOUT:5000}
          idle-timeout-seconds: ${GRPC_PEER_IDLE_TIMEOUT:300}
          keep-alive-enabled: ${GRPC_PEER_KEEPALIVE:true}
          keep-alive-time-seconds: ${GRPC_PEER_KEEPALIVE_TIME:30}
          keep-alive-timeout-seconds: ${GRPC_PEER_KEEPALIVE_TIMEOUT:10}
    

    Variables d’environnement

    Variable Default Description
    GRPC_ENABLED false Active le module gRPC
    GRPC_PORT 9400 Port du serveur gRPC
    SOCLE_ID ${app_name} Identifiant unique du Socle
    GRPC_SESSION_TTL 1800 TTL des sessions en secondes
    GRPC_MAX_PARTICIPANTS 100 Max participants par session
    GRPC_PIPELINE_ENABLED true Active le pipeline de traitement

    Services gRPC

    DiscoveryService

    Service de decouverte et health check.

    service DiscoveryService {
        rpc GetCapabilities(CapabilitiesRequest) returns (CapabilitiesResponse);
        rpc Ping(PingRequest) returns (PingResponse);
    }
    

    Test avec grpcurl :

    # Ping
    grpcurl -plaintext localhost:9400 socle.DiscoveryService/Ping
    
    # Capabilities
    grpcurl -plaintext localhost:9400 socle.DiscoveryService/GetCapabilities
    

    SessionService

    Gestion du cycle de vie des sessions.

    service SessionService {
        rpc CreateSession(CreateSessionRequest) returns (SessionInfo);
        rpc JoinSession(JoinSessionRequest) returns (JoinSessionResponse);
        rpc LeaveSession(LeaveSessionRequest) returns (LeaveSessionResponse);
        rpc GetSession(GetSessionRequest) returns (SessionInfo);
        rpc CloseSession(CloseSessionRequest) returns (CloseSessionResponse);
    }
    

    Exemples :

    # Creer une session
    grpcurl -plaintext -d '{
      "session_type": "chat",
      "owner_id": "user1",
      "ttl_seconds": 3600
    }' localhost:9400 socle.SessionService/CreateSession
    
    # Joindre une session
    grpcurl -plaintext -d '{
      "session_id": "uuid-de-la-session",
      "participant_id": "user2",
      "display_name": "User 2"
    }' localhost:9400 socle.SessionService/JoinSession
    
    # Obtenir info session
    grpcurl -plaintext -d '{
      "session_id": "uuid-de-la-session"
    }' localhost:9400 socle.SessionService/GetSession
    

    SocleComm

    Streaming bidirectionnel pour l’echange de messages.

    service SocleComm {
        rpc Exchange(stream SessionMessage) returns (stream SessionMessage);
    }
    

    Format des messages :

    message SessionMessage {
        string session_id = 1;
        string sender_id = 2;
        MessageKind kind = 3;        // JOIN, LEAVE, DATA, REQUEST, RESPONSE, etc.
        repeated string target_ids = 4;  // Vide = broadcast
        string correlation_id = 5;
        int64 timestamp = 6;
        string payload = 7;          // JSON
        map<string, string> headers = 8;
    }
    

    Sessions

    Cycle de vie

    CREATE ──► ACTIVE ──► CLOSING ──► CLOSED
                  │
                  └──► EXPIRED (TTL depasse)
    

    Stockage

    Les sessions sont stockees dans :

    1. Cache memoire : Acces rapide, stream observers
    2. Redis (si KvBus en mode redis) : Partage entre instances
    3. TechDB : Audit et persistance

    Tables TechDB

    -- Sessions (audit)
    CREATE TABLE grpc_sessions (
        x_id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
        x_dateCreated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
        x_dateChanged TIMESTAMP WITH TIME ZONE,
        x_sub VARCHAR(255),
        x_partition VARCHAR(30),
        x_comment CLOB,
        session_id VARCHAR(36) NOT NULL UNIQUE,
        session_type VARCHAR(100) NOT NULL,
        owner_id VARCHAR(100) NOT NULL,
        status VARCHAR(20) DEFAULT 'ACTIVE',
        datas CLOB
    );
    
    -- Configuration pipeline par type de session
    CREATE TABLE grpc_pipeline_configs (
        x_id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
        x_dateCreated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
        x_dateChanged TIMESTAMP WITH TIME ZONE,
        x_sub VARCHAR(255),
        x_partition VARCHAR(30),
        x_comment CLOB,
        session_type VARCHAR(100) NOT NULL UNIQUE,
        janino_pre VARCHAR(255),
        jdm_rules VARCHAR(255),
        janino_post VARCHAR(255),
        default_targets VARCHAR(50) DEFAULT 'broadcast',
        enabled BOOLEAN DEFAULT TRUE,
        datas CLOB
    );
    

    Pipeline de traitement

    Le pipeline permet de transformer les messages et determiner leur routage.

    Etapes

    Message entrant
          │
          ▼
    ┌─────────────────┐
    │  1. Janino PRE  │  Transformation du payload
    └────────┬────────┘
             │
             ▼
    ┌─────────────────┐
    │  2. JDM Rules   │  Determination des cibles (routing)
    └────────┬────────┘
             │
             ▼
    ┌─────────────────┐
    │  3. Janino POST │  Transformation finale
    └────────┬────────┘
             │
             ▼
        Routage
       /        \
    Broadcast   Cible
    

    Configuration du pipeline

    Inserez dans grpc_pipeline_configs :

    INSERT INTO grpc_pipeline_configs
    (session_type, janino_pre, jdm_rules, janino_post, default_targets, enabled)
    VALUES
    ('chat', NULL, 'chat_routing', NULL, 'broadcast', TRUE);
    

    Exemple script Janino PRE

    // repository/scripts/java/grpc/chat_filter.java
    public class ChatFilter {
        public Object execute(Map<String, Object> context) {
            Map<String, Object> payload = (Map<String, Object>) context.get("payload");
            String senderId = (String) context.get("senderId");
    
            // Ajouter metadata
            payload.put("processed_at", System.currentTimeMillis());
            payload.put("sender", senderId);
    
            return payload;
        }
    }
    

    Exemple modele JDM pour routing

    {
      "name": "chat_routing",
      "nodes": [
        {
          "id": "input",
          "type": "inputNode",
          "content": {
            "fields": [
              {"field": "payload.target", "type": "string"},
              {"field": "payload.type", "type": "string"}
            ]
          }
        },
        {
          "id": "decision",
          "type": "decisionTableNode",
          "content": {
            "hitPolicy": "first",
            "inputs": [
              {"field": "payload.type"}
            ],
            "outputs": [
              {"field": "targets", "type": "string"},
              {"field": "broadcast", "type": "boolean"}
            ],
            "rules": [
              {"_input": ["private"], "_output": ["${payload.target}", false]},
              {"_input": ["broadcast"], "_output": ["", true]},
              {"_input": ["*"], "_output": ["", true]}
            ]
          }
        }
      ]
    }
    

    Connexions Peer

    Ajouter un peer programmatiquement

    @Autowired
    private PeerConnectionPool peerPool;
    
    public void connectToPeer() {
        // Ajouter et connecter
        boolean connected = peerPool.addPeer("socle-b", "192.168.1.100", 9400);
    
        if (connected) {
            // Ouvrir un stream
            peerPool.openStream("socle-b", message -> {
                // Handler pour messages entrants
                System.out.println("Received: " + message.getPayload());
            });
    
            // Envoyer un message
            SessionMessage msg = SessionMessage.newBuilder()
                .setSessionId("...")
                .setSenderId("local-participant")
                .setKind(MessageKind.DATA)
                .setPayload("{\"text\":\"Hello\"}")
                .build();
    
            peerPool.sendToPeer("socle-b", msg);
        }
    }
    

    Broadcast vers tous les peers

    SessionMessage msg = SessionMessage.newBuilder()
        .setSessionId(sessionId)
        .setSenderId(myId)
        .setKind(MessageKind.DATA)
        .setPayload(jsonPayload)
        .build();
    
    int sent = peerPool.broadcast(msg, null);  // null = inclure tous
    

    Worker gRPC

    Priorites

    Methode Valeur Description
    getStartPriority() 800 Demarre apres Janino (25) et JDM (30)
    getStopPriority() 10 S’arrete tot pour drain des connexions

    Statistiques

    @Autowired
    private GrpcServerWorker grpcWorker;
    
    Map<String, Object> stats = grpcWorker.getStats();
    // {
    //   "running": true,
    //   "port": 9400,
    //   "sessions": { "cached_sessions": 5, "active_streams": 12 },
    //   "messaging": { "messages_received": 1234, "messages_sent": 5678 },
    //   "peers": { "connected_peers": 2, "total_messages_sent": 100 }
    // }
    

    Types de sessions supportes

    Type Description
    chat Communication temps reel
    sync Synchronisation de donnees
    broadcast Diffusion un-vers-plusieurs
    pipeline Traitement en chaine
    custom Type personnalise

    Securite

    TLS (a venir)

    Le support TLS sera ajoute dans une version future :

    socle:
      grpc:
        tls:
          enabled: true
          cert-path: /path/to/server.crt
          key-path: /path/to/server.key
          ca-path: /path/to/ca.crt  # Pour mTLS
    

    Authentification

    L’authentification peut etre implementee via :

    • Metadata gRPC (tokens dans headers)
    • Intercepteurs personnalises
    • Integration avec le module Auth existant

    Monitoring

    Metriques exposees

    Les metriques sont disponibles via l’endpoint Prometheus /actuator/prometheus :

    • grpc_sessions_active : Nombre de sessions actives
    • grpc_messages_received_total : Total messages recus
    • grpc_messages_sent_total : Total messages envoyes
    • grpc_peers_connected : Nombre de peers connectes

    Logs

    [grpc_server] Started on port 9400
    [grpc_session] Session created: abc-123 (type=chat, owner=user1)
    [grpc_session] Participant joined: user2 -> abc-123 (total: 2)
    [grpc_comm] Message received: session=abc-123, sender=user1, kind=DATA
    [grpc_pipeline] Executed for session abc-123 (targets=broadcast)
    

    Troubleshooting

    Le serveur ne demarre pas

    1. Verifiez que le port 9400 n’est pas utilise
    2. Verifiez GRPC_ENABLED=true
    3. Consultez les logs pour les erreurs d’initialisation

    Sessions expirent trop vite

    Augmentez le TTL :

    socle:
      grpc:
        session:
          ttl-seconds: 7200  # 2 heures
    

    Messages non delivres

    1. Verifiez que le destinataire a un stream actif
    2. Verifiez les logs du pipeline pour les erreurs
    3. Verifiez la configuration du routing dans grpc_pipeline_configs

    Erreurs de connexion peer

    1. Verifiez la connectivite reseau
    2. Verifiez que le peer a gRPC active
    3. Augmentez le timeout de connexion :
    socle:
      grpc:
        peer:
          connection-timeout-ms: 10000
    

    Structure des fichiers

    src/main/java/eu/lmvi/socle/grpc/
    ├── GrpcConfiguration.java        # Configuration Spring
    ├── GrpcServerWorker.java         # Worker principal
    ├── session/
    │   ├── Session.java              # Record session
    │   ├── Participant.java          # Record participant
    │   ├── SessionStatus.java        # Enum ACTIVE/CLOSING/CLOSED/EXPIRED
    │   ├── SessionManager.java       # Gestion sessions + streams
    │   └── SessionRedisRepository.java  # Persistance Redis
    ├── pipeline/
    │   ├── PipelineConfig.java       # Config pipeline
    │   ├── PipelineResult.java       # Resultat pipeline
    │   └── PipelineExecutor.java     # Execution Janino + JDM
    ├── peer/
    │   ├── PeerConnection.java       # Connexion unique
    │   └── PeerConnectionPool.java   # Pool avec reconnexion
    └── service/
        ├── SocleCommService.java     # Streaming bidirectionnel
        ├── SessionServiceImpl.java   # CRUD sessions
        └── DiscoveryServiceImpl.java # Discovery + Ping
    
    src/main/proto/
    └── socle_comm.proto              # Definition Protocol Buffers
    
  • Socle V004 – Métriques

    Socle V004 – Métriques

    15 – Metrics

    Version : 4.0.0 Date : 2025-12-09

    1. Introduction

    Le Socle V4 expose des métriques au format Prometheus pour le monitoring et l’alerting.

    Types de métriques

    • Counter : Valeur qui ne fait qu’augmenter (requêtes, erreurs)
    • Gauge : Valeur qui peut monter et descendre (connexions actives)
    • Histogram : Distribution de valeurs (latences)
    • Summary : Similaire à histogram avec percentiles pré-calculés

    2. Configuration

    2.1 application.yml

    management:
      endpoints:
        web:
          exposure:
            include: prometheus,health,info,metrics
          base-path: /actuator
      endpoint:
        prometheus:
          enabled: true
      metrics:
        export:
          prometheus:
            enabled: true
        tags:
          application: ${socle.app_name}
          environment: ${socle.env_name}
          region: ${socle.region}
    

    2.2 Dépendances Maven

    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-registry-prometheus</artifactId>
    </dependency>
    

    3. Métriques Socle

    3.1 Métriques Workers

    # Nombre de workers
    socle_workers_total{application="socle-v4"} 5
    
    # Workers healthy
    socle_workers_healthy{application="socle-v4"} 5
    
    # Workers unhealthy
    socle_workers_unhealthy{application="socle-v4"} 0
    
    # État par worker
    socle_worker_status{worker="kafka-consumer",status="RUNNING"} 1
    socle_worker_status{worker="order-processor",status="RUNNING"} 1
    
    # Heartbeats par worker
    socle_worker_heartbeats_total{worker="kafka-consumer"} 1234
    socle_worker_missed_heartbeats{worker="kafka-consumer"} 0
    

    3.2 Métriques KvBus

    # Opérations
    socle_kvbus_operations_total{operation="get"} 12345
    socle_kvbus_operations_total{operation="put"} 6789
    socle_kvbus_operations_total{operation="delete"} 234
    
    # Latence
    socle_kvbus_operation_duration_seconds{operation="get",quantile="0.5"} 0.001
    socle_kvbus_operation_duration_seconds{operation="get",quantile="0.95"} 0.005
    socle_kvbus_operation_duration_seconds{operation="get",quantile="0.99"} 0.01
    
    # Nombre de clés
    socle_kvbus_keys_count 456
    

    3.3 Métriques Pipeline

    # Exécutions
    socle_pipeline_executions_total{pipeline="order-processing",status="SUCCESS"} 1234
    socle_pipeline_executions_total{pipeline="order-processing",status="FAILURE"} 12
    
    # Durée
    socle_pipeline_duration_seconds{pipeline="order-processing",quantile="0.5"} 0.5
    socle_pipeline_duration_seconds{pipeline="order-processing",quantile="0.95"} 2.0
    socle_pipeline_duration_seconds{pipeline="order-processing",quantile="0.99"} 5.0
    
    # Steps
    socle_pipeline_step_duration_seconds{step="validation",quantile="0.5"} 0.01
    socle_pipeline_step_duration_seconds{step="processing",quantile="0.5"} 0.3
    

    3.4 Métriques Resilience

    # Circuit breaker état (0=CLOSED, 1=HALF_OPEN, 2=OPEN)
    socle_circuit_breaker_state{name="payment-gateway"} 0
    
    # Tentatives de retry
    socle_retry_attempts_total{operation="external-api",attempt="1",success="true"} 1000
    socle_retry_attempts_total{operation="external-api",attempt="2",success="true"} 50
    socle_retry_attempts_total{operation="external-api",attempt="3",success="false"} 5
    

    3.5 Métriques TechDB (V4)

    # Opérations
    socle_techdb_operations_total{operation="saveOffset"} 5678
    socle_techdb_operations_total{operation="getOffset"} 12345
    
    # Taille des tables
    socle_techdb_rows_count{table="socle_offsets"} 23
    socle_techdb_rows_count{table="socle_events"} 456
    socle_techdb_rows_count{table="socle_log_fallback"} 0
    

    3.6 Métriques LogForwarder (V4)

    # Queue
    socle_logforwarder_queue_size 45
    socle_logforwarder_queue_capacity 10000
    
    # Logs envoyés
    socle_logforwarder_logs_sent_total 123456
    socle_logforwarder_logs_failed_total 23
    socle_logforwarder_logs_fallback_total 0
    
    # Batches
    socle_logforwarder_batches_sent_total 1234
    socle_logforwarder_batch_size{quantile="0.5"} 100
    

    4. Implémentation

    4.1 Enregistrement des métriques

    package eu.lmvi.socle.metrics;
    
    @Component
    public class SocleMetrics {
    
        private final MeterRegistry registry;
    
        // Counters
        private final Counter requestsTotal;
        private final Counter errorsTotal;
    
        // Gauges
        private final AtomicInteger activeConnections = new AtomicInteger(0);
    
        // Timers
        private final Timer requestDuration;
    
        public SocleMetrics(MeterRegistry registry) {
            this.registry = registry;
    
            // Counter
            this.requestsTotal = Counter.builder("socle_requests_total")
                .description("Total number of requests")
                .register(registry);
    
            this.errorsTotal = Counter.builder("socle_errors_total")
                .description("Total number of errors")
                .register(registry);
    
            // Gauge
            Gauge.builder("socle_active_connections", activeConnections, AtomicInteger::get)
                .description("Number of active connections")
                .register(registry);
    
            // Timer
            this.requestDuration = Timer.builder("socle_request_duration_seconds")
                .description("Request duration in seconds")
                .publishPercentiles(0.5, 0.95, 0.99)
                .register(registry);
        }
    
        public void recordRequest() {
            requestsTotal.increment();
        }
    
        public void recordError() {
            errorsTotal.increment();
        }
    
        public void connectionOpened() {
            activeConnections.incrementAndGet();
        }
    
        public void connectionClosed() {
            activeConnections.decrementAndGet();
        }
    
        public Timer.Sample startTimer() {
            return Timer.start(registry);
        }
    
        public void stopTimer(Timer.Sample sample) {
            sample.stop(requestDuration);
        }
    }
    

    4.2 Utilisation dans le code

    @Service
    public class OrderService {
    
        @Autowired
        private SocleMetrics metrics;
    
        public Order processOrder(Order order) {
            Timer.Sample sample = metrics.startTimer();
            metrics.recordRequest();
    
            try {
                Order result = doProcess(order);
                return result;
            } catch (Exception e) {
                metrics.recordError();
                throw e;
            } finally {
                metrics.stopTimer(sample);
            }
        }
    }
    

    4.3 Métriques avec tags

    @Component
    public class WorkerMetrics {
    
        private final MeterRegistry registry;
    
        public void recordWorkerStatus(String workerName, String status) {
            Gauge.builder("socle_worker_status", () -> 1)
                .tag("worker", workerName)
                .tag("status", status)
                .register(registry);
        }
    
        public void recordProcessed(String workerName, String type) {
            Counter.builder("socle_worker_processed_total")
                .tag("worker", workerName)
                .tag("type", type)
                .register(registry)
                .increment();
        }
    }
    

    5. Endpoint Prometheus

    5.1 Accès

    curl http://localhost:8080/actuator/prometheus
    

    5.2 Sortie

    # HELP socle_workers_total Number of workers
    # TYPE socle_workers_total gauge
    socle_workers_total{application="socle-v4",environment="PROD",region="MTQ"} 5
    
    # HELP socle_workers_healthy Number of healthy workers
    # TYPE socle_workers_healthy gauge
    socle_workers_healthy{application="socle-v4",environment="PROD",region="MTQ"} 5
    
    # HELP socle_requests_total Total number of requests
    # TYPE socle_requests_total counter
    socle_requests_total{application="socle-v4",environment="PROD",region="MTQ"} 12345
    
    # HELP socle_request_duration_seconds Request duration in seconds
    # TYPE socle_request_duration_seconds summary
    socle_request_duration_seconds{application="socle-v4",quantile="0.5"} 0.05
    socle_request_duration_seconds{application="socle-v4",quantile="0.95"} 0.2
    socle_request_duration_seconds{application="socle-v4",quantile="0.99"} 0.5
    socle_request_duration_seconds_count{application="socle-v4"} 12345
    socle_request_duration_seconds_sum{application="socle-v4"} 617.25
    

    6. Prometheus Configuration

    6.1 prometheus.yml

    global:
      scrape_interval: 15s
    
    scrape_configs:
      - job_name: 'socle-v4'
        metrics_path: '/actuator/prometheus'
        static_configs:
          - targets: ['socle-app:8080']
            labels:
              app: 'socle-v4'
              env: 'prod'
    
      - job_name: 'socle-v4-kubernetes'
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
            action: keep
            regex: true
          - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
            action: replace
            target_label: __metrics_path__
            regex: (.+)
    

    6.2 Kubernetes annotations

    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/path: "/actuator/prometheus"
        prometheus.io/port: "8080"
    

    7. Grafana Dashboards

    7.1 Exemple de requêtes

    # Taux de requêtes par seconde
    rate(socle_requests_total[5m])
    
    # Taux d'erreurs
    rate(socle_errors_total[5m]) / rate(socle_requests_total[5m]) * 100
    
    # Latence P95
    histogram_quantile(0.95, rate(socle_request_duration_seconds_bucket[5m]))
    
    # Workers unhealthy
    socle_workers_unhealthy
    
    # Circuit breakers ouverts
    socle_circuit_breaker_state == 2
    
    # Queue LogForwarder
    socle_logforwarder_queue_size / socle_logforwarder_queue_capacity * 100
    

    7.2 Dashboard JSON

    {
      "title": "Socle V4 Dashboard",
      "panels": [
        {
          "title": "Request Rate",
          "type": "graph",
          "targets": [
            {
              "expr": "rate(socle_requests_total[5m])",
              "legendFormat": "{{application}}"
            }
          ]
        },
        {
          "title": "Error Rate",
          "type": "graph",
          "targets": [
            {
              "expr": "rate(socle_errors_total[5m]) / rate(socle_requests_total[5m]) * 100",
              "legendFormat": "Error %"
            }
          ]
        },
        {
          "title": "P95 Latency",
          "type": "graph",
          "targets": [
            {
              "expr": "histogram_quantile(0.95, rate(socle_request_duration_seconds_bucket[5m]))",
              "legendFormat": "P95"
            }
          ]
        },
        {
          "title": "Workers Status",
          "type": "stat",
          "targets": [
            {
              "expr": "socle_workers_healthy",
              "legendFormat": "Healthy"
            }
          ]
        }
      ]
    }
    

    8. Alerting

    8.1 Prometheus Alertmanager rules

    groups:
      - name: socle-alerts
        rules:
          - alert: SocleHighErrorRate
            expr: rate(socle_errors_total[5m]) / rate(socle_requests_total[5m]) > 0.05
            for: 5m
            labels:
              severity: warning
            annotations:
              summary: "High error rate on {{ $labels.application }}"
              description: "Error rate is {{ $value | humanizePercentage }}"
    
          - alert: SocleWorkerUnhealthy
            expr: socle_workers_unhealthy > 0
            for: 2m
            labels:
              severity: critical
            annotations:
              summary: "Unhealthy workers on {{ $labels.application }}"
              description: "{{ $value }} workers are unhealthy"
    
          - alert: SocleCircuitBreakerOpen
            expr: socle_circuit_breaker_state == 2
            for: 5m
            labels:
              severity: warning
            annotations:
              summary: "Circuit breaker {{ $labels.name }} is OPEN"
    
          - alert: SocleLogForwarderQueueHigh
            expr: socle_logforwarder_queue_size / socle_logforwarder_queue_capacity > 0.8
            for: 5m
            labels:
              severity: warning
            annotations:
              summary: "LogForwarder queue is {{ $value | humanizePercentage }} full"
    

    9. Bonnes pratiques

    DO

    • Utiliser des noms de métriques cohérents (socle_*)
    • Ajouter des tags pertinents (application, environment, region)
    • Utiliser des histogrammes pour les latences
    • Définir des alertes sur les métriques critiques
    • Documenter les métriques

    DON’T

    • Ne pas créer trop de métriques (cardinalité)
    • Ne pas utiliser de valeurs à haute cardinalité dans les tags
    • Ne pas oublier les métriques d’erreur
    • Ne pas ignorer les métriques de queue/buffer

    10. Références

  • Socle V004 – Supervisor

    Socle V004 – Supervisor

    08 – Supervisor

    Version : 4.0.0 Date : 2025-12-09

    1. Introduction

    Le Supervisor est le composant de supervision du Socle V4. Il surveille l’état de santé des Workers via heartbeats et expose des métriques de santé.

    Fonctionnalités

    • Collecte des heartbeats des Workers
    • Détection des Workers défaillants
    • Agrégation de l’état de santé global
    • Exposition via API REST et métriques
    • Intégration avec SharedDataRegistry

    2. Architecture

    ┌─────────────────────────────────────────────────────────────┐
    │                        Supervisor                            │
    │                                                              │
    │   ┌──────────────────┐    ┌──────────────────┐              │
    │   │ HeartbeatCollector│    │ HealthAggregator │              │
    │   └────────┬─────────┘    └────────┬─────────┘              │
    │            │                       │                         │
    │            ▼                       ▼                         │
    │   ┌──────────────────┐    ┌──────────────────┐              │
    │   │ Worker States    │    │ Global Health    │              │
    │   │ (per worker)     │    │ (aggregated)     │              │
    │   └──────────────────┘    └──────────────────┘              │
    │                                                              │
    └──────────────────────────────────────────────────────────────┘
             │                           │
             ▼                           ▼
    ┌─────────────────┐         ┌─────────────────┐
    │ /admin/health   │         │ /admin/workers  │
    └─────────────────┘         └─────────────────┘
    

    3. Configuration

    3.1 application.yml

    socle:
      supervisor:
        heartbeat-interval-ms: ${SUPERVISOR_HEARTBEAT_MS:10000}
        unhealthy-threshold: ${SUPERVISOR_UNHEALTHY_THRESHOLD:3}
        check-interval-ms: ${SUPERVISOR_CHECK_INTERVAL_MS:5000}
        stale-timeout-ms: ${SUPERVISOR_STALE_TIMEOUT_MS:60000}
    

    3.2 Variables d’environnement

    Variable Description Défaut
    SUPERVISOR_HEARTBEAT_MS Intervalle heartbeat attendu 10000 (10s)
    SUPERVISOR_UNHEALTHY_THRESHOLD Heartbeats manqués avant UNHEALTHY 3
    SUPERVISOR_CHECK_INTERVAL_MS Intervalle de vérification 5000 (5s)
    SUPERVISOR_STALE_TIMEOUT_MS Timeout avant worker STALE 60000 (1min)

    4. Interface Supervisor

    package eu.lmvi.socle.supervisor;
    
    public interface Supervisor {
    
        /**
         * Enregistre un worker dans le supervisor
         */
        void registerWorker(Worker worker);
    
        /**
         * Désenregistre un worker
         */
        void unregisterWorker(String workerName);
    
        /**
         * Reçoit un heartbeat d'un worker
         */
        void heartbeat(String workerName);
    
        /**
         * Reçoit un heartbeat avec métriques
         */
        void heartbeat(String workerName, Map<String, Object> metrics);
    
        /**
         * Récupère l'état d'un worker
         */
        WorkerState getWorkerState(String workerName);
    
        /**
         * Récupère l'état de tous les workers
         */
        Map<String, WorkerState> getAllWorkerStates();
    
        /**
         * Vérifie si un worker est healthy
         */
        boolean isWorkerHealthy(String workerName);
    
        /**
         * Récupère l'état de santé global
         */
        HealthStatus getGlobalHealth();
    
        /**
         * Liste les workers unhealthy
         */
        List<String> getUnhealthyWorkers();
    
        /**
         * Démarre la supervision
         */
        void start();
    
        /**
         * Arrête la supervision
         */
        void stop();
    }
    

    5. États des Workers

    5.1 WorkerState

    package eu.lmvi.socle.supervisor;
    
    public record WorkerState(
        String workerName,
        WorkerStatus status,
        Instant lastHeartbeat,
        int missedHeartbeats,
        Map<String, Object> lastMetrics,
        Instant registeredAt
    ) {
        public boolean isHealthy() {
            return status == WorkerStatus.RUNNING;
        }
    
        public boolean isStale() {
            return status == WorkerStatus.STALE;
        }
    }
    

    5.2 WorkerStatus

    public enum WorkerStatus {
        /**
         * Worker enregistré mais pas encore démarré
         */
        REGISTERED,
    
        /**
         * Worker en cours d'exécution, heartbeats reçus
         */
        RUNNING,
    
        /**
         * Heartbeats manqués mais pas encore timeout
         */
        DEGRADED,
    
        /**
         * Trop de heartbeats manqués, worker considéré unhealthy
         */
        UNHEALTHY,
    
        /**
         * Aucun heartbeat depuis longtemps, worker potentiellement mort
         */
        STALE,
    
        /**
         * Worker arrêté proprement
         */
        STOPPED
    }
    

    5.3 Diagramme d’états

                        ┌────────────────┐
                        │   REGISTERED   │
                        └───────┬────────┘
                                │ first heartbeat
                                ▼
                        ┌────────────────┐
             ┌─────────│    RUNNING     │─────────┐
             │         └───────┬────────┘         │
             │                 │                  │
             │ heartbeat       │ missed           │ stop()
             │ received        │ heartbeat        │
             │                 ▼                  │
             │         ┌────────────────┐         │
             └────────►│   DEGRADED     │         │
                       └───────┬────────┘         │
                               │ threshold        │
                               │ exceeded         │
                               ▼                  │
                       ┌────────────────┐         │
                       │   UNHEALTHY    │         │
                       └───────┬────────┘         │
                               │ stale            │
                               │ timeout          │
                               ▼                  │
                       ┌────────────────┐         │
                       │     STALE      │         │
                       └────────────────┘         │
                                                  │
                       ┌────────────────┐         │
                       │    STOPPED     │◄────────┘
                       └────────────────┘
    

    6. Implémentation

    package eu.lmvi.socle.supervisor;
    
    @Component
    public class DefaultSupervisor implements Supervisor {
    
        private static final Logger log = LoggerFactory.getLogger(DefaultSupervisor.class);
    
        private final SocleConfiguration config;
        private final ConcurrentHashMap<String, WorkerStateInternal> workers = new ConcurrentHashMap<>();
        private final ScheduledExecutorService scheduler;
        private volatile boolean running = false;
    
        public DefaultSupervisor(SocleConfiguration config) {
            this.config = config;
            this.scheduler = Executors.newSingleThreadScheduledExecutor(
                r -> new Thread(r, "supervisor-checker"));
        }
    
        @Override
        public void registerWorker(Worker worker) {
            String name = worker.getName();
            workers.put(name, new WorkerStateInternal(
                name, WorkerStatus.REGISTERED, Instant.now(), 0, Map.of(), Instant.now()));
            log.info("Worker registered: {}", name);
        }
    
        @Override
        public void unregisterWorker(String workerName) {
            WorkerStateInternal state = workers.remove(workerName);
            if (state != null) {
                log.info("Worker unregistered: {}", workerName);
            }
        }
    
        @Override
        public void heartbeat(String workerName) {
            heartbeat(workerName, Map.of());
        }
    
        @Override
        public void heartbeat(String workerName, Map<String, Object> metrics) {
            workers.computeIfPresent(workerName, (name, state) -> {
                log.debug("Heartbeat received: {}", name);
                return new WorkerStateInternal(
                    name,
                    WorkerStatus.RUNNING,
                    Instant.now(),
                    0,
                    metrics,
                    state.registeredAt
                );
            });
        }
    
        @Override
        public WorkerState getWorkerState(String workerName) {
            WorkerStateInternal internal = workers.get(workerName);
            return internal != null ? internal.toPublic() : null;
        }
    
        @Override
        public Map<String, WorkerState> getAllWorkerStates() {
            return workers.entrySet().stream()
                .collect(Collectors.toMap(
                    Map.Entry::getKey,
                    e -> e.getValue().toPublic()
                ));
        }
    
        @Override
        public boolean isWorkerHealthy(String workerName) {
            WorkerStateInternal state = workers.get(workerName);
            return state != null && state.status == WorkerStatus.RUNNING;
        }
    
        @Override
        public HealthStatus getGlobalHealth() {
            if (workers.isEmpty()) {
                return HealthStatus.HEALTHY;
            }
    
            boolean hasUnhealthy = workers.values().stream()
                .anyMatch(s -> s.status == WorkerStatus.UNHEALTHY || s.status == WorkerStatus.STALE);
    
            if (hasUnhealthy) {
                return HealthStatus.UNHEALTHY;
            }
    
            boolean hasDegraded = workers.values().stream()
                .anyMatch(s -> s.status == WorkerStatus.DEGRADED);
    
            if (hasDegraded) {
                return HealthStatus.DEGRADED;
            }
    
            return HealthStatus.HEALTHY;
        }
    
        @Override
        public List<String> getUnhealthyWorkers() {
            return workers.entrySet().stream()
                .filter(e -> e.getValue().status == WorkerStatus.UNHEALTHY
                          || e.getValue().status == WorkerStatus.STALE)
                .map(Map.Entry::getKey)
                .toList();
        }
    
        @Override
        public void start() {
            running = true;
            long checkInterval = config.getSupervisor().getCheckIntervalMs();
            scheduler.scheduleAtFixedRate(
                this::checkWorkers,
                checkInterval,
                checkInterval,
                TimeUnit.MILLISECONDS
            );
            log.info("Supervisor started with check interval: {}ms", checkInterval);
        }
    
        @Override
        public void stop() {
            running = false;
            scheduler.shutdown();
            log.info("Supervisor stopped");
        }
    
        private void checkWorkers() {
            if (!running) return;
    
            long heartbeatInterval = config.getSupervisor().getHeartbeatIntervalMs();
            int unhealthyThreshold = config.getSupervisor().getUnhealthyThreshold();
            long staleTimeout = config.getSupervisor().getStaleTimeoutMs();
    
            Instant now = Instant.now();
    
            workers.replaceAll((name, state) -> {
                if (state.status == WorkerStatus.STOPPED) {
                    return state;
                }
    
                long msSinceLastHeartbeat = Duration.between(state.lastHeartbeat, now).toMillis();
    
                // Stale check
                if (msSinceLastHeartbeat > staleTimeout) {
                    if (state.status != WorkerStatus.STALE) {
                        log.warn("Worker STALE: {} (no heartbeat for {}ms)", name, msSinceLastHeartbeat);
                    }
                    return state.withStatus(WorkerStatus.STALE);
                }
    
                // Missed heartbeat check
                int expectedHeartbeats = (int) (msSinceLastHeartbeat / heartbeatInterval);
                if (expectedHeartbeats > 0) {
                    int newMissedCount = state.missedHeartbeats + 1;
    
                    if (newMissedCount >= unhealthyThreshold) {
                        if (state.status != WorkerStatus.UNHEALTHY) {
                            log.warn("Worker UNHEALTHY: {} (missed {} heartbeats)", name, newMissedCount);
                        }
                        return state.withStatus(WorkerStatus.UNHEALTHY).withMissedCount(newMissedCount);
                    } else {
                        if (state.status == WorkerStatus.RUNNING) {
                            log.info("Worker DEGRADED: {} (missed {} heartbeats)", name, newMissedCount);
                        }
                        return state.withStatus(WorkerStatus.DEGRADED).withMissedCount(newMissedCount);
                    }
                }
    
                return state;
            });
        }
    
        private record WorkerStateInternal(
            String workerName,
            WorkerStatus status,
            Instant lastHeartbeat,
            int missedHeartbeats,
            Map<String, Object> lastMetrics,
            Instant registeredAt
        ) {
            WorkerState toPublic() {
                return new WorkerState(workerName, status, lastHeartbeat, missedHeartbeats, lastMetrics, registeredAt);
            }
    
            WorkerStateInternal withStatus(WorkerStatus newStatus) {
                return new WorkerStateInternal(workerName, newStatus, lastHeartbeat, missedHeartbeats, lastMetrics, registeredAt);
            }
    
            WorkerStateInternal withMissedCount(int newCount) {
                return new WorkerStateInternal(workerName, status, lastHeartbeat, newCount, lastMetrics, registeredAt);
            }
        }
    }
    

    7. Heartbeat depuis les Workers

    7.1 Heartbeat manuel

    @Component
    public class MyWorker implements Worker {
    
        @Autowired
        private Supervisor supervisor;
    
        @Override
        public void doWork() {
            // Envoyer heartbeat avec métriques
            supervisor.heartbeat(getName(), Map.of(
                "processed", processedCount,
                "queueSize", queue.size()
            ));
    
            // Traitement...
        }
    }
    

    7.2 Heartbeat automatique via AbstractWorker

    public abstract class AbstractWorker implements Worker {
    
        @Autowired
        private Supervisor supervisor;
    
        @Override
        public final void doWork() {
            // Heartbeat automatique
            supervisor.heartbeat(getName(), getStats());
    
            // Appel au traitement réel
            doProcess();
        }
    
        protected abstract void doProcess();
    }
    

    7.3 Heartbeat via thread dédié

    @Component
    public class LongRunningWorker implements Worker {
    
        @Autowired
        private Supervisor supervisor;
    
        private ScheduledExecutorService heartbeatExecutor;
    
        @Override
        public void start() {
            heartbeatExecutor = Executors.newSingleThreadScheduledExecutor();
            heartbeatExecutor.scheduleAtFixedRate(
                () -> supervisor.heartbeat(getName()),
                0, 10, TimeUnit.SECONDS
            );
        }
    
        @Override
        public void stop() {
            if (heartbeatExecutor != null) {
                heartbeatExecutor.shutdown();
            }
        }
    
        @Override
        public void doWork() {
            // Long processing - heartbeat handled by separate thread
            processLongTask();
        }
    }
    

    8. API REST

    8.1 Endpoints

    @RestController
    @RequestMapping("/admin")
    public class SupervisorController {
    
        @Autowired
        private Supervisor supervisor;
    
        @GetMapping("/health")
        public ResponseEntity<HealthResponse> health() {
            HealthStatus status = supervisor.getGlobalHealth();
            return ResponseEntity
                .status(status == HealthStatus.HEALTHY ? 200 : 503)
                .body(new HealthResponse(status, supervisor.getUnhealthyWorkers()));
        }
    
        @GetMapping("/workers")
        public Map<String, WorkerState> workers() {
            return supervisor.getAllWorkerStates();
        }
    
        @GetMapping("/workers/{name}")
        public ResponseEntity<WorkerState> worker(@PathVariable String name) {
            WorkerState state = supervisor.getWorkerState(name);
            return state != null
                ? ResponseEntity.ok(state)
                : ResponseEntity.notFound().build();
        }
    }
    

    8.2 Réponses

    // GET /admin/health
    {
      "status": "HEALTHY",
      "unhealthyWorkers": []
    }
    
    // GET /admin/workers
    {
      "kafka-consumer": {
        "workerName": "kafka-consumer",
        "status": "RUNNING",
        "lastHeartbeat": "2025-12-09T10:30:00Z",
        "missedHeartbeats": 0,
        "lastMetrics": {
          "processed": 12345,
          "lag": 23
        },
        "registeredAt": "2025-12-09T10:00:00Z"
      },
      "order-processor": {
        "workerName": "order-processor",
        "status": "DEGRADED",
        "lastHeartbeat": "2025-12-09T10:29:45Z",
        "missedHeartbeats": 1,
        "lastMetrics": {},
        "registeredAt": "2025-12-09T10:00:00Z"
      }
    }
    

    9. Intégration Kubernetes

    9.1 Liveness Probe

    livenessProbe:
      httpGet:
        path: /admin/health
        port: 8080
      initialDelaySeconds: 30
      periodSeconds: 10
      failureThreshold: 3
    

    9.2 Readiness Probe

    readinessProbe:
      httpGet:
        path: /admin/health
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 5
    

    9.3 Health Controller adapté

    @GetMapping("/health/live")
    public ResponseEntity<Void> live() {
        // Liveness: l'application répond
        return ResponseEntity.ok().build();
    }
    
    @GetMapping("/health/ready")
    public ResponseEntity<Void> ready() {
        // Readiness: tous les workers sont healthy
        HealthStatus status = supervisor.getGlobalHealth();
        return status == HealthStatus.HEALTHY
            ? ResponseEntity.ok().build()
            : ResponseEntity.status(503).build();
    }
    

    10. Métriques Prometheus

    @Component
    public class SupervisorMetrics {
    
        private final Supervisor supervisor;
        private final MeterRegistry registry;
    
        @PostConstruct
        public void registerMetrics() {
            Gauge.builder("socle_workers_total", supervisor,
                s -> s.getAllWorkerStates().size())
                .register(registry);
    
            Gauge.builder("socle_workers_healthy", supervisor,
                s -> s.getAllWorkerStates().values().stream()
                    .filter(WorkerState::isHealthy).count())
                .register(registry);
    
            Gauge.builder("socle_workers_unhealthy", supervisor,
                s -> s.getUnhealthyWorkers().size())
                .register(registry);
        }
    }
    

    11. Bonnes pratiques

    DO

    • Envoyer des heartbeats réguliers depuis tous les workers actifs
    • Inclure des métriques utiles dans les heartbeats
    • Configurer des timeouts adaptés à vos workers
    • Utiliser les probes Kubernetes pour la haute disponibilité

    DON’T

    • Ne pas oublier d’envoyer des heartbeats dans les workers long-running
    • Ne pas ignorer les états DEGRADED
    • Ne pas configurer des timeouts trop courts (faux positifs)
    • Ne pas bloquer l’envoi de heartbeat avec du traitement lourd

    12. Références

  • Socle V004 – Configuration

    Socle V004 – Configuration

    04 – Configuration

    Version : 4.0.0 Date : 2025-01-25

    1. Introduction

    Le Socle V4 utilise une configuration centralisée via SocleConfiguration qui charge les paramètres depuis application.yml et les variables d’environnement.

    Priorité de configuration

    1. Variables d'environnement (priorité maximale)
    2. application.yml
    3. Valeurs par défaut dans le code
    

    2. Fichier application.yml

    2.1 Configuration minimale

    socle:
      app_name: ${APP_NAME:my-app}
      env_name: ${ENV_NAME:DEV}
      exec_id: ${EXEC_ID:${socle.app_name}-${random.uuid}}
      region: ${REGION:local}
      version: ${APP_VERSION:4.0.0}
    
    spring:
      application:
        name: ${socle.app_name}
    
    server:
      port: ${HTTP_PORT:8080}
    
    logging:
      config: classpath:log4j2.xml
    

    2.2 Configuration complète

    socle:
      # === Identification ===
      app_name: ${APP_NAME:socle-v4}
      env_name: ${ENV_NAME:DEV}
      exec_id: ${EXEC_ID:${socle.app_name}-${random.uuid}}
      region: ${REGION:local}
      version: ${APP_VERSION:4.0.0}
    
      # === HTTP Server ===
      http:
        enabled: ${HTTP_ENABLED:true}
        port: ${HTTP_PORT:8080}
        context-path: ${CONTEXT_PATH:/}
    
      # === KvBus ===
      kvbus:
        mode: ${KVBUS_MODE:in_memory}
        redis:
          host: ${REDIS_HOST:localhost}
          port: ${REDIS_PORT:6379}
          password: ${REDIS_PASSWORD:}
          database: ${REDIS_DATABASE:0}
          prefix: ${REDIS_PREFIX:socle}
    
      # === Supervisor ===
      supervisor:
        heartbeat-interval-ms: ${SUPERVISOR_HEARTBEAT_MS:10000}
        unhealthy-threshold: ${SUPERVISOR_UNHEALTHY_THRESHOLD:3}
        check-interval-ms: ${SUPERVISOR_CHECK_INTERVAL_MS:5000}
        stale-timeout-ms: ${SUPERVISOR_STALE_TIMEOUT_MS:60000}
    
      # === StatusDashboard (V4) ===
      status_dashboard:
        enabled: ${STATUS_DASHBOARD_ENABLED:true}
        port: ${STATUS_DASHBOARD_PORT:9374}
        refresh_interval: ${STATUS_DASHBOARD_REFRESH:5}
    
      # === Scheduler ===
      scheduler:
        enabled: ${SCHEDULER_ENABLED:true}
        thread-pool-size: ${SCHEDULER_POOL_SIZE:4}
    
      # === Admin API ===
      admin:
        enabled: ${ADMIN_ENABLED:true}
        auth:
          enabled: ${ADMIN_AUTH_ENABLED:false}
          username: ${ADMIN_USERNAME:admin}
          password: ${ADMIN_PASSWORD:admin}
    
      # === TechDB (V4) ===
      techdb:
        enabled: ${TECHDB_ENABLED:true}
        url: jdbc:h2:file:${TECHDB_PATH:./data/socle-techdb};MODE=PostgreSQL;DB_CLOSE_DELAY=-1
        username: socle
        password: ${TECHDB_PASSWORD:socle}
        console:
          enabled: ${H2_CONSOLE_ENABLED:false}
          path: /h2-console
    
      # === Logging (V4) ===
      logging:
        forwarder:
          enabled: ${LOG_FORWARDER_ENABLED:false}
          transport-mode: ${LOG_TRANSPORT_MODE:http}
          log-hub-url: ${LOG_HUB_URL:}
          nats-url: ${NATS_URL:}
          batch-size: ${LOG_BATCH_SIZE:100}
          flush-interval-ms: ${LOG_FLUSH_INTERVAL_MS:5000}
    
      # === Auth Client (V4) ===
      auth:
        enabled: ${AUTH_ENABLED:false}
        server-url: ${AUTH_SERVER_URL:}
        source-name: ${SOURCE_NAME:${socle.app_name}}
        api-key: ${API_KEY:}
    
      # === Worker Registry (V4) ===
      worker-registry:
        enabled: ${WORKER_REGISTRY_ENABLED:false}
        server-url: ${WORKER_REGISTRY_URL:}
        heartbeat-interval-ms: ${REGISTRY_HEARTBEAT_MS:30000}
    
    spring:
      application:
        name: ${socle.app_name}
    
    server:
      port: ${socle.http.port}
    
    logging:
      config: classpath:log4j2.xml
    

    3. Variables d’environnement

    3.1 Variables essentielles

    Variable Description Défaut Obligatoire
    APP_NAME Nom de l’application socle-v4 Recommandé
    ENV_NAME Environnement (DEV/STAGING/PROD) DEV Recommandé
    REGION Région géographique local Recommandé
    HTTP_PORT Port HTTP 8080 Non

    3.2 Variables KvBus

    Variable Description Défaut
    KVBUS_MODE Mode (in_memory/redis) in_memory
    REDIS_HOST Hôte Redis localhost
    REDIS_PORT Port Redis 6379
    REDIS_PASSWORD Mot de passe Redis
    REDIS_DATABASE Database Redis 0
    REDIS_PREFIX Préfixe des clés socle

    3.3 Variables Supervisor et Dashboard

    Variable Description Défaut
    SUPERVISOR_HEARTBEAT_MS Intervalle heartbeat attendu 10000 (10s)
    SUPERVISOR_UNHEALTHY_THRESHOLD Heartbeats manqués avant UNHEALTHY 3
    SUPERVISOR_CHECK_INTERVAL_MS Fréquence de vérification 5000 (5s)
    SUPERVISOR_STALE_TIMEOUT_MS Timeout avant STALE 60000 (1min)
    STATUS_DASHBOARD_ENABLED Activer le dashboard true
    STATUS_DASHBOARD_PORT Port du dashboard 9374
    STATUS_DASHBOARD_REFRESH Rafraîchissement (secondes) 5

    3.4 Variables V4

    Variable Description Défaut
    TECHDB_ENABLED Activer H2 TechDB true
    TECHDB_PATH Chemin fichier H2 ./data/socle-techdb
    H2_CONSOLE_ENABLED Console H2 web false
    LOG_FORWARDER_ENABLED Activer LogForwarder false
    LOG_TRANSPORT_MODE Mode transport logs http
    AUTH_ENABLED Activer auth JWT false
    WORKER_REGISTRY_ENABLED Activer registry false

    4. Classe SocleConfiguration

    package eu.lmvi.socle.config;
    
    @Configuration
    @ConfigurationProperties(prefix = "socle")
    public class SocleConfiguration {
    
        // === Identification ===
        private String app_name = "socle-v4";
        private String env_name = "DEV";
        private String exec_id;
        private String region = "local";
        private String version = "4.0.0";
    
        // === HTTP ===
        private HttpConfig http = new HttpConfig();
    
        // === KvBus ===
        private KvBusConfig kvbus = new KvBusConfig();
    
        // === Supervisor ===
        private SupervisorConfig supervisor = new SupervisorConfig();
    
        // === TechDB (V4) ===
        private TechDbConfig techdb = new TechDbConfig();
    
        // === Logging (V4) ===
        private LoggingConfig logging = new LoggingConfig();
    
        // === Auth (V4) ===
        private AuthConfig auth = new AuthConfig();
    
        // === Worker Registry (V4) ===
        private WorkerRegistryConfig workerRegistry = new WorkerRegistryConfig();
    
        // Getters / Setters...
    
        @PostConstruct
        public void init() {
            if (exec_id == null || exec_id.isEmpty()) {
                exec_id = app_name + "-" + UUID.randomUUID().toString().substring(0, 8);
            }
        }
    }
    

    4.1 Sous-configurations

    public static class HttpConfig {
        private boolean enabled = true;
        private int port = 8080;
        private String contextPath = "/";
    }
    
    public static class KvBusConfig {
        private String mode = "in_memory";
        private RedisConfig redis = new RedisConfig();
    }
    
    public static class TechDbConfig {
        private boolean enabled = true;
        private String url = "jdbc:h2:file:./data/socle-techdb";
        private String username = "socle";
        private String password = "socle";
        private ConsoleConfig console = new ConsoleConfig();
    }
    
    public static class LoggingConfig {
        private ForwarderConfig forwarder = new ForwarderConfig();
    }
    
    public static class AuthConfig {
        private boolean enabled = false;
        private String serverUrl;
        private String sourceName;
        private String apiKey;
    }
    
    public static class WorkerRegistryConfig {
        private boolean enabled = false;
        private String serverUrl;
        private long heartbeatIntervalMs = 30000;
    }
    

    5. Accès à la configuration

    5.1 Injection

    @Service
    public class MonService {
    
        @Autowired
        private SocleConfiguration config;
    
        public void doSomething() {
            String appName = config.getApp_name();
            String region = config.getRegion();
            boolean techDbEnabled = config.getTechdb().isEnabled();
        }
    }
    

    5.2 Dans un Worker

    public class MonWorker implements Worker {
    
        private final SocleConfiguration config;
    
        public MonWorker(SocleConfiguration config) {
            this.config = config;
        }
    
        @Override
        public void doWork() {
            log.info("Running in region: {}", config.getRegion());
        }
    }
    

    6. Profils Spring

    6.1 Activation

    # Via variable d'environnement
    export SPRING_PROFILES_ACTIVE=prod
    
    # Via ligne de commande
    java -jar app.jar --spring.profiles.active=prod
    

    6.2 Fichiers par profil

    src/main/resources/
    ├── application.yml           # Configuration de base
    ├── application-dev.yml       # Overrides DEV
    ├── application-staging.yml   # Overrides STAGING
    └── application-prod.yml      # Overrides PROD
    

    6.3 Exemple application-prod.yml

    socle:
      env_name: PROD
      techdb:
        console:
          enabled: false
      logging:
        forwarder:
          enabled: true
      auth:
        enabled: true
      worker-registry:
        enabled: true
    
    logging:
      config: classpath:log4j2-prod.xml
    

    7. Configuration Docker

    7.1 Dockerfile

    FROM eclipse-temurin:21-jre
    
    WORKDIR /app
    
    COPY target/socle-v004-4.0.0.jar app.jar
    
    # Variables par défaut (peuvent être overridées)
    ENV APP_NAME=socle-v4
    ENV ENV_NAME=PROD
    ENV HTTP_PORT=8080
    ENV TECHDB_PATH=/app/data/techdb
    
    EXPOSE 8080
    
    ENTRYPOINT ["java", "-jar", "app.jar"]
    

    7.2 docker-compose.yml

    version: '3.8'
    
    services:
      socle-app:
        image: socle-v4:latest
        environment:
          - APP_NAME=my-service
          - ENV_NAME=PROD
          - REGION=MTQ
          - KVBUS_MODE=redis
          - REDIS_HOST=redis
          - LOG_FORWARDER_ENABLED=true
          - LOG_HUB_URL=http://loghub:8080/api/ingest-logs
        ports:
          - "8080:8080"
        volumes:
          - ./data:/app/data
        depends_on:
          - redis
    
      redis:
        image: redis:7-alpine
        ports:
          - "6379:6379"
    

    8. Configuration Kubernetes

    8.1 ConfigMap

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: socle-config
    data:
      APP_NAME: "my-service"
      ENV_NAME: "PROD"
      REGION: "MTQ"
      KVBUS_MODE: "redis"
      LOG_FORWARDER_ENABLED: "true"
    

    8.2 Secret

    apiVersion: v1
    kind: Secret
    metadata:
      name: socle-secrets
    type: Opaque
    stringData:
      REDIS_PASSWORD: "secret-password"
      API_KEY: "my-api-key"
      TECHDB_PASSWORD: "techdb-password"
    

    8.3 Deployment

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: socle-app
    spec:
      replicas: 2
      template:
        spec:
          containers:
            - name: socle
              image: socle-v4:latest
              envFrom:
                - configMapRef:
                    name: socle-config
                - secretRef:
                    name: socle-secrets
              ports:
                - containerPort: 8080
    

    9. Validation de configuration

    9.1 Validation au démarrage

    @Component
    public class ConfigurationValidator implements ApplicationListener<ApplicationReadyEvent> {
    
        @Autowired
        private SocleConfiguration config;
    
        @Override
        public void onApplicationEvent(ApplicationReadyEvent event) {
            validateRequired();
            validateConsistency();
        }
    
        private void validateRequired() {
            if (config.getApp_name() == null || config.getApp_name().isEmpty()) {
                throw new IllegalStateException("APP_NAME is required");
            }
        }
    
        private void validateConsistency() {
            if (config.getAuth().isEnabled() && config.getAuth().getApiKey() == null) {
                throw new IllegalStateException("API_KEY required when AUTH_ENABLED=true");
            }
        }
    }
    

    9.2 Endpoint de configuration

    GET /admin/config
    

    Retourne la configuration actuelle (sans les secrets).

    10. Bonnes pratiques

    DO

    • Utiliser les variables d’environnement pour les valeurs spécifiques à l’environnement
    • Définir des valeurs par défaut sensées
    • Utiliser des profils Spring pour les environnements
    • Valider la configuration au démarrage
    • Ne jamais committer de secrets

    DON’T

    • Ne pas hardcoder de valeurs dans le code
    • Ne pas mettre de mots de passe dans application.yml
    • Ne pas utiliser de valeurs par défaut dangereuses en prod

    11. Références

  • Socle V004 – Démarrage Rapide

    Socle V004 – Démarrage Rapide

    03 – Guide de Démarrage Rapide

    Version : 4.0.0 Date : 2025-01-25 Temps estimé : 5 minutes

    1. Prérequis

    • JDK 21+ installé
    • Maven 3.9+ installé
    • Git (optionnel)

    Vérification :

    java -version   # java version "21.x.x"
    mvn -version    # Maven 3.9.x
    

    2. Installation

    Option A : Cloner le projet

    git clone <repo>/socle-v004.git
    cd socle-v004
    

    Option B : Créer un nouveau projet

    mvn archetype:generate \
      -DgroupId=com.mycompany \
      -DartifactId=my-socle-app \
      -DarchetypeArtifactId=maven-archetype-quickstart \
      -DinteractiveMode=false
    
    cd my-socle-app
    # Ajouter la dépendance socle-v004 dans pom.xml
    

    3. Configuration minimale

    3.1 pom.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
             http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
    
        <groupId>eu.lmvi</groupId>
        <artifactId>socle-v004</artifactId>
        <version>4.0.0</version>
        <packaging>jar</packaging>
    
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>3.2.1</version>
        </parent>
    
        <properties>
            <java.version>21</java.version>
        </properties>
    
        <dependencies>
            <!-- Spring Boot Web (sans Logback) -->
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
                <exclusions>
                    <exclusion>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-logging</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
    
            <!-- Log4j2 -->
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-log4j2</artifactId>
            </dependency>
    
            <!-- LMAX Disruptor (AsyncLoggers) -->
            <dependency>
                <groupId>com.lmax</groupId>
                <artifactId>disruptor</artifactId>
                <version>4.0.0</version>
            </dependency>
    
            <!-- H2 Database -->
            <dependency>
                <groupId>com.h2database</groupId>
                <artifactId>h2</artifactId>
                <version>2.2.224</version>
            </dependency>
        </dependencies>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    </project>
    

    3.2 application.yml

    spring:
      application:
        name: my-socle-app
    
    socle:
      app_name: ${APP_NAME:my-socle-app}
      version: 4.0.0
      env_name: ${ENV_NAME:DEV}
      region: ${REGION:local}
    
      # HTTP
      http_port: ${HTTP_PORT:8080}
    
      # KV Bus
      kv_impl: in_memory
    
      # Supervisor
      supervisor_enabled: true
    
      # H2 TechDB
      techdb:
        enabled: true
        url: jdbc:h2:file:./data/techdb;MODE=PostgreSQL
        console:
          enabled: true
          path: /h2-console
    
    logging:
      config: classpath:log4j2.xml
    

    3.3 log4j2.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <Configuration status="WARN">
        <Appenders>
            <Console name="Console" target="SYSTEM_OUT">
                <PatternLayout pattern="%d{ISO8601} %-5level [%thread] %logger{36} - %msg%n"/>
            </Console>
        </Appenders>
        <Loggers>
            <Root level="INFO">
                <AppenderRef ref="Console"/>
            </Root>
        </Loggers>
    </Configuration>
    

    4. Créer un Worker simple

    package com.mycompany.worker;
    
    import eu.lmvi.socle.worker.Worker;
    import eu.lmvi.socle.supervisor.Supervisor;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.stereotype.Component;
    
    import java.util.Map;
    
    @Component
    public class HelloWorker implements Worker {
    
        private static final Logger log = LoggerFactory.getLogger(HelloWorker.class);
    
        @Autowired
        private Supervisor supervisor;  // Injection du Supervisor
    
        private int counter = 0;
        private volatile boolean running = false;
    
        @Override
        public String getName() {
            return "hello-worker";
        }
    
        @Override
        public void initialize() {
            log.info("HelloWorker initializing...");
        }
    
        @Override
        public void start() {
            running = true;
            log.info("HelloWorker started!");
        }
    
        @Override
        public void doWork() {
            if (!running) return;  // Vérifier le flag d'arrêt
    
            try {
                counter++;
                log.info("Hello from worker! Count: {}", counter);
    
                // IMPORTANT: Envoyer un heartbeat avec métriques
                supervisor.heartbeat(getName(), getStats());
    
            } catch (Exception e) {
                log.error("Erreur dans doWork", e);
                // NE PAS propager l'exception
            }
        }
    
        @Override
        public void stop() {
            running = false;
            log.info("HelloWorker stopped. Final count: {}", counter);
        }
    
        @Override
        public boolean isHealthy() {
            return running;
        }
    
        @Override
        public Map<String, Object> getStats() {
            return Map.of(
                "counter", counter,
                "running", running
            );
        }
    
        @Override
        public long getCycleIntervalMs() {
            return 5000; // doWork() toutes les 5 secondes
        }
    }
    

    5. Build et Run

    # Build
    mvn clean package -DskipTests
    
    # Run
    java -jar target/socle-v004-4.0.0.jar
    
    # Ou avec variables d'environnement
    APP_NAME=my-app ENV_NAME=PROD java -jar target/socle-v004-4.0.0.jar
    

    6. Vérification

    Health check

    curl http://localhost:8080/health
    

    Réponse attendue :

    {
      "status": "UP",
      "app": "my-socle-app",
      "version": "4.0.0"
    }
    

    H2 Console (dev)

    Ouvrir : http://localhost:8080/h2-console

    • JDBC URL : jdbc:h2:file:./data/techdb
    • User : socle
    • Password : socle

    Logs

    Vous devriez voir :

    INFO  [main] MainOrchestratorProcess - MOP démarré
    INFO  [main] HelloWorker - HelloWorker started!
    INFO  [scheduler-1] HelloWorker - Hello from worker! Count: 1
    INFO  [scheduler-1] HelloWorker - Hello from worker! Count: 2
    

    StatusDashboard (supervision)

    Ouvrir : http://localhost:9374/

    Ce dashboard affiche en temps réel :

    • L’état de chaque worker (RUNNING, DEGRADED, UNHEALTHY)
    • Les métriques transmises par les heartbeats
    • L’historique des changements d’état

    7. Prochaines étapes

    Document Description
    04-CONFIGURATION Configuration complète
    05-WORKERS Guide des Workers
    08-SUPERVISOR Supervision et heartbeats
    21-H2-TECHDB Base H2 TechDB
    22-LOG4J2-LOGFORWARDER Logging centralisé
    27-STATUS-DASHBOARD Dashboard de supervision
    GUIDE-METHODOLOGIQUE Bonnes pratiques

    8. Troubleshooting

    Port déjà utilisé

    # Changer le port
    HTTP_PORT=9090 java -jar target/socle-v004-4.0.0.jar
    

    H2 Console inaccessible

    Vérifier que socle.techdb.console.enabled: true dans application.yml.

    Logs non visibles

    Vérifier que log4j2.xml existe dans src/main/resources/.

  • Socle V004 – Configuration

    Socle V004 – Configuration

    04 – Configuration

    Version : 4.0.0 Date : 2025-01-25

    1. Introduction

    Le Socle V4 utilise une configuration centralisée via SocleConfiguration qui charge les paramètres depuis application.yml et les variables d’environnement.

    Priorité de configuration

    1. Variables d'environnement (priorité maximale)
    2. application.yml
    3. Valeurs par défaut dans le code
    

    2. Fichier application.yml

    2.1 Configuration minimale

    socle:
      app_name: ${APP_NAME:my-app}
      env_name: ${ENV_NAME:DEV}
      exec_id: ${EXEC_ID:${socle.app_name}-${random.uuid}}
      region: ${REGION:local}
      version: ${APP_VERSION:4.0.0}
    
    spring:
      application:
        name: ${socle.app_name}
    
    server:
      port: ${HTTP_PORT:8080}
    
    logging:
      config: classpath:log4j2.xml
    

    2.2 Configuration complète

    socle:
      # === Identification ===
      app_name: ${APP_NAME:socle-v4}
      env_name: ${ENV_NAME:DEV}
      exec_id: ${EXEC_ID:${socle.app_name}-${random.uuid}}
      region: ${REGION:local}
      version: ${APP_VERSION:4.0.0}
    
      # === HTTP Server ===
      http:
        enabled: ${HTTP_ENABLED:true}
        port: ${HTTP_PORT:8080}
        context-path: ${CONTEXT_PATH:/}
    
      # === KvBus ===
      kvbus:
        mode: ${KVBUS_MODE:in_memory}
        redis:
          host: ${REDIS_HOST:localhost}
          port: ${REDIS_PORT:6379}
          password: ${REDIS_PASSWORD:}
          database: ${REDIS_DATABASE:0}
          prefix: ${REDIS_PREFIX:socle}
    
      # === Supervisor ===
      supervisor:
        heartbeat-interval-ms: ${SUPERVISOR_HEARTBEAT_MS:10000}
        unhealthy-threshold: ${SUPERVISOR_UNHEALTHY_THRESHOLD:3}
        check-interval-ms: ${SUPERVISOR_CHECK_INTERVAL_MS:5000}
        stale-timeout-ms: ${SUPERVISOR_STALE_TIMEOUT_MS:60000}
    
      # === StatusDashboard (V4) ===
      status_dashboard:
        enabled: ${STATUS_DASHBOARD_ENABLED:true}
        port: ${STATUS_DASHBOARD_PORT:9374}
        refresh_interval: ${STATUS_DASHBOARD_REFRESH:5}
    
      # === Scheduler ===
      scheduler:
        enabled: ${SCHEDULER_ENABLED:true}
        thread-pool-size: ${SCHEDULER_POOL_SIZE:4}
    
      # === Admin API ===
      admin:
        enabled: ${ADMIN_ENABLED:true}
        auth:
          enabled: ${ADMIN_AUTH_ENABLED:false}
          username: ${ADMIN_USERNAME:admin}
          password: ${ADMIN_PASSWORD:admin}
    
      # === TechDB (V4) ===
      techdb:
        enabled: ${TECHDB_ENABLED:true}
        url: jdbc:h2:file:${TECHDB_PATH:./data/socle-techdb};MODE=PostgreSQL;DB_CLOSE_DELAY=-1
        username: socle
        password: ${TECHDB_PASSWORD:socle}
        console:
          enabled: ${H2_CONSOLE_ENABLED:false}
          path: /h2-console
    
      # === Logging (V4) ===
      logging:
        forwarder:
          enabled: ${LOG_FORWARDER_ENABLED:false}
          transport-mode: ${LOG_TRANSPORT_MODE:http}
          log-hub-url: ${LOG_HUB_URL:}
          nats-url: ${NATS_URL:}
          batch-size: ${LOG_BATCH_SIZE:100}
          flush-interval-ms: ${LOG_FLUSH_INTERVAL_MS:5000}
    
      # === Auth Client (V4) ===
      auth:
        enabled: ${AUTH_ENABLED:false}
        server-url: ${AUTH_SERVER_URL:}
        source-name: ${SOURCE_NAME:${socle.app_name}}
        api-key: ${API_KEY:}
    
      # === Worker Registry (V4) ===
      worker-registry:
        enabled: ${WORKER_REGISTRY_ENABLED:false}
        server-url: ${WORKER_REGISTRY_URL:}
        heartbeat-interval-ms: ${REGISTRY_HEARTBEAT_MS:30000}
    
    spring:
      application:
        name: ${socle.app_name}
    
    server:
      port: ${socle.http.port}
    
    logging:
      config: classpath:log4j2.xml
    

    3. Variables d’environnement

    3.1 Variables essentielles

    Variable Description Défaut Obligatoire
    APP_NAME Nom de l’application socle-v4 Recommandé
    ENV_NAME Environnement (DEV/STAGING/PROD) DEV Recommandé
    REGION Région géographique local Recommandé
    HTTP_PORT Port HTTP 8080 Non

    3.2 Variables KvBus

    Variable Description Défaut
    KVBUS_MODE Mode (in_memory/redis) in_memory
    REDIS_HOST Hôte Redis localhost
    REDIS_PORT Port Redis 6379
    REDIS_PASSWORD Mot de passe Redis
    REDIS_DATABASE Database Redis 0
    REDIS_PREFIX Préfixe des clés socle

    3.3 Variables Supervisor et Dashboard

    Variable Description Défaut
    SUPERVISOR_HEARTBEAT_MS Intervalle heartbeat attendu 10000 (10s)
    SUPERVISOR_UNHEALTHY_THRESHOLD Heartbeats manqués avant UNHEALTHY 3
    SUPERVISOR_CHECK_INTERVAL_MS Fréquence de vérification 5000 (5s)
    SUPERVISOR_STALE_TIMEOUT_MS Timeout avant STALE 60000 (1min)
    STATUS_DASHBOARD_ENABLED Activer le dashboard true
    STATUS_DASHBOARD_PORT Port du dashboard 9374
    STATUS_DASHBOARD_REFRESH Rafraîchissement (secondes) 5

    3.4 Variables V4

    Variable Description Défaut
    TECHDB_ENABLED Activer H2 TechDB true
    TECHDB_PATH Chemin fichier H2 ./data/socle-techdb
    H2_CONSOLE_ENABLED Console H2 web false
    LOG_FORWARDER_ENABLED Activer LogForwarder false
    LOG_TRANSPORT_MODE Mode transport logs http
    AUTH_ENABLED Activer auth JWT false
    WORKER_REGISTRY_ENABLED Activer registry false

    4. Classe SocleConfiguration

    package eu.lmvi.socle.config;
    
    @Configuration
    @ConfigurationProperties(prefix = "socle")
    public class SocleConfiguration {
    
        // === Identification ===
        private String app_name = "socle-v4";
        private String env_name = "DEV";
        private String exec_id;
        private String region = "local";
        private String version = "4.0.0";
    
        // === HTTP ===
        private HttpConfig http = new HttpConfig();
    
        // === KvBus ===
        private KvBusConfig kvbus = new KvBusConfig();
    
        // === Supervisor ===
        private SupervisorConfig supervisor = new SupervisorConfig();
    
        // === TechDB (V4) ===
        private TechDbConfig techdb = new TechDbConfig();
    
        // === Logging (V4) ===
        private LoggingConfig logging = new LoggingConfig();
    
        // === Auth (V4) ===
        private AuthConfig auth = new AuthConfig();
    
        // === Worker Registry (V4) ===
        private WorkerRegistryConfig workerRegistry = new WorkerRegistryConfig();
    
        // Getters / Setters...
    
        @PostConstruct
        public void init() {
            if (exec_id == null || exec_id.isEmpty()) {
                exec_id = app_name + "-" + UUID.randomUUID().toString().substring(0, 8);
            }
        }
    }
    

    4.1 Sous-configurations

    public static class HttpConfig {
        private boolean enabled = true;
        private int port = 8080;
        private String contextPath = "/";
    }
    
    public static class KvBusConfig {
        private String mode = "in_memory";
        private RedisConfig redis = new RedisConfig();
    }
    
    public static class TechDbConfig {
        private boolean enabled = true;
        private String url = "jdbc:h2:file:./data/socle-techdb";
        private String username = "socle";
        private String password = "socle";
        private ConsoleConfig console = new ConsoleConfig();
    }
    
    public static class LoggingConfig {
        private ForwarderConfig forwarder = new ForwarderConfig();
    }
    
    public static class AuthConfig {
        private boolean enabled = false;
        private String serverUrl;
        private String sourceName;
        private String apiKey;
    }
    
    public static class WorkerRegistryConfig {
        private boolean enabled = false;
        private String serverUrl;
        private long heartbeatIntervalMs = 30000;
    }
    

    5. Accès à la configuration

    5.1 Injection

    @Service
    public class MonService {
    
        @Autowired
        private SocleConfiguration config;
    
        public void doSomething() {
            String appName = config.getApp_name();
            String region = config.getRegion();
            boolean techDbEnabled = config.getTechdb().isEnabled();
        }
    }
    

    5.2 Dans un Worker

    public class MonWorker implements Worker {
    
        private final SocleConfiguration config;
    
        public MonWorker(SocleConfiguration config) {
            this.config = config;
        }
    
        @Override
        public void doWork() {
            log.info("Running in region: {}", config.getRegion());
        }
    }
    

    6. Profils Spring

    6.1 Activation

    # Via variable d'environnement
    export SPRING_PROFILES_ACTIVE=prod
    
    # Via ligne de commande
    java -jar app.jar --spring.profiles.active=prod
    

    6.2 Fichiers par profil

    src/main/resources/
    ├── application.yml           # Configuration de base
    ├── application-dev.yml       # Overrides DEV
    ├── application-staging.yml   # Overrides STAGING
    └── application-prod.yml      # Overrides PROD
    

    6.3 Exemple application-prod.yml

    socle:
      env_name: PROD
      techdb:
        console:
          enabled: false
      logging:
        forwarder:
          enabled: true
      auth:
        enabled: true
      worker-registry:
        enabled: true
    
    logging:
      config: classpath:log4j2-prod.xml
    

    7. Configuration Docker

    7.1 Dockerfile

    FROM eclipse-temurin:21-jre
    
    WORKDIR /app
    
    COPY target/socle-v004-4.0.0.jar app.jar
    
    # Variables par défaut (peuvent être overridées)
    ENV APP_NAME=socle-v4
    ENV ENV_NAME=PROD
    ENV HTTP_PORT=8080
    ENV TECHDB_PATH=/app/data/techdb
    
    EXPOSE 8080
    
    ENTRYPOINT ["java", "-jar", "app.jar"]
    

    7.2 docker-compose.yml

    version: '3.8'
    
    services:
      socle-app:
        image: socle-v4:latest
        environment:
          - APP_NAME=my-service
          - ENV_NAME=PROD
          - REGION=MTQ
          - KVBUS_MODE=redis
          - REDIS_HOST=redis
          - LOG_FORWARDER_ENABLED=true
          - LOG_HUB_URL=http://loghub:8080/api/ingest-logs
        ports:
          - "8080:8080"
        volumes:
          - ./data:/app/data
        depends_on:
          - redis
    
      redis:
        image: redis:7-alpine
        ports:
          - "6379:6379"
    

    8. Configuration Kubernetes

    8.1 ConfigMap

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: socle-config
    data:
      APP_NAME: "my-service"
      ENV_NAME: "PROD"
      REGION: "MTQ"
      KVBUS_MODE: "redis"
      LOG_FORWARDER_ENABLED: "true"
    

    8.2 Secret

    apiVersion: v1
    kind: Secret
    metadata:
      name: socle-secrets
    type: Opaque
    stringData:
      REDIS_PASSWORD: "secret-password"
      API_KEY: "my-api-key"
      TECHDB_PASSWORD: "techdb-password"
    

    8.3 Deployment

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: socle-app
    spec:
      replicas: 2
      template:
        spec:
          containers:
            - name: socle
              image: socle-v4:latest
              envFrom:
                - configMapRef:
                    name: socle-config
                - secretRef:
                    name: socle-secrets
              ports:
                - containerPort: 8080
    

    9. Validation de configuration

    9.1 Validation au démarrage

    @Component
    public class ConfigurationValidator implements ApplicationListener<ApplicationReadyEvent> {
    
        @Autowired
        private SocleConfiguration config;
    
        @Override
        public void onApplicationEvent(ApplicationReadyEvent event) {
            validateRequired();
            validateConsistency();
        }
    
        private void validateRequired() {
            if (config.getApp_name() == null || config.getApp_name().isEmpty()) {
                throw new IllegalStateException("APP_NAME is required");
            }
        }
    
        private void validateConsistency() {
            if (config.getAuth().isEnabled() && config.getAuth().getApiKey() == null) {
                throw new IllegalStateException("API_KEY required when AUTH_ENABLED=true");
            }
        }
    }
    

    9.2 Endpoint de configuration

    GET /admin/config
    

    Retourne la configuration actuelle (sans les secrets).

    10. Bonnes pratiques

    DO

    • Utiliser les variables d’environnement pour les valeurs spécifiques à l’environnement
    • Définir des valeurs par défaut sensées
    • Utiliser des profils Spring pour les environnements
    • Valider la configuration au démarrage
    • Ne jamais committer de secrets

    DON’T

    • Ne pas hardcoder de valeurs dans le code
    • Ne pas mettre de mots de passe dans application.yml
    • Ne pas utiliser de valeurs par défaut dangereuses en prod

    11. Références

  • Socle V004 – Démarrage Rapide

    Socle V004 – Démarrage Rapide

    03 – Guide de Démarrage Rapide

    Version : 4.0.0 Date : 2025-01-25 Temps estimé : 5 minutes

    1. Prérequis

    • JDK 21+ installé
    • Maven 3.9+ installé
    • Git (optionnel)

    Vérification :

    java -version   # java version "21.x.x"
    mvn -version    # Maven 3.9.x
    

    2. Installation

    Option A : Cloner le projet

    git clone <repo>/socle-v004.git
    cd socle-v004
    

    Option B : Créer un nouveau projet

    mvn archetype:generate \
      -DgroupId=com.mycompany \
      -DartifactId=my-socle-app \
      -DarchetypeArtifactId=maven-archetype-quickstart \
      -DinteractiveMode=false
    
    cd my-socle-app
    # Ajouter la dépendance socle-v004 dans pom.xml
    

    3. Configuration minimale

    3.1 pom.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
             http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
    
        <groupId>eu.lmvi</groupId>
        <artifactId>socle-v004</artifactId>
        <version>4.0.0</version>
        <packaging>jar</packaging>
    
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>3.2.1</version>
        </parent>
    
        <properties>
            <java.version>21</java.version>
        </properties>
    
        <dependencies>
            <!-- Spring Boot Web (sans Logback) -->
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-web</artifactId>
                <exclusions>
                    <exclusion>
                        <groupId>org.springframework.boot</groupId>
                        <artifactId>spring-boot-starter-logging</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
    
            <!-- Log4j2 -->
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-log4j2</artifactId>
            </dependency>
    
            <!-- LMAX Disruptor (AsyncLoggers) -->
            <dependency>
                <groupId>com.lmax</groupId>
                <artifactId>disruptor</artifactId>
                <version>4.0.0</version>
            </dependency>
    
            <!-- H2 Database -->
            <dependency>
                <groupId>com.h2database</groupId>
                <artifactId>h2</artifactId>
                <version>2.2.224</version>
            </dependency>
        </dependencies>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    </project>
    

    3.2 application.yml

    spring:
      application:
        name: my-socle-app
    
    socle:
      app_name: ${APP_NAME:my-socle-app}
      version: 4.0.0
      env_name: ${ENV_NAME:DEV}
      region: ${REGION:local}
    
      # HTTP
      http_port: ${HTTP_PORT:8080}
    
      # KV Bus
      kv_impl: in_memory
    
      # Supervisor
      supervisor_enabled: true
    
      # H2 TechDB
      techdb:
        enabled: true
        url: jdbc:h2:file:./data/techdb;MODE=PostgreSQL
        console:
          enabled: true
          path: /h2-console
    
    logging:
      config: classpath:log4j2.xml
    

    3.3 log4j2.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <Configuration status="WARN">
        <Appenders>
            <Console name="Console" target="SYSTEM_OUT">
                <PatternLayout pattern="%d{ISO8601} %-5level [%thread] %logger{36} - %msg%n"/>
            </Console>
        </Appenders>
        <Loggers>
            <Root level="INFO">
                <AppenderRef ref="Console"/>
            </Root>
        </Loggers>
    </Configuration>
    

    4. Créer un Worker simple

    package com.mycompany.worker;
    
    import eu.lmvi.socle.worker.Worker;
    import eu.lmvi.socle.supervisor.Supervisor;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.stereotype.Component;
    
    import java.util.Map;
    
    @Component
    public class HelloWorker implements Worker {
    
        private static final Logger log = LoggerFactory.getLogger(HelloWorker.class);
    
        @Autowired
        private Supervisor supervisor;  // Injection du Supervisor
    
        private int counter = 0;
        private volatile boolean running = false;
    
        @Override
        public String getName() {
            return "hello-worker";
        }
    
        @Override
        public void initialize() {
            log.info("HelloWorker initializing...");
        }
    
        @Override
        public void start() {
            running = true;
            log.info("HelloWorker started!");
        }
    
        @Override
        public void doWork() {
            if (!running) return;  // Vérifier le flag d'arrêt
    
            try {
                counter++;
                log.info("Hello from worker! Count: {}", counter);
    
                // IMPORTANT: Envoyer un heartbeat avec métriques
                supervisor.heartbeat(getName(), getStats());
    
            } catch (Exception e) {
                log.error("Erreur dans doWork", e);
                // NE PAS propager l'exception
            }
        }
    
        @Override
        public void stop() {
            running = false;
            log.info("HelloWorker stopped. Final count: {}", counter);
        }
    
        @Override
        public boolean isHealthy() {
            return running;
        }
    
        @Override
        public Map<String, Object> getStats() {
            return Map.of(
                "counter", counter,
                "running", running
            );
        }
    
        @Override
        public long getCycleIntervalMs() {
            return 5000; // doWork() toutes les 5 secondes
        }
    }
    

    5. Build et Run

    # Build
    mvn clean package -DskipTests
    
    # Run
    java -jar target/socle-v004-4.0.0.jar
    
    # Ou avec variables d'environnement
    APP_NAME=my-app ENV_NAME=PROD java -jar target/socle-v004-4.0.0.jar
    

    6. Vérification

    Health check

    curl http://localhost:8080/health
    

    Réponse attendue :

    {
      "status": "UP",
      "app": "my-socle-app",
      "version": "4.0.0"
    }
    

    H2 Console (dev)

    Ouvrir : http://localhost:8080/h2-console

    • JDBC URL : jdbc:h2:file:./data/techdb
    • User : socle
    • Password : socle

    Logs

    Vous devriez voir :

    INFO  [main] MainOrchestratorProcess - MOP démarré
    INFO  [main] HelloWorker - HelloWorker started!
    INFO  [scheduler-1] HelloWorker - Hello from worker! Count: 1
    INFO  [scheduler-1] HelloWorker - Hello from worker! Count: 2
    

    StatusDashboard (supervision)

    Ouvrir : http://localhost:9374/

    Ce dashboard affiche en temps réel :

    • L’état de chaque worker (RUNNING, DEGRADED, UNHEALTHY)
    • Les métriques transmises par les heartbeats
    • L’historique des changements d’état

    7. Prochaines étapes

    Document Description
    04-CONFIGURATION Configuration complète
    05-WORKERS Guide des Workers
    08-SUPERVISOR Supervision et heartbeats
    21-H2-TECHDB Base H2 TechDB
    22-LOG4J2-LOGFORWARDER Logging centralisé
    27-STATUS-DASHBOARD Dashboard de supervision
    GUIDE-METHODOLOGIQUE Bonnes pratiques

    8. Troubleshooting

    Port déjà utilisé

    # Changer le port
    HTTP_PORT=9090 java -jar target/socle-v004-4.0.0.jar
    

    H2 Console inaccessible

    Vérifier que socle.techdb.console.enabled: true dans application.yml.

    Logs non visibles

    Vérifier que log4j2.xml existe dans src/main/resources/.

  • Socle V004 – Configuration

    Socle V004 – Configuration

    04 – Configuration

    Version : 4.0.0 Date : 2025-01-25

    1. Introduction

    Le Socle V4 utilise une configuration centralisée via SocleConfiguration qui charge les paramètres depuis application.yml et les variables d’environnement.

    Priorité de configuration

    1. Variables d'environnement (priorité maximale)
    2. application.yml
    3. Valeurs par défaut dans le code
    

    2. Fichier application.yml

    2.1 Configuration minimale

    socle:
      app_name: ${APP_NAME:my-app}
      env_name: ${ENV_NAME:DEV}
      exec_id: ${EXEC_ID:${socle.app_name}-${random.uuid}}
      region: ${REGION:local}
      version: ${APP_VERSION:4.0.0}
    
    spring:
      application:
        name: ${socle.app_name}
    
    server:
      port: ${HTTP_PORT:8080}
    
    logging:
      config: classpath:log4j2.xml
    

    2.2 Configuration complète

    socle:
      # === Identification ===
      app_name: ${APP_NAME:socle-v4}
      env_name: ${ENV_NAME:DEV}
      exec_id: ${EXEC_ID:${socle.app_name}-${random.uuid}}
      region: ${REGION:local}
      version: ${APP_VERSION:4.0.0}
    
      # === HTTP Server ===
      http:
        enabled: ${HTTP_ENABLED:true}
        port: ${HTTP_PORT:8080}
        context-path: ${CONTEXT_PATH:/}
    
      # === KvBus ===
      kvbus:
        mode: ${KVBUS_MODE:in_memory}
        redis:
          host: ${REDIS_HOST:localhost}
          port: ${REDIS_PORT:6379}
          password: ${REDIS_PASSWORD:}
          database: ${REDIS_DATABASE:0}
          prefix: ${REDIS_PREFIX:socle}
    
      # === Supervisor ===
      supervisor:
        heartbeat-interval-ms: ${SUPERVISOR_HEARTBEAT_MS:10000}
        unhealthy-threshold: ${SUPERVISOR_UNHEALTHY_THRESHOLD:3}
        check-interval-ms: ${SUPERVISOR_CHECK_INTERVAL_MS:5000}
        stale-timeout-ms: ${SUPERVISOR_STALE_TIMEOUT_MS:60000}
    
      # === StatusDashboard (V4) ===
      status_dashboard:
        enabled: ${STATUS_DASHBOARD_ENABLED:true}
        port: ${STATUS_DASHBOARD_PORT:9374}
        refresh_interval: ${STATUS_DASHBOARD_REFRESH:5}
    
      # === Scheduler ===
      scheduler:
        enabled: ${SCHEDULER_ENABLED:true}
        thread-pool-size: ${SCHEDULER_POOL_SIZE:4}
    
      # === Admin API ===
      admin:
        enabled: ${ADMIN_ENABLED:true}
        auth:
          enabled: ${ADMIN_AUTH_ENABLED:false}
          username: ${ADMIN_USERNAME:admin}
          password: ${ADMIN_PASSWORD:admin}
    
      # === TechDB (V4) ===
      techdb:
        enabled: ${TECHDB_ENABLED:true}
        url: jdbc:h2:file:${TECHDB_PATH:./data/socle-techdb};MODE=PostgreSQL;DB_CLOSE_DELAY=-1
        username: socle
        password: ${TECHDB_PASSWORD:socle}
        console:
          enabled: ${H2_CONSOLE_ENABLED:false}
          path: /h2-console
    
      # === Logging (V4) ===
      logging:
        forwarder:
          enabled: ${LOG_FORWARDER_ENABLED:false}
          transport-mode: ${LOG_TRANSPORT_MODE:http}
          log-hub-url: ${LOG_HUB_URL:}
          nats-url: ${NATS_URL:}
          batch-size: ${LOG_BATCH_SIZE:100}
          flush-interval-ms: ${LOG_FLUSH_INTERVAL_MS:5000}
    
      # === Auth Client (V4) ===
      auth:
        enabled: ${AUTH_ENABLED:false}
        server-url: ${AUTH_SERVER_URL:}
        source-name: ${SOURCE_NAME:${socle.app_name}}
        api-key: ${API_KEY:}
    
      # === Worker Registry (V4) ===
      worker-registry:
        enabled: ${WORKER_REGISTRY_ENABLED:false}
        server-url: ${WORKER_REGISTRY_URL:}
        heartbeat-interval-ms: ${REGISTRY_HEARTBEAT_MS:30000}
    
    spring:
      application:
        name: ${socle.app_name}
    
    server:
      port: ${socle.http.port}
    
    logging:
      config: classpath:log4j2.xml
    

    3. Variables d’environnement

    3.1 Variables essentielles

    Variable Description Défaut Obligatoire
    APP_NAME Nom de l’application socle-v4 Recommandé
    ENV_NAME Environnement (DEV/STAGING/PROD) DEV Recommandé
    REGION Région géographique local Recommandé
    HTTP_PORT Port HTTP 8080 Non

    3.2 Variables KvBus

    Variable Description Défaut
    KVBUS_MODE Mode (in_memory/redis) in_memory
    REDIS_HOST Hôte Redis localhost
    REDIS_PORT Port Redis 6379
    REDIS_PASSWORD Mot de passe Redis
    REDIS_DATABASE Database Redis 0
    REDIS_PREFIX Préfixe des clés socle

    3.3 Variables Supervisor et Dashboard

    Variable Description Défaut
    SUPERVISOR_HEARTBEAT_MS Intervalle heartbeat attendu 10000 (10s)
    SUPERVISOR_UNHEALTHY_THRESHOLD Heartbeats manqués avant UNHEALTHY 3
    SUPERVISOR_CHECK_INTERVAL_MS Fréquence de vérification 5000 (5s)
    SUPERVISOR_STALE_TIMEOUT_MS Timeout avant STALE 60000 (1min)
    STATUS_DASHBOARD_ENABLED Activer le dashboard true
    STATUS_DASHBOARD_PORT Port du dashboard 9374
    STATUS_DASHBOARD_REFRESH Rafraîchissement (secondes) 5

    3.4 Variables V4

    Variable Description Défaut
    TECHDB_ENABLED Activer H2 TechDB true
    TECHDB_PATH Chemin fichier H2 ./data/socle-techdb
    H2_CONSOLE_ENABLED Console H2 web false
    LOG_FORWARDER_ENABLED Activer LogForwarder false
    LOG_TRANSPORT_MODE Mode transport logs http
    AUTH_ENABLED Activer auth JWT false
    WORKER_REGISTRY_ENABLED Activer registry false

    4. Classe SocleConfiguration

    package eu.lmvi.socle.config;
    
    @Configuration
    @ConfigurationProperties(prefix = "socle")
    public class SocleConfiguration {
    
        // === Identification ===
        private String app_name = "socle-v4";
        private String env_name = "DEV";
        private String exec_id;
        private String region = "local";
        private String version = "4.0.0";
    
        // === HTTP ===
        private HttpConfig http = new HttpConfig();
    
        // === KvBus ===
        private KvBusConfig kvbus = new KvBusConfig();
    
        // === Supervisor ===
        private SupervisorConfig supervisor = new SupervisorConfig();
    
        // === TechDB (V4) ===
        private TechDbConfig techdb = new TechDbConfig();
    
        // === Logging (V4) ===
        private LoggingConfig logging = new LoggingConfig();
    
        // === Auth (V4) ===
        private AuthConfig auth = new AuthConfig();
    
        // === Worker Registry (V4) ===
        private WorkerRegistryConfig workerRegistry = new WorkerRegistryConfig();
    
        // Getters / Setters...
    
        @PostConstruct
        public void init() {
            if (exec_id == null || exec_id.isEmpty()) {
                exec_id = app_name + "-" + UUID.randomUUID().toString().substring(0, 8);
            }
        }
    }
    

    4.1 Sous-configurations

    public static class HttpConfig {
        private boolean enabled = true;
        private int port = 8080;
        private String contextPath = "/";
    }
    
    public static class KvBusConfig {
        private String mode = "in_memory";
        private RedisConfig redis = new RedisConfig();
    }
    
    public static class TechDbConfig {
        private boolean enabled = true;
        private String url = "jdbc:h2:file:./data/socle-techdb";
        private String username = "socle";
        private String password = "socle";
        private ConsoleConfig console = new ConsoleConfig();
    }
    
    public static class LoggingConfig {
        private ForwarderConfig forwarder = new ForwarderConfig();
    }
    
    public static class AuthConfig {
        private boolean enabled = false;
        private String serverUrl;
        private String sourceName;
        private String apiKey;
    }
    
    public static class WorkerRegistryConfig {
        private boolean enabled = false;
        private String serverUrl;
        private long heartbeatIntervalMs = 30000;
    }
    

    5. Accès à la configuration

    5.1 Injection

    @Service
    public class MonService {
    
        @Autowired
        private SocleConfiguration config;
    
        public void doSomething() {
            String appName = config.getApp_name();
            String region = config.getRegion();
            boolean techDbEnabled = config.getTechdb().isEnabled();
        }
    }
    

    5.2 Dans un Worker

    public class MonWorker implements Worker {
    
        private final SocleConfiguration config;
    
        public MonWorker(SocleConfiguration config) {
            this.config = config;
        }
    
        @Override
        public void doWork() {
            log.info("Running in region: {}", config.getRegion());
        }
    }
    

    6. Profils Spring

    6.1 Activation

    # Via variable d'environnement
    export SPRING_PROFILES_ACTIVE=prod
    
    # Via ligne de commande
    java -jar app.jar --spring.profiles.active=prod
    

    6.2 Fichiers par profil

    src/main/resources/
    ├── application.yml           # Configuration de base
    ├── application-dev.yml       # Overrides DEV
    ├── application-staging.yml   # Overrides STAGING
    └── application-prod.yml      # Overrides PROD
    

    6.3 Exemple application-prod.yml

    socle:
      env_name: PROD
      techdb:
        console:
          enabled: false
      logging:
        forwarder:
          enabled: true
      auth:
        enabled: true
      worker-registry:
        enabled: true
    
    logging:
      config: classpath:log4j2-prod.xml
    

    7. Configuration Docker

    7.1 Dockerfile

    FROM eclipse-temurin:21-jre
    
    WORKDIR /app
    
    COPY target/socle-v004-4.0.0.jar app.jar
    
    # Variables par défaut (peuvent être overridées)
    ENV APP_NAME=socle-v4
    ENV ENV_NAME=PROD
    ENV HTTP_PORT=8080
    ENV TECHDB_PATH=/app/data/techdb
    
    EXPOSE 8080
    
    ENTRYPOINT ["java", "-jar", "app.jar"]
    

    7.2 docker-compose.yml

    version: '3.8'
    
    services:
      socle-app:
        image: socle-v4:latest
        environment:
          - APP_NAME=my-service
          - ENV_NAME=PROD
          - REGION=MTQ
          - KVBUS_MODE=redis
          - REDIS_HOST=redis
          - LOG_FORWARDER_ENABLED=true
          - LOG_HUB_URL=http://loghub:8080/api/ingest-logs
        ports:
          - "8080:8080"
        volumes:
          - ./data:/app/data
        depends_on:
          - redis
    
      redis:
        image: redis:7-alpine
        ports:
          - "6379:6379"
    

    8. Configuration Kubernetes

    8.1 ConfigMap

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: socle-config
    data:
      APP_NAME: "my-service"
      ENV_NAME: "PROD"
      REGION: "MTQ"
      KVBUS_MODE: "redis"
      LOG_FORWARDER_ENABLED: "true"
    

    8.2 Secret

    apiVersion: v1
    kind: Secret
    metadata:
      name: socle-secrets
    type: Opaque
    stringData:
      REDIS_PASSWORD: "secret-password"
      API_KEY: "my-api-key"
      TECHDB_PASSWORD: "techdb-password"
    

    8.3 Deployment

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: socle-app
    spec:
      replicas: 2
      template:
        spec:
          containers:
            - name: socle
              image: socle-v4:latest
              envFrom:
                - configMapRef:
                    name: socle-config
                - secretRef:
                    name: socle-secrets
              ports:
                - containerPort: 8080
    

    9. Validation de configuration

    9.1 Validation au démarrage

    @Component
    public class ConfigurationValidator implements ApplicationListener<ApplicationReadyEvent> {
    
        @Autowired
        private SocleConfiguration config;
    
        @Override
        public void onApplicationEvent(ApplicationReadyEvent event) {
            validateRequired();
            validateConsistency();
        }
    
        private void validateRequired() {
            if (config.getApp_name() == null || config.getApp_name().isEmpty()) {
                throw new IllegalStateException("APP_NAME is required");
            }
        }
    
        private void validateConsistency() {
            if (config.getAuth().isEnabled() && config.getAuth().getApiKey() == null) {
                throw new IllegalStateException("API_KEY required when AUTH_ENABLED=true");
            }
        }
    }
    

    9.2 Endpoint de configuration

    GET /admin/config
    

    Retourne la configuration actuelle (sans les secrets).

    10. Bonnes pratiques

    DO

    • Utiliser les variables d’environnement pour les valeurs spécifiques à l’environnement
    • Définir des valeurs par défaut sensées
    • Utiliser des profils Spring pour les environnements
    • Valider la configuration au démarrage
    • Ne jamais committer de secrets

    DON’T

    • Ne pas hardcoder de valeurs dans le code
    • Ne pas mettre de mots de passe dans application.yml
    • Ne pas utiliser de valeurs par défaut dangereuses en prod

    11. Références

  • Socle V004 – gRPC Inter-Socles

    Socle V004 – gRPC Inter-Socles

    31 – Communication gRPC Inter-Socles

    Vue d’ensemble

    Le module gRPC permet aux instances Socle V4 de communiquer entre elles via streaming bidirectionnel. Il offre :

    • Sessions : Gestion de sessions multi-participants avec TTL
    • Streaming bidirectionnel : Communication temps reel entre Socles
    • Pipeline de traitement : Transformation des messages via Janino et JDM
    • Fan-out : Routage broadcast ou cible vers les participants
    • Pool de connexions : Connexions persistantes vers les peers

    Architecture

    ┌─────────────────────────────────────────────────────────────────┐
    │                         Socle A                                  │
    │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │
    │  │ SessionMgr  │  │  Pipeline   │  │    GrpcServerWorker     │  │
    │  │             │  │  Executor   │  │    (port 9400)          │  │
    │  └─────────────┘  └─────────────┘  └─────────────────────────┘  │
    │         │               │                      │                 │
    │         └───────────────┼──────────────────────┘                 │
    │                         │                                        │
    │              ┌──────────┴──────────┐                            │
    │              │   gRPC Services     │                            │
    │              │  - SocleComm        │                            │
    │              │  - SessionService   │                            │
    │              │  - DiscoveryService │                            │
    │              └──────────┬──────────┘                            │
    └─────────────────────────┼───────────────────────────────────────┘
                              │ gRPC/HTTP2
                              ▼
    ┌─────────────────────────────────────────────────────────────────┐
    │                         Socle B                                  │
    │              ┌──────────────────────┐                           │
    │              │  PeerConnectionPool  │                           │
    │              └──────────────────────┘                           │
    └─────────────────────────────────────────────────────────────────┘
    

    Configuration

    application.yml

    socle:
      grpc:
        # Activation du module
        enabled: ${GRPC_ENABLED:false}
    
        # Port du serveur gRPC
        port: ${GRPC_PORT:9400}
    
        # Identification du Socle
        socle-id: ${SOCLE_ID:${socle.app_name}}
        socle-version: ${socle.version}
    
        # Limites serveur
        max-inbound-message-size: ${GRPC_MAX_MESSAGE_SIZE:4194304}  # 4MB
        max-concurrent-calls-per-connection: ${GRPC_MAX_CONCURRENT_CALLS:100}
    
        # Sessions
        session:
          ttl-seconds: ${GRPC_SESSION_TTL:1800}           # 30 min
          max-participants: ${GRPC_MAX_PARTICIPANTS:100}
          persist-to-tech-db: ${GRPC_PERSIST_SESSIONS:true}
          cache-in-redis: ${GRPC_CACHE_REDIS:true}
    
        # Pipeline de traitement
        pipeline:
          enabled: ${GRPC_PIPELINE_ENABLED:true}
          config-cache-ttl-seconds: ${GRPC_PIPELINE_CACHE_TTL:300}
    
        # Connexions peer
        peer:
          max-channels-per-peer: ${GRPC_PEER_MAX_CHANNELS:4}
          connection-timeout-ms: ${GRPC_PEER_CONNECT_TIMEOUT:5000}
          idle-timeout-seconds: ${GRPC_PEER_IDLE_TIMEOUT:300}
          keep-alive-enabled: ${GRPC_PEER_KEEPALIVE:true}
          keep-alive-time-seconds: ${GRPC_PEER_KEEPALIVE_TIME:30}
          keep-alive-timeout-seconds: ${GRPC_PEER_KEEPALIVE_TIMEOUT:10}
    

    Variables d’environnement

    Variable Default Description
    GRPC_ENABLED false Active le module gRPC
    GRPC_PORT 9400 Port du serveur gRPC
    SOCLE_ID ${app_name} Identifiant unique du Socle
    GRPC_SESSION_TTL 1800 TTL des sessions en secondes
    GRPC_MAX_PARTICIPANTS 100 Max participants par session
    GRPC_PIPELINE_ENABLED true Active le pipeline de traitement

    Services gRPC

    DiscoveryService

    Service de decouverte et health check.

    service DiscoveryService {
        rpc GetCapabilities(CapabilitiesRequest) returns (CapabilitiesResponse);
        rpc Ping(PingRequest) returns (PingResponse);
    }
    

    Test avec grpcurl :

    # Ping
    grpcurl -plaintext localhost:9400 socle.DiscoveryService/Ping
    
    # Capabilities
    grpcurl -plaintext localhost:9400 socle.DiscoveryService/GetCapabilities
    

    SessionService

    Gestion du cycle de vie des sessions.

    service SessionService {
        rpc CreateSession(CreateSessionRequest) returns (SessionInfo);
        rpc JoinSession(JoinSessionRequest) returns (JoinSessionResponse);
        rpc LeaveSession(LeaveSessionRequest) returns (LeaveSessionResponse);
        rpc GetSession(GetSessionRequest) returns (SessionInfo);
        rpc CloseSession(CloseSessionRequest) returns (CloseSessionResponse);
    }
    

    Exemples :

    # Creer une session
    grpcurl -plaintext -d '{
      "session_type": "chat",
      "owner_id": "user1",
      "ttl_seconds": 3600
    }' localhost:9400 socle.SessionService/CreateSession
    
    # Joindre une session
    grpcurl -plaintext -d '{
      "session_id": "uuid-de-la-session",
      "participant_id": "user2",
      "display_name": "User 2"
    }' localhost:9400 socle.SessionService/JoinSession
    
    # Obtenir info session
    grpcurl -plaintext -d '{
      "session_id": "uuid-de-la-session"
    }' localhost:9400 socle.SessionService/GetSession
    

    SocleComm

    Streaming bidirectionnel pour l’echange de messages.

    service SocleComm {
        rpc Exchange(stream SessionMessage) returns (stream SessionMessage);
    }
    

    Format des messages :

    message SessionMessage {
        string session_id = 1;
        string sender_id = 2;
        MessageKind kind = 3;        // JOIN, LEAVE, DATA, REQUEST, RESPONSE, etc.
        repeated string target_ids = 4;  // Vide = broadcast
        string correlation_id = 5;
        int64 timestamp = 6;
        string payload = 7;          // JSON
        map<string, string> headers = 8;
    }
    

    Sessions

    Cycle de vie

    CREATE ──► ACTIVE ──► CLOSING ──► CLOSED
                  │
                  └──► EXPIRED (TTL depasse)
    

    Stockage

    Les sessions sont stockees dans :

    1. Cache memoire : Acces rapide, stream observers
    2. Redis (si KvBus en mode redis) : Partage entre instances
    3. TechDB : Audit et persistance

    Tables TechDB

    -- Sessions (audit)
    CREATE TABLE grpc_sessions (
        x_id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
        x_dateCreated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
        x_dateChanged TIMESTAMP WITH TIME ZONE,
        x_sub VARCHAR(255),
        x_partition VARCHAR(30),
        x_comment CLOB,
        session_id VARCHAR(36) NOT NULL UNIQUE,
        session_type VARCHAR(100) NOT NULL,
        owner_id VARCHAR(100) NOT NULL,
        status VARCHAR(20) DEFAULT 'ACTIVE',
        datas CLOB
    );
    
    -- Configuration pipeline par type de session
    CREATE TABLE grpc_pipeline_configs (
        x_id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
        x_dateCreated TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
        x_dateChanged TIMESTAMP WITH TIME ZONE,
        x_sub VARCHAR(255),
        x_partition VARCHAR(30),
        x_comment CLOB,
        session_type VARCHAR(100) NOT NULL UNIQUE,
        janino_pre VARCHAR(255),
        jdm_rules VARCHAR(255),
        janino_post VARCHAR(255),
        default_targets VARCHAR(50) DEFAULT 'broadcast',
        enabled BOOLEAN DEFAULT TRUE,
        datas CLOB
    );
    

    Pipeline de traitement

    Le pipeline permet de transformer les messages et determiner leur routage.

    Etapes

    Message entrant
          │
          ▼
    ┌─────────────────┐
    │  1. Janino PRE  │  Transformation du payload
    └────────┬────────┘
             │
             ▼
    ┌─────────────────┐
    │  2. JDM Rules   │  Determination des cibles (routing)
    └────────┬────────┘
             │
             ▼
    ┌─────────────────┐
    │  3. Janino POST │  Transformation finale
    └────────┬────────┘
             │
             ▼
        Routage
       /        \
    Broadcast   Cible
    

    Configuration du pipeline

    Inserez dans grpc_pipeline_configs :

    INSERT INTO grpc_pipeline_configs
    (session_type, janino_pre, jdm_rules, janino_post, default_targets, enabled)
    VALUES
    ('chat', NULL, 'chat_routing', NULL, 'broadcast', TRUE);
    

    Exemple script Janino PRE

    // repository/scripts/java/grpc/chat_filter.java
    public class ChatFilter {
        public Object execute(Map<String, Object> context) {
            Map<String, Object> payload = (Map<String, Object>) context.get("payload");
            String senderId = (String) context.get("senderId");
    
            // Ajouter metadata
            payload.put("processed_at", System.currentTimeMillis());
            payload.put("sender", senderId);
    
            return payload;
        }
    }
    

    Exemple modele JDM pour routing

    {
      "name": "chat_routing",
      "nodes": [
        {
          "id": "input",
          "type": "inputNode",
          "content": {
            "fields": [
              {"field": "payload.target", "type": "string"},
              {"field": "payload.type", "type": "string"}
            ]
          }
        },
        {
          "id": "decision",
          "type": "decisionTableNode",
          "content": {
            "hitPolicy": "first",
            "inputs": [
              {"field": "payload.type"}
            ],
            "outputs": [
              {"field": "targets", "type": "string"},
              {"field": "broadcast", "type": "boolean"}
            ],
            "rules": [
              {"_input": ["private"], "_output": ["${payload.target}", false]},
              {"_input": ["broadcast"], "_output": ["", true]},
              {"_input": ["*"], "_output": ["", true]}
            ]
          }
        }
      ]
    }
    

    Connexions Peer

    Ajouter un peer programmatiquement

    @Autowired
    private PeerConnectionPool peerPool;
    
    public void connectToPeer() {
        // Ajouter et connecter
        boolean connected = peerPool.addPeer("socle-b", "192.168.1.100", 9400);
    
        if (connected) {
            // Ouvrir un stream
            peerPool.openStream("socle-b", message -> {
                // Handler pour messages entrants
                System.out.println("Received: " + message.getPayload());
            });
    
            // Envoyer un message
            SessionMessage msg = SessionMessage.newBuilder()
                .setSessionId("...")
                .setSenderId("local-participant")
                .setKind(MessageKind.DATA)
                .setPayload("{\"text\":\"Hello\"}")
                .build();
    
            peerPool.sendToPeer("socle-b", msg);
        }
    }
    

    Broadcast vers tous les peers

    SessionMessage msg = SessionMessage.newBuilder()
        .setSessionId(sessionId)
        .setSenderId(myId)
        .setKind(MessageKind.DATA)
        .setPayload(jsonPayload)
        .build();
    
    int sent = peerPool.broadcast(msg, null);  // null = inclure tous
    

    Worker gRPC

    Priorites

    Methode Valeur Description
    getStartPriority() 800 Demarre apres Janino (25) et JDM (30)
    getStopPriority() 10 S’arrete tot pour drain des connexions

    Statistiques

    @Autowired
    private GrpcServerWorker grpcWorker;
    
    Map<String, Object> stats = grpcWorker.getStats();
    // {
    //   "running": true,
    //   "port": 9400,
    //   "sessions": { "cached_sessions": 5, "active_streams": 12 },
    //   "messaging": { "messages_received": 1234, "messages_sent": 5678 },
    //   "peers": { "connected_peers": 2, "total_messages_sent": 100 }
    // }
    

    Types de sessions supportes

    Type Description
    chat Communication temps reel
    sync Synchronisation de donnees
    broadcast Diffusion un-vers-plusieurs
    pipeline Traitement en chaine
    custom Type personnalise

    Securite

    TLS (a venir)

    Le support TLS sera ajoute dans une version future :

    socle:
      grpc:
        tls:
          enabled: true
          cert-path: /path/to/server.crt
          key-path: /path/to/server.key
          ca-path: /path/to/ca.crt  # Pour mTLS
    

    Authentification

    L’authentification peut etre implementee via :

    • Metadata gRPC (tokens dans headers)
    • Intercepteurs personnalises
    • Integration avec le module Auth existant

    Monitoring

    Metriques exposees

    Les metriques sont disponibles via l’endpoint Prometheus /actuator/prometheus :

    • grpc_sessions_active : Nombre de sessions actives
    • grpc_messages_received_total : Total messages recus
    • grpc_messages_sent_total : Total messages envoyes
    • grpc_peers_connected : Nombre de peers connectes

    Logs

    [grpc_server] Started on port 9400
    [grpc_session] Session created: abc-123 (type=chat, owner=user1)
    [grpc_session] Participant joined: user2 -> abc-123 (total: 2)
    [grpc_comm] Message received: session=abc-123, sender=user1, kind=DATA
    [grpc_pipeline] Executed for session abc-123 (targets=broadcast)
    

    Troubleshooting

    Le serveur ne demarre pas

    1. Verifiez que le port 9400 n’est pas utilise
    2. Verifiez GRPC_ENABLED=true
    3. Consultez les logs pour les erreurs d’initialisation

    Sessions expirent trop vite

    Augmentez le TTL :

    socle:
      grpc:
        session:
          ttl-seconds: 7200  # 2 heures
    

    Messages non delivres

    1. Verifiez que le destinataire a un stream actif
    2. Verifiez les logs du pipeline pour les erreurs
    3. Verifiez la configuration du routing dans grpc_pipeline_configs

    Erreurs de connexion peer

    1. Verifiez la connectivite reseau
    2. Verifiez que le peer a gRPC active
    3. Augmentez le timeout de connexion :
    socle:
      grpc:
        peer:
          connection-timeout-ms: 10000
    

    Structure des fichiers

    src/main/java/eu/lmvi/socle/grpc/
    ├── GrpcConfiguration.java        # Configuration Spring
    ├── GrpcServerWorker.java         # Worker principal
    ├── session/
    │   ├── Session.java              # Record session
    │   ├── Participant.java          # Record participant
    │   ├── SessionStatus.java        # Enum ACTIVE/CLOSING/CLOSED/EXPIRED
    │   ├── SessionManager.java       # Gestion sessions + streams
    │   └── SessionRedisRepository.java  # Persistance Redis
    ├── pipeline/
    │   ├── PipelineConfig.java       # Config pipeline
    │   ├── PipelineResult.java       # Resultat pipeline
    │   └── PipelineExecutor.java     # Execution Janino + JDM
    ├── peer/
    │   ├── PeerConnection.java       # Connexion unique
    │   └── PeerConnectionPool.java   # Pool avec reconnexion
    └── service/
        ├── SocleCommService.java     # Streaming bidirectionnel
        ├── SessionServiceImpl.java   # CRUD sessions
        └── DiscoveryServiceImpl.java # Discovery + Ping
    
    src/main/proto/
    └── socle_comm.proto              # Definition Protocol Buffers