Security and Performance Enhancements in Traefik Configuration in pangolin

Enhanced Traefik Configuration Structure and Security Implementation Guide

Let me explain the new folder structure and organization that has been implemented to enhance Traefik’s configuration management, security, and maintainability.

Directory Structure Overview

/root/config/
├── crowdsec/               # CrowdSec configuration and data
├── crowdsec_logs/         # CrowdSec log files
├── db/                    # Database files
├── letsencrypt/          # SSL certificates
├── logs/                 # General application logs
├── traefik/              # Traefik configuration root
│   ├── conf/            # Additional configuration files
│   ├── logs/            # Traefik-specific logs
│   ├── rules/           # Modular configuration rules
│   │   ├── bouncer-crowdsec.yaml
│   │   ├── default-headers.yaml
│   │   ├── forwardAuth-authentik.yaml
│   │   ├── middlewares-authentik.yaml
│   │   ├── middlewares-buffering.yaml
│   │   ├── middlewares-compress.yaml
│   │   ├── middlewares-http-cache.yaml
│   │   ├── middlewares-https-redirectscheme.yaml
│   │   ├── middlewares-rate-limit.yaml
│   │   ├── middlewares-secure-headers.yaml
│   │   ├── routing.yaml
│   │   ├── tls-opts.yaml
│   │   └── traefik-dashboard-auth.yaml
│   └── traefik_config.yml
├── config.yml             # Main application configuration
└── key                    # Security keys

Key Improvements in Organization

The new structure implements several important organizational improvements over the original configuration:

  1. Modular Configuration Files

    • The original configuration had everything in a single traefik_config.yml file
    • Now separated into logical components under /traefik/rules/
    • Each middleware has its own configuration file
    • Makes maintenance and updates much simpler
  2. Security Integration

    • New /crowdsec/ directory for threat detection
    • Dedicated /crowdsec_logs/ for security monitoring
    • Separate bouncer-crowdsec.yaml for security rules
    • SSL certificates organized in /letsencrypt/
  3. Logging Organization

    • Hierarchical log structure with:
      • Main logs in /logs/
      • Traefik-specific logs in /traefik/logs/
      • Security logs in /crowdsec_logs/
    • Better log segregation for easier troubleshooting
  4. Configuration Management

    • Main configuration remains in /config/config.yml
    • Traefik configuration split between:
      • Core settings in traefik_config.yml
      • Modular rules in /rules/
    • Additional configurations in /conf/ directory
  5. Access Control

    • New traefik-dashboard-auth.yaml for dashboard security
    • Authentication configurations separated into dedicated files
    • Integration with external auth providers through specific modules

This new structure provides several benefits:

  • Easier maintenance and updates
  • Better security organization
  • Clearer separation of concerns
  • Simplified troubleshooting
  • More flexible configuration management
  • Enhanced modularity for future extensions

The organization allows for easy addition of new features while maintaining clarity and security. Each component has its specific location and purpose, making the system more maintainable and scalable.

Comprehensive Analysis of Security and Performance Enhancements in Traefik Configuration

1. Modular Configuration Structure

The new setup adopts a more modular approach by splitting the configuration into separate YAML files under the /rules directory. This improves maintainability and organization compared to the original monolithic configuration. Key modules include:

  • bouncer-crowdsec.yaml: CrowdSec integration for security
  • middlewares-secure-headers.yaml: Enhanced security headers
  • middlewares-compress.yaml: Response compression
  • middlewares-http-cache.yaml: Caching configuration
  • tls-opts.yaml: TLS security settings

2. Security Enhancements

CrowdSec Integration

The new configuration implements CrowdSec for threat detection and mitigation, which wasn’t present in the original setup. This adds:

  • Real-time threat detection
  • CAPTCHA protection using Cloudflare Turnstile
  • Web Application Firewall (WAF) functionality
  • Rate limiting and automated IP blocking

Enhanced TLS Security

The new tls-opts.yaml introduces stricter TLS configuration:

  • Enforces TLS 1.2 minimum version
  • Implements modern cipher suites
  • Adds curve preferences for better security
  • Enables SNI strict checking

Security Headers

The new configuration adds comprehensive security headers:

  • Content Security Policy (CSP)
  • Permissions Policy
  • Referrer Policy
  • X-Robots-Tag for search engine control
  • Enhanced XSS protection

3. Performance Optimizations

HTTP Caching

Added Souin plugin for HTTP caching with:

  • Configurable TTL per route
  • Selective caching based on content type
  • Cache exclusions for sensitive endpoints
  • Support for surrogate keys

Compression

New compression middleware with:

  • Selective compression based on content type
  • Minimum response size threshold
  • Optimized encoding settings

4. Monitoring and Logging

Enhanced Access Logging

The new configuration implements detailed access logging:

  • JSON formatted logs
  • Selective field capture
  • Header filtering
  • Privacy-aware logging (redacting sensitive data)

Metrics Integration

Added support for:

  • Prometheus metrics endpoint
  • Detailed request tracking
  • Performance monitoring

5. Infrastructure Changes

Docker Compose Updates

  • Updated Traefik version to 3.3.3
  • Added volume mounts for rules directory
  • Improved container networking configuration
  • Added CrowdSec container integration

6. Authentication and Authorization

Basic Authentication

Added support for:

  • Dashboard authentication
  • Multiple authentication methods
  • Integration with external auth providers

This enhanced configuration significantly improves the security posture, performance, and maintainability of the Traefik setup while maintaining compatibility with the existing Pangolin infrastructure. The modular approach makes it easier to manage and update individual components without affecting the entire system.

Dont Use extra middlewares if you don’t know how to navigate them.

Ask If in doubt

This is not approved by the developers but my own way of keeping the stack clean.

For env either you can create directly or make an .env file.e.g {{env "DOMAINNAME"}} etc

All Files are below.

main docker-compose.yml

 name: pangolin
networks:
  default:
    driver: bridge
    name: pangolin
services:
  crowdsec:
    command: -t
    container_name: crowdsec
    environment:
      ACQUIRE_FILES: /var/log/traefik/*.log
      COLLECTIONS: crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules
      ENROLL_INSTANCE_NAME: pangolin-crowdsec
      ENROLL_TAGS: docker
      GID: "1000"
      PARSERS: crowdsecurity/whitelists
    expose:
      - 6060
    healthcheck:
      test:
        - CMD
        - cscli
        - capi
        - status
    image: crowdsecurity/crowdsec:latest
    labels:
      - traefik.enable=false
    ports:
      - 6060:6060
    restart: unless-stopped
    volumes:
      - ./config/crowdsec:/etc/crowdsec
      - ./config/crowdsec/db:/var/lib/crowdsec/data
      - ./config/crowdsec_logs/auth.log:/var/log/auth.log:ro
      - ./config/crowdsec_logs/syslog:/var/log/syslog:ro
      - ./config/crowdsec_logs:/var/log
      - ./config/traefik/logs:/var/log/traefik
  gerbil:
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    command:
      - --reachableAt=http://gerbil:3003
      - --generateAndSaveKeyTo=/var/config/key
      - --remoteConfig=http://pangolin:3001/api/v1/gerbil/get-config
      - --reportBandwidthTo=http://pangolin:3001/api/v1/gerbil/receive-bandwidth
    container_name: gerbil
    depends_on:
      pangolin:
        condition: service_healthy
    image: fosrl/gerbil:1.0.0
    ports:
      - 51820:51820/udp
      - 443:443
      - 80:80
    restart: unless-stopped
    volumes:
      - ./config/:/var/config
  pangolin:
    container_name: pangolin
    healthcheck:
      interval: 3s
      retries: 5
      test:
        - CMD
        - curl
        - -f
        - http://localhost:3001/api/v1/
      timeout: 3s
    image: fosrl/pangolin:1.0.1
    restart: unless-stopped
    volumes:
      - ./config:/app/config
  traefik:
    command:
      - --configFile=/etc/traefik/traefik_config.yml
    container_name: traefik
    depends_on:
      pangolin:
        condition: service_healthy
    image: traefik:v3.3.4
    network_mode: service:gerbil
    restart: unless-stopped
    volumes:
      - ./config/traefik:/etc/traefik:ro
      - ./config/letsencrypt:/letsencrypt
      - ./config/traefik/logs:/var/log/traefik
      - ./config/traefik/rules:/rules # New addition
      - ./config/traefik/conf/:/etc/traefik/conf/ # Volume to store the captcha.html file

traefik_config.yml

################################################################
# API and Dashboard
################################################################

api:
  insecure: true
  dashboard: true

################################################################
# Providers - https://doc.traefik.io/traefik/providers/docker/
################################################################

providers:
  http:
    endpoint: "http://pangolin:3001/api/v1/traefik-config"
    pollInterval: "5s"
  file:
    directory: /rules
    watch: true

################################################################
# Plugins- Badger, Bouncer traefik  (crowdsec), souin
################################################################

experimental:
  plugins:
    badger:
      moduleName: "github.com/fosrl/badger"
      version: "v1.0.0-beta.3"
    crowdsec:
      moduleName: "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
      version: "v1.3.5"
    # http cache for traefik
    souin:
      moduleName: github.com/darkweak/souin
      version: v1.6.50

################################################################
# Logs - https://doc.traefik.io/traefik/observability/logs/
################################################################

log:
  level: "INFO"
  format: "json"

################################################################
# Access logs - https://doc.traefik.io/traefik/observability/access-logs/
################################################################

accessLog:
  filePath: "/var/log/traefik/access.log"
  format: json
  filters:
    statusCodes:
      - "200-299"  # Success codes
      - "400-499"  # Client errors
      - "500-599"  # Server errors
    retryAttempts: true
    minDuration: "100ms"  # Increased to focus on slower requests
  bufferingSize: 100      # Add buffering for better performance
  fields:
    defaultMode: drop     # Start with dropping all fields
    names:
      ClientAddr: keep
      ClientHost: keep
      RequestMethod: keep
      RequestPath: keep
      RequestProtocol: keep
      DownstreamStatus: keep
      DownstreamContentSize: keep
      Duration: keep
      ServiceName: keep
      StartUTC: keep
      TLSVersion: keep
      TLSCipher: keep
      RetryAttempts: keep
    headers:
      defaultMode: drop
      names:
        User-Agent: keep
        X-Real-Ip: keep
        X-Forwarded-For: keep
        X-Forwarded-Proto: keep
        Content-Type: keep
        Authorization: redact  # Redact sensitive information
        Cookie: redact        # Redact sensitive information

################################################################
# Let's Encrypt (ACME)
################################################################

certificatesResolvers:
  letsencrypt:
    acme:
      httpChallenge:
        entryPoint: web
      email: "discourse@hhf.technology"
      storage: "/letsencrypt/acme.json"
      caServer: "https://acme-v02.api.letsencrypt.org/directory"
      #dnsChallenge:
        #provider: cloudflare
        #delayBeforeCheck: 30 # Default is 2m0s.  This changes the delay (in seconds)
        # Custom DNS server resolution
        #resolvers:
          #- "1.1.1.1:53"
          #- "8.8.8.8:53"


################################################################
# Entrypoints - https://doc.traefik.io/traefik/routing/entrypoints/
################################################################

entryPoints:
  web:
    address: ":80"
    forwardedHeaders:
      trustedIPs:
        # Cloudflare (https://www.cloudflare.com/ips-v4)
        - "173.245.48.0/20"
        - "103.21.244.0/22"
        - "103.22.200.0/22"
        - "103.31.4.0/22"
        - "141.101.64.0/18"
        - "108.162.192.0/18"
        - "190.93.240.0/20"
        - "188.114.96.0/20"
        - "197.234.240.0/22"
        - "198.41.128.0/17"
        - "162.158.0.0/15"
        - "104.16.0.0/13"
        - "104.24.0.0/14"
        - "172.64.0.0/13"
        - "131.0.72.0/22"
        # Local IPs
        - "127.0.0.1/32"
        - "10.0.0.0/8"
        - "192.168.0.0/16"
        - "172.16.0.0/12"  
  websecure:
    address: ":443"
    transport:
      respondingTimeouts:
        readTimeout: "30m"
    http:
      middlewares:                                                                   # CHANGE MADE HERE (BOUNCER ENABLED) !!!
        - crowdsec@file
        - middlewares-compress@file
        - middlewares-secure-headers@file
      tls:
        options: tls-opts@file
        certResolver: letsencrypt
    http3: {}
    forwardedHeaders:
      trustedIPs:
        # Cloudflare (https://www.cloudflare.com/ips-v4)
        - "173.245.48.0/20"
        - "103.21.244.0/22"
        - "103.22.200.0/22"
        - "103.31.4.0/22"
        - "141.101.64.0/18"
        - "108.162.192.0/18"
        - "190.93.240.0/20"
        - "188.114.96.0/20"
        - "197.234.240.0/22"
        - "198.41.128.0/17"
        - "162.158.0.0/15"
        - "104.16.0.0/13"
        - "104.24.0.0/14"
        - "172.64.0.0/13"
        - "131.0.72.0/22"
        # Local IPs
        - "127.0.0.1/32"
        - "10.0.0.0/8"
        - "192.168.0.0/16"
        - "172.16.0.0/12"


serversTransport:
  insecureSkipVerify: true

Delete dynamic-traefik.yml

Folder # Rules

1. bouncer-crowdsec.yaml

http:
  middlewares:
    # CrowdSec configuration with proper IP forwarding
    crowdsec:
      plugin:
        crowdsec:
          enabled: true
          logLevel: INFO
          updateIntervalSeconds: 15
          updateMaxFailure: 0
          defaultDecisionSeconds: 15
          httpTimeoutSeconds: 10
          crowdsecMode: live
          crowdsecAppsecEnabled: true
          crowdsecAppsecHost: crowdsec:7422
          crowdsecAppsecFailureBlock: true
          crowdsecAppsecUnreachableBlock: true
          crowdsecLapiKey: "csdcdssffdsfdsfdsf+xXWwpif8j0WeBpeo"
          crowdsecLapiHost: crowdsec:8080
          crowdsecLapiScheme: http
          captchaProvider: turnstile
          captchaSiteKey: "dgdfgdfgdfdfgbfdgbdfbdf"
          captchaSecretKey: "0x4AAAAAAA6KvsdvdsvDMdsfdsda8a8qmj1E"
          captchaGracePeriodSeconds: 1800
          captchaHTMLFilePath: "/etc/traefik/conf/captcha.html"
          forwardedHeadersTrustedIPs:
            - "0.0.0.0/0"                                                              # Cloudflare tunnel IP address
          clientTrustedIPs:
            - "10.0.0.0/8"                                                              # Internal LAN IP addresses
            - "172.16.0.0/12"                                                              # Internal LAN IP addresses
            - "192.168.0.0/16"                                                              # Internal LAN IP addresses
            - "100.89.137.0/20"                                                              # Internal LAN IP addresses
          # banHTMLFilePath: ./ban.html

2. default-headers.yaml

http:
  middlewares:
    default-headers:
      headers:
        frameDeny: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https

3. forwardAuth-authentik.yaml

################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    ################################################################
    # Forward Authentication - OAUTH / 2FA
    ################################################################
    #
    # https://git.hhf.technology/goauthentik/authentik/issues/2366
    forwardAuth-authentik:
      forwardAuth:
        address: "http://authentik_server:9000/outpost.goauthentik.io/auth/traefik"
        trustForwardHeader: true
        authResponseHeaders:
          - X-authentik-username
          - X-authentik-groups
          - X-authentik-email
          - X-authentik-name
          - X-authentik-uid
          - X-authentik-jwt
          - X-authentik-meta-jwks
          - X-authentik-meta-outpost
          - X-authentik-meta-provider
          - X-authentik-meta-app
          - X-authentik-meta-version

4. middlewares-authentik.yaml

http:
  middlewares:
    # https://git.hhf.technology/goauthentik/authentik/issues/2366
    middlewares-authentik:
      forwardAuth:
        address: "http://authentik_server:9000/outpost.goauthentik.io/auth/traefik"
        trustForwardHeader: true
        authResponseHeaders:
          - X-authentik-username
          - X-authentik-groups
          - X-authentik-email
          - X-authentik-name
          - X-authentik-uid
          - X-authentik-jwt
          - X-authentik-meta-jwks
          - X-authentik-meta-outpost
          - X-authentik-meta-provider
          - X-authentik-meta-app
          - X-authentik-meta-version

5. middlewares-buffering.yaml

################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    # Prevent too large of a body
    # https://stackoverflow.com/questions/49717670/how-to-config-upload-body-size-restriction-in-traefik
    middlewares-buffering:
      buffering:
        maxRequestBodyBytes: 10485760
        memRequestBodyBytes: 2097152
        maxResponseBodyBytes: 10485760
        memResponseBodyBytes: 2097152
        retryExpression: "IsNetworkError() && Attempts() <= 2"

6. middlewares-compress.yaml

################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    middlewares-compress:
      compress:
        includedContentTypes:
          - application/json
          - text/html
          - text/plain
        minResponseBodyBytes: 1024
        defaultEncoding: gzip

7. middlewares-http-cache.yaml

http:
  middlewares:
    http-cache:
      plugin:
        souin:
          api:
            # prometheus: {}
            souin: {}
          default_cache:
            ttl: 60s  # TTL par 
            allowed_http_verbs:
              - GET
              - HEAD
              - POST
            default_cache_control: public, max-age=60
            force: true
          log_level: debug

    portainer-cache:
      plugin:
        souin:
          api:
            prometheus: {}
            souin: {}
          default_cache:
            regex:
              exclude: '/api/.*|/auth/.*|/settings/.*'  
            ttl: 60s  
            allowed_http_verbs:
              - GET
              - HEAD
            default_cache_control: public, max-age=60
          log_level: debug
          urls:
            'portainer.{{ env "DOMAINNAME"}}':
              ttl: 120s
              default_cache_control: public, max-age=120
            'portainer.{{ env "DOMAINNAME"}}/login':
              ttl: 0s  
          ykeys:
            Portainer_API_Key:
              headers:
                Content-Type: 'application/json'
          surrogate_keys:
            Portainer_Surrogate_Key:
              headers:
                Content-Type: 'application/json'

    servarr-cache:
      plugin:
        souin:
          api:
            prometheus: {}
            souin: {}
          default_cache:
            regex:
              exclude: '/api/.*|/auth/.*|/indexers/.*|/download/.*|/profile/.*'  
            ttl: 60s  
            allowed_http_verbs:
              - GET
              - HEAD
            default_cache_control: public, max-age=60
          log_level: debug
          urls:
            'sonarr.{{ env "DOMAINNAME" }}/':
              ttl: 300s
            'radarr.{{ env "DOMAINNAME" }}/':
              ttl: 300s 
            'lidarr.{{ env "DOMAINNAME" }}/':
              ttl: 300s 
            'readarr.{{ env "DOMAINNAME" }}/':
              ttl: 300s 
          ykeys:
            Servarr_Static_Key:
              headers:
                Content-Type: 'text/html'
          surrogate_keys:
            Servarr_Surrogate_Key:
              headers:
                Content-Type: 'application/json'

8. middlewares-https-redirectscheme.yaml

################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    # Middleware for Redirection
    # This can be used instead of global redirection
    middlewares-https-redirectscheme:
      redirectScheme:
        scheme: https
        permanent: true

9. middlewares-rate-limit.yaml

################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    # DDoS Prevention
    middlewares-rate-limit:
      rateLimit:
        average: 100
        burst: 50

10. middlewares-secure-headers.yaml

################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    ################################################################
    # Good Basic Security Practices
    ################################################################
    middlewares-secure-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlMaxAge: 100
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        stsSeconds: 63072000
        stsIncludeSubdomains: true
        stsPreload: true
        forceSTSHeader: true
        # customFrameOptionsValue: "allow-from https:{{env "DOMAINNAME"}}" #CSP takes care of this but may be needed for organizr.
        customFrameOptionsValue: SAMEORIGIN # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
        contentTypeNosniff: true
        browserXssFilter: true
        sslForceHost: true # add sslHost to all of the services
        sslHost: "*.testing.hhf.technology"
        referrerPolicy: "same-origin"
        permissionsPolicy: "camera=(), microphone=(), geolocation=(), payment=(), usb=()"
        customResponseHeaders:
          X-Robots-Tag: "none,noarchive,nosnippet,notranslate,noimageindex,noindex,nofollow" #global not tracking with websearch
          # X-Robots-Tag: "noindex,nofollow" "  # nextcloud recommandation 
          server: ""
          # https://community.traefik.io/t/how-to-make-websockets-work-with-traefik-2-0-setting-up-rancher/1732
          # X-Forwarded-Proto: "https"

11. routing.yaml

http:
      # if you want add athentik on your service just add this on rour router
  routers:

    # HTTP to HTTPS redirect router
    main-app-router-redirect:
      rule: "Host(`pangolin.testing.hhf.technology`)"
      service: next-service
      entryPoints:
        - web
      middlewares:
        - middlewares-https-redirectscheme

    # Next.js router (handles everything except API and WebSocket paths)
    next-router:
      rule: "Host(`pangolin.testing.hhf.technology`) && !PathPrefix(`/api/v1`)"
      service: next-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

    # API router (handles /api/v1 paths)
    api-router:
      rule: "Host(`pangolin.testing.hhf.technology`) && PathPrefix(`/api/v1`)"
      service: api-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

    # WebSocket router
    ws-router:
      rule: "Host(`pangolin.testing.hhf.technology`)"
      service: api-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

# if you want add athentik on your service just add this on rour router

    #authentik-rtr:
      #service: authentik-svc
      #rule: Host(`{{ env "AUTHENTIK_HOST" }}`)
    
    #authentik-output-rtr:
      #service: authentik-svc
      #rule: HostRegexp(`{subdomain:[a-z0-9-]+}.{{ env "DOMAINNAME" }}`) && PathPrefix(`{{ env "AUTHENTIK_OUTPOST_PATH_PREFIX" }}`)

    traefik-rtr:
      rule: "Host(`traefik.testing.hhf.technology`)"
      service: api@internal
      middlewares:
        - traefik-dashboard-auth@file

 

  services:

    next-service:
      loadBalancer:
        servers:
          - url: "http://pangolin:3002"  # Next.js server

    api-service:
      loadBalancer:
        servers:
          - url: "http://pangolin:3000"  # API/WebSocket server

    #authentik-svc:
      #loadBalancer:
        #servers:
          #- url: {{ env "AUTHENTIK_URL" }}

 
  serversTransports:
    insecureTransport:
      insecureSkipVerify: true

12. tls-opts.yaml

################################################################
# Dynamic configuration
# https://doc.traefik.io/traefik/https/tls/
################################################################
tls:
  options:
    tls-opts:
      sniStrict: true
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
        - TLS_AES_128_GCM_SHA256
        - TLS_AES_256_GCM_SHA384
        - TLS_CHACHA20_POLY1305_SHA256
        - TLS_FALLBACK_SCSV # Client is doing version fallback. See RFC 7507
      curvePreferences:
        - secp521r1 # CurveP521
        - secp384r1 # CurveP384
    mintls13:
      minVersion: VersionTLS13

13. traefik-dashboard-auth.yaml

http:
  middlewares:
    traefik-dashboard-auth:
      basicAuth:
        users:
          - "VFF0WVRkak56YzBOek5rWkRGaiJ9"

If you need .env e.g.(This is not compulsory)

################################################################
# Base Configuration
################################################################
DOCKERDIR=/CHAMGEME/Homelab-docker-server
PUID=root
PGID=root
DOMAINNAME=CHANGE_ME
################################################################
# SMTP  Configuration base conf wit google smtp
# https://support.google.com/accounts/answer/185833?hl=fr
################################################################
SMPT_EMAIL_HOST=smtp.gmail.com
SMPT_EMAIL_PORT=25
SMPT_EMAIL_USERNAME=gmail_smtp_username # secrets name
SMPT_EMAIL_PASSWORD=gmail_smtp_password # secrets name
SMPT_EMAIL_USE_TLS=true
SMPT_EMAIL_USE_SSL=false
SMPT_EMAIL_TIMEOUT=10
SMPT_EMAIL_FROM=gmail_smtp_username # secrets name

################################################################
#################### Traefik 3 - June 2024 #####################
# Cloudflare IPs (IPv4 and/or IPv6): https://www.cloudflare.com/ips/
################################################################

CF_EMAIL=CHANGEME
CLOUDFLARE_IPS=173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/13,104.24.0.0/14,172.64.0.0/13,131.0.72.0/22
LOCAL_IPS=127.0.0.1/32,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12

################################################################
# Secrets command
################################################################
SECRETS_RUN=/run/secrets/
SECRETS_FILE=file://${SECRETS_RUN}

################################################################
# Proxy services
################################################################

DOCKER_HOST=tcp://socket-proxy:2375

################################################################
# Traefik Configuration
# generate TRAEFIK_DASHBOARD_CREDENTIALS here :  https://www.web2generators.com/apache-tools/htpasswd-generator
################################################################

TRAEFIK_DASHBOARD_CREDENTIALS=CHANGE_ME
TRAEFIK_DASHBOARD_NAME=traefik-dashboard
TRAEFIK_DASHBOARD_HOST=${TRAEFIK_DASHBOARD_NAME}.${DOMAINNAME}

################################################################
# Authentik Configuration
################################################################

AUTHENTIK_SERVICE_NAME=authentik_server
AUTHENTIK_SERVICE_PORT=9000
AUTHENTIK_COOKIE_DOMAIN=${DOMAINNAME}
AUTHENTIK_HOST=authentik.${DOMAINNAME}
AUTHENTIK_URL=http://${AUTHENTIK_SERVICE_NAME}:${AUTHENTIK_SERVICE_PORT}
AUTHENTIK_OUTPOST_PATH_PREFIX=/outpost.goauthentik.io/

POSTGRES_PASSWORD_FILE=${SECRETS_RUN}authentik_postgresql_password
#POSTGRES_USER_FILE=${SECRETS_RUN}authentik_postgresql_user
POSTGRES_USER_FILE=${SECRETS_RUN}authentik_postgresql_db
POSTGRES_DB_FILE=${SECRETS_RUN}authentik_postgresql_db
AUTHENTIK_REDIS__HOST=authentik_redis
AUTHENTIK_POSTGRESQL__HOST=authentik_postgresql
AUTHENTIK_POSTGRESQL__NAME=${SECRETS_FILE}authentik_postgresql_db
#AUTHENTIK_POSTGRESQL__USER=${SECRETS_FILE}authentik_postgresql_user
AUTHENTIK_POSTGRESQL__USER=${SECRETS_FILE}authentik_postgresql_db
AUTHENTIK_POSTGRESQL__PASSWORD=${SECRETS_FILE}authentik_postgresql_password
AUTHENTIK_DISABLE_STARTUP_ANALYTICS=true
AUTHENTIK_DISABLE_UPDATE_CHECK=false
AUTHENTIK_ERROR_REPORTING__ENABLED=false
AUTHENTIK_LOG_LEVEL=info # debug, info, warning, error, trace
AUTHENTIK_SECRET_KEY=${SECRETS_FILE}authentik_secret_key # openssl rand 60 | base64 -w 0
AUTHENTIK_COOKIE_DOMAIN=${DOMAINNAME}
# AUTHENTIK_LISTEN__TRUSTED_PROXY_CIDRS: CHANGEME_IFAPPLICABLE # Defaults to all of: 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fe80::/10, ::1/128
DOCKER_HOST=tcp://socket-proxy:2375 # Use this if you have Socket Proxy enabled.

# SMPT authentik configuration
AUTHENTIK_EMAIL__HOST=${SMPT_EMAIL_HOST}
AUTHENTIK_EMAIL__PORT=${SMPT_EMAIL_PORT}
AUTHENTIK_EMAIL__USERNAME=${SECRETS_FILE}${SMPT_EMAIL_USERNAME}
AUTHENTIK_EMAIL__PASSWORD=${SECRETS_FILE}${SMPT_EMAIL_PASSWORD}
AUTHENTIK_EMAIL__USE_TLS=${SMPT_EMAIL_USE_TLS}
AUTHENTIK_EMAIL__USE_SSL=${SMPT_EMAIL_USE_SSL}
AUTHENTIK_EMAIL__TIMEOUT=${SMPT_EMAIL_TIMEOUT}
AUTHENTIK_EMAIL__FROM=${SECRETS_FILE}${SMPT_EMAIL_FROM}


################################################################
# GeoIP Configuration
# Go to https://dev.maxmind.com/geoip/geolite2-free-geolocation-data in order to generate a free license key 
# https://www.maxmind.com/en/accounts/current/license-key for use.
################################################################
GEOIPUPDATE_EDITION_IDS="GeoLite2-City GeoLite2-ASN"
GEOIPUPDATE_FREQUENCY=8
GEOIPUPDATE_ACCOUNT_ID_FILE=${SECRETS_RUN}geoip_acccount_id
GEOIPUPDATE_LICENSE_KEY_FILE=${SECRETS_RUN}geoip_license_key

################################################################
# Crowdsec Configuration
################################################################
CROWDSEC_TRAEFIK_BOUNCER_LAPI_KEY=CHANGE_ME #to get api key : docker exec crowdsec cscli bouncers add traefik-bouncer
8 Likes

Do you mind creating a git repo to be used as template?
Also it seems pangolin 1.0.1 deprecated env vars in favor of flags:

Remove support for environment variables overriding config variables

From Release 1.0.1 · fosrl/pangolin · GitHub

1 Like

.env is not related to pangolin, it related to traefik. Sure bro will do it this week in github repo.

2 Likes

You are right. I misread the docker compose file and the env vars you are using are for the crowdsec container, not the pangolin one. Sorry for the noise!

:clap:

no issue. its optional. have fun. and don’t forget to check out the new bot for pangolin
Pangolin Discord bot to Manage the stack - Networking

2 Likes

How to access the pangolin menu now?

I followed the steps and there is no open port now or based on the domain I added in .env gives me 404.

Did I missed or maybe a misconfiguration at my side?

Getting a “404” page. Seems like i am not the only one, here are my logs

traefik  | {"level":"error","entryPointName":"web","routerName":"main-app-router-redirect@file","error":"middleware \"middlewares-https-redirectscheme@file\" does not exist","time":"2025-03-21T19:51:55Z"}
traefik  | {"level":"error","entryPointName":"websecure","routerName":"1-router@http","error":"invalid middleware \"badger@http\" configuration: invalid middleware type or middleware does not exist","time":"2025-03-21T19:51:55Z"}
traefik  | {"level":"error","entryPointName":"websecure","routerName":"ws-router@file","error":"invalid middleware \"crowdsec@file\" configuration: invalid middleware type or middleware does not exist","time":"2025-03-21T19:51:55Z"}
traefik  | {"level":"error","entryPointName":"websecure","routerName":"4-router@http","error":"invalid middleware \"badger@http\" configuration: invalid middleware type or middleware does not exist","time":"2025-03-21T19:51:55Z"}
traefik  | {"level":"error","entryPointName":"websecure","routerName":"2-router@http","error":"invalid middleware \"badger@http\" configuration: invalid middleware type or middleware does not exist","time":"2025-03-21T19:51:55Z"}
traefik  | {"level":"error","entryPointName":"websecure","routerName":"api-router@file","error":"invalid middleware \"crowdsec@file\" configuration: invalid middleware type or middleware does not exist","time":"2025-03-21T19:51:55Z"}
traefik  | {"level":"error","entryPointName":"websecure","routerName":"next-router@file","error":"invalid middleware \"crowdsec@file\" configuration: invalid middleware type or middleware does not exist","time":"2025-03-21T19:51:55Z"}
traefik  | {"level":"error","entryPointName":"websecure","routerName":"3-router@http","error":"invalid middleware \"badger@http\" configuration: invalid middleware type or middleware does not exist","time":"2025-03-21T19:51:55Z"}

You will have to share all your traefik files. Have your completely changed the folder structure?

Share the screenshot of the file/folder structure of your traefik files you adopted from above


Is this will be enough?


Is this will be enough?





Here are some screenshots

routing

http:
      # if you want add athentik on your service just add this on rour router
  routers:

    # HTTP to HTTPS redirect router
    main-app-router-redirect:
      rule: "Host(`pangolin.crowdsec.hhf.technology`)"
      service: next-service
      entryPoints:
        - web
      middlewares:
        - middlewares-https-redirectscheme

    # Next.js router (handles everything except API and WebSocket paths)
    next-router:
      rule: "Host(`pangolin.crowdsec.hhf.technology`) && !PathPrefix(`/api/v1`)"
      service: next-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

    # API router (handles /api/v1 paths)
    api-router:
      rule: "Host(`pangolin.crowdsec.hhf.technology`) && PathPrefix(`/api/v1`)"
      service: api-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

    # WebSocket router
    ws-router:
      rule: "Host(`pangolin.crowdsec.hhf.technology`)"
      service: api-service
      entryPoints:
        - websecure
      tls:
        certResolver: letsencrypt

# if you want add athentik on your service just add this on rour router

    #authentik-rtr:
      #service: authentik-svc
      #rule: Host(`{{ env "AUTHENTIK_HOST" }}`)
    
    #authentik-output-rtr:
      #service: authentik-svc
      #rule: HostRegexp(`{subdomain:[a-z0-9-]+}.{{ env "DOMAINNAME" }}`) && PathPrefix(`{{ env "AUTHENTIK_OUTPOST_PATH_PREFIX" }}`)

 

  services:

    next-service:
      loadBalancer:
        servers:
          - url: "http://pangolin:3002"  # Next.js server

    api-service:
      loadBalancer:
        servers:
          - url: "http://pangolin:3000"  # API/WebSocket server

 
  serversTransports:
    insecureTransport:
      insecureSkipVerify: true 
################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    ################################################################
    # Good Basic Security Practices
    ################################################################
    middlewares-secure-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlMaxAge: 100
        hostsProxyHeaders:
          - "X-Forwarded-Host"
        stsSeconds: 63072000
        stsIncludeSubdomains: true
        stsPreload: true
        forceSTSHeader: true
        # customFrameOptionsValue: "allow-from https:{{env "DOMAINNAME"}}" #CSP takes care of this but may be needed for organizr.
        customFrameOptionsValue: SAMEORIGIN # https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Frame-Options
        contentTypeNosniff: true
        browserXssFilter: true
        sslForceHost: true # add sslHost to all of the services
        sslHost: "*.testing.hhf.technology"
        referrerPolicy: "same-origin"
        permissionsPolicy: "camera=(), microphone=(), geolocation=(), payment=(), usb=()"
        customResponseHeaders:
          X-Robots-Tag: "none,noarchive,nosnippet,notranslate,noimageindex,noindex,nofollow" #global not tracking with websearch
          # X-Robots-Tag: "noindex,nofollow" "  # nextcloud recommandation 
          server: ""
          # https://community.traefik.io/t/how-to-make-websockets-work-with-traefik-2-0-setting-up-rancher/1732
          # X-Forwarded-Proto: "https"
################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    # DDoS Prevention
    middlewares-rate-limit:
      rateLimit:
        average: 100
        burst: 50
################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    # Middleware for Redirection
    # This can be used instead of global redirection
    middlewares-https-redirectscheme:
      redirectScheme:
        scheme: https
        permanent: true 
################################################################
# Dynamic configuration
################################################################
http:
  middlewares:
    middlewares-compress:
      compress:
        includedContentTypes:
          - application/json
          - text/html
          - text/plain
        minResponseBodyBytes: 1024
        defaultEncoding: gzip 
http:
  middlewares:
    default-headers:
      headers:
        frameDeny: true
        browserXssFilter: true
        contentTypeNosniff: true
        forceSTSHeader: true
        stsIncludeSubdomains: true
        stsPreload: true
        stsSeconds: 15552000
        customFrameOptionsValue: SAMEORIGIN
        customRequestHeaders:
          X-Forwarded-Proto: https
http:
  middlewares:
    crowdsec:
      plugin:
        crowdsec:
          clientTrustedIPs:
            - 10.0.0.0/8
            - 172.16.0.0/12
            - 192.168.0.0/16
            - 100.89.137.0/20
          crowdsecAppsecEnabled: true
          crowdsecAppsecFailureBlock: true
          crowdsecAppsecHost: crowdsec:7422
          crowdsecAppsecUnreachableBlock: true
          crowdsecLapiHost: crowdsec:8080
          crowdsecLapiKey: 82veNXhNok2cDycbjQUnHeQDQQiKqTNLfmFm29ljr34
          crowdsecLapiScheme: http
          crowdsecMode: live
          defaultDecisionSeconds: 15
          enabled: true
          forwardedHeadersTrustedIPs:
            - 0.0.0.0/0
          httpTimeoutSeconds: 10
          logLevel: INFO
          updateIntervalSeconds: 15
          updateMaxFailure: 0

image

################################################################
# API and Dashboard
################################################################

api:
  insecure: true
  dashboard: true

################################################################
# Providers - https://doc.traefik.io/traefik/providers/docker/
################################################################

providers:
  http:
    endpoint: "http://pangolin:3001/api/v1/traefik-config"
    pollInterval: "5s"
  file:
    directory: /rules
    watch: true

################################################################
# Plugins- Badger, Bouncer traefik  (crowdsec), souin
################################################################

experimental:
  plugins:
    badger:
      moduleName: "github.com/fosrl/badger"
      version: "v1.0.0"
    crowdsec:
      moduleName: "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
      version: "v1.3.5"
    # http cache for traefik
    souin:
      moduleName: github.com/darkweak/souin
      version: v1.6.50

################################################################
# Logs - https://doc.traefik.io/traefik/observability/logs/
################################################################

log:
  level: "INFO"
  format: "json"

################################################################
# Access logs - https://doc.traefik.io/traefik/observability/access-logs/
################################################################

accessLog:
  filePath: "/var/log/traefik/access.log"
  format: json
  filters:
    statusCodes:
      - "200-299"  # Success codes
      - "400-499"  # Client errors
      - "500-599"  # Server errors
    retryAttempts: true
    minDuration: "100ms"  # Increased to focus on slower requests
  bufferingSize: 100      # Add buffering for better performance
  fields:
    defaultMode: drop     # Start with dropping all fields
    names:
      ClientAddr: keep
      ClientHost: keep
      RequestMethod: keep
      RequestPath: keep
      RequestProtocol: keep
      DownstreamStatus: keep
      DownstreamContentSize: keep
      Duration: keep
      ServiceName: keep
      StartUTC: keep
      TLSVersion: keep
      TLSCipher: keep
      RetryAttempts: keep
    headers:
      defaultMode: drop
      names:
        User-Agent: keep
        X-Real-Ip: keep
        X-Forwarded-For: keep
        X-Forwarded-Proto: keep
        Content-Type: keep
        Authorization: redact  # Redact sensitive information
        Cookie: redact        # Redact sensitive information

################################################################
# Let's Encrypt (ACME)
################################################################

certificatesResolvers:
  letsencrypt:
    acme:
      httpChallenge:
        entryPoint: web
      email: "discourse@hhf.technology"
      storage: "/letsencrypt/acme.json"
      caServer: "https://acme-v02.api.letsencrypt.org/directory"
      #dnsChallenge:
        #provider: cloudflare
        #delayBeforeCheck: 30 # Default is 2m0s.  This changes the delay (in seconds)
        # Custom DNS server resolution
        #resolvers:
          #- "1.1.1.1:53"
          #- "8.8.8.8:53"


################################################################
# Entrypoints - https://doc.traefik.io/traefik/routing/entrypoints/
################################################################

entryPoints:
  web:
    address: ":80"
    forwardedHeaders:
      trustedIPs:
        # Cloudflare (https://www.cloudflare.com/ips-v4)
        - "173.245.48.0/20"
        - "103.21.244.0/22"
        - "103.22.200.0/22"
        - "103.31.4.0/22"
        - "141.101.64.0/18"
        - "108.162.192.0/18"
        - "190.93.240.0/20"
        - "188.114.96.0/20"
        - "197.234.240.0/22"
        - "198.41.128.0/17"
        - "162.158.0.0/15"
        - "104.16.0.0/13"
        - "104.24.0.0/14"
        - "172.64.0.0/13"
        - "131.0.72.0/22"
        # Local IPs
        - "127.0.0.1/32"
        - "10.0.0.0/8"
        - "192.168.0.0/16"
        - "172.16.0.0/12"  
  websecure:
    address: ":443"
    transport:
      respondingTimeouts:
        readTimeout: "30m"
    http:
      middlewares:                                                                   # CHANGE MADE HERE (BOUNCER ENABLED) !!!
        - crowdsec@file
        - middlewares-compress@file
        - middlewares-secure-headers@file
      tls:
        options: tls-opts@file
        certResolver: letsencrypt
    http3: {}
    forwardedHeaders:
      trustedIPs:
        # Cloudflare (https://www.cloudflare.com/ips-v4)
        - "173.245.48.0/20"
        - "103.21.244.0/22"
        - "103.22.200.0/22"
        - "103.31.4.0/22"
        - "141.101.64.0/18"
        - "108.162.192.0/18"
        - "190.93.240.0/20"
        - "188.114.96.0/20"
        - "197.234.240.0/22"
        - "198.41.128.0/17"
        - "162.158.0.0/15"
        - "104.16.0.0/13"
        - "104.24.0.0/14"
        - "172.64.0.0/13"
        - "131.0.72.0/22"
        # Local IPs
        - "127.0.0.1/32"
        - "10.0.0.0/8"
        - "192.168.0.0/16"
        - "172.16.0.0/12"


serversTransport:
  insecureSkipVerify: true

docker compose

name: pangolin
networks:
  default:
    driver: bridge
    name: pangolin
services:
  crowdsec:
    command: -t
    container_name: crowdsec
    environment:
      ACQUIRE_FILES: /var/log/traefik/*.log
      COLLECTIONS: crowdsecurity/traefik crowdsecurity/appsec-virtual-patching crowdsecurity/appsec-generic-rules
      ENROLL_INSTANCE_NAME: pangolin-crowdsec
      ENROLL_TAGS: docker
      GID: "1000"
      PARSERS: crowdsecurity/whitelists
    expose:
      - 6060
    healthcheck:
      test:
        - CMD
        - cscli
        - capi
        - status
    image: crowdsecurity/crowdsec:v1.6.6-rc5
    labels:
      - traefik.enable=false
    ports:
      - 6060:6060
    restart: unless-stopped
    volumes:
      - ./config/crowdsec:/etc/crowdsec
      - ./config/crowdsec/db:/var/lib/crowdsec/data
      - ./config/crowdsec_logs/auth.log:/var/log/auth.log:ro
      - ./config/crowdsec_logs/syslog:/var/log/syslog:ro
      - ./config/crowdsec_logs:/var/log
      - ./config/traefik/logs:/var/log/traefik
  gerbil:
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    command:
      - --reachableAt=http://gerbil:3003
      - --generateAndSaveKeyTo=/var/config/key
      - --remoteConfig=http://pangolin:3001/api/v1/gerbil/get-config
      - --reportBandwidthTo=http://pangolin:3001/api/v1/gerbil/receive-bandwidth
    container_name: gerbil
    depends_on:
      pangolin:
        condition: service_healthy
    image: fosrl/gerbil:1.0.0
    ports:
      - 51820:51820/udp
      - 443:443
      - 80:80
    restart: unless-stopped
    volumes:
      - ./config/:/var/config
  pangolin:
    container_name: pangolin
    healthcheck:
      interval: 3s
      retries: 5
      test:
        - CMD
        - curl
        - -f
        - http://localhost:3001/api/v1/
      timeout: 3s
    image: fosrl/pangolin:1.0.1
    restart: unless-stopped
    volumes:
      - ./config:/app/config
  traefik:
    command:
      - --configFile=/etc/traefik/traefik_config.yml
    container_name: traefik
    depends_on:
      pangolin:
        condition: service_healthy
    image: traefik:v3.3.3
    network_mode: service:gerbil
    restart: unless-stopped
    volumes:
      - ./config/traefik:/etc/traefik:ro
      - ./config/letsencrypt:/letsencrypt
      - ./config/traefik/logs:/var/log/traefik
      - ./config/traefik/rules:/rules # New addition
root@ip-10-0-13-223:~# docker compose logs -f
traefik   | {"level":"info","version":"3.3.3","time":"2025-03-22T02:09:21Z","mes                                                                             sage":"Traefik version 3.3.3 built on 2025-01-31T14:55:01Z"}
traefik   | {"level":"info","time":"2025-03-22T02:09:21Z","message":"\nStats col                                                                             lection is disabled.\nHelp us improve Traefik by turning this feature on :)\nMor                                                                             e details on: https://doc.traefik.io/traefik/contributing/data-collection/\n"}
traefik   | {"level":"info","plugins":["badger","crowdsec","souin"],"time":"2025                                                                             -03-22T02:09:21Z","message":"Loading plugins..."}
traefik   | {"level":"info","plugins":["badger","crowdsec","souin"],"time":"2025                                                                             -03-22T02:09:24Z","message":"Plugins loaded."}
traefik   | {"level":"info","time":"2025-03-22T02:09:24Z","message":"Starting pr                                                                             ovider aggregator *aggregator.ProviderAggregator"}
traefik   | {"level":"info","time":"2025-03-22T02:09:24Z","message":"Starting pr                                                                             ovider *file.Provider"}
traefik   | {"level":"info","time":"2025-03-22T02:09:24Z","message":"Starting pr                                                                             ovider *traefik.Provider"}
traefik   | {"level":"info","time":"2025-03-22T02:09:24Z","message":"Starting pr                                                                             ovider *http.Provider"}
traefik   | {"level":"info","time":"2025-03-22T02:09:24Z","message":"Starting pr                                                                             ovider *acme.ChallengeTLSALPN"}
traefik   | {"level":"info","time":"2025-03-22T02:09:24Z","message":"Starting pr                                                                             ovider *acme.Provider"}
traefik   | {"level":"info","providerName":"letsencrypt.acme","acmeCA":"https://                                                                             acme-v02.api.letsencrypt.org/directory","time":"2025-03-22T02:09:24Z","message":                                                                             "Testing certificate renew..."}
pangolin  |
pangolin  | > @fosrl/pangolin@0.0.0 start
pangolin  | > NODE_OPTIONS=--enable-source-maps NODE_ENV=development ENVIRONMENT                                                                             =prod sh -c 'node dist/migrations.mjs && node dist/server.mjs'
pangolin  |
pangolin  | Starting migrations from version 1.0.1
pangolin  | Migrations to run:
pangolin  | All migrations completed successfully
pangolin  | 2025-03-22T02:09:19.012Z [warn]: Email SMTP configuration is missing                                                                             . Emails will not be sent.
pangolin  | 2025-03-22T02:09:19.279Z [info]: Server admin (discourse@hhf.technol                                                                             ogy) already exists
pangolin  | 2025-03-22T02:09:19.833Z [info]: API server is running on http://loc                                                                             alhost:3000
pangolin  | 2025-03-22T02:09:19.833Z [info]: Internal server is running on http:                                                                             //localhost:3001
pangolin  | 2025-03-22T02:09:20.094Z [info]: Next.js server is running on http:/                                                                             /localhost:3002
gerbil    | INFO: 2025/03/22 02:09:20 Fetching remote config from http://pangoli                                                                             n:3001/api/v1/gerbil/get-config
gerbil    | INFO: 2025/03/22 02:09:20 Created WireGuard interface wg0
gerbil    | INFO: 2025/03/22 02:09:20 Assigned IP address 100.89.128.1/24 to int                                                                             erface wg0
gerbil    | INFO: 2025/03/22 02:09:20 Attempting to delete existing MSS clamping                                                                              rule for chain INPUT
gerbil    | INFO: 2025/03/22 02:09:20 Attempting to delete existing MSS clamping                                                                              rule for chain OUTPUT
gerbil    | INFO: 2025/03/22 02:09:20 Attempting to delete existing MSS clamping                                                                              rule for chain FORWARD
gerbil    | INFO: 2025/03/22 02:09:20 Adding MSS clamping rule for chain INPUT
gerbil    | INFO: 2025/03/22 02:09:20 Successfully added and verified MSS clampi                                                                             ng rule for chain INPUT
gerbil    | INFO: 2025/03/22 02:09:20 Adding MSS clamping rule for chain OUTPUT
gerbil    | INFO: 2025/03/22 02:09:20 Successfully added and verified MSS clampi                                                                             ng rule for chain OUTPUT
gerbil    | INFO: 2025/03/22 02:09:20 Adding MSS clamping rule for chain FORWARD
gerbil    | INFO: 2025/03/22 02:09:20 Successfully added and verified MSS clampi                                                                             ng rule for chain FORWARD
gerbil    | INFO: 2025/03/22 02:09:20 WireGuard interface wg0 created and config                                                                             ured
gerbil    | INFO: 2025/03/22 02:09:20 Starting server on :3003
crowdsec  | Local agent already registered
crowdsec  | Check if lapi needs to register an additional agent
crowdsec  | sqlite database permissions updated
crowdsec  | /etc/crowdsec was found in a volume
crowdsec  | Running hub update
crowdsec  | Skipping hub update, index file is recent
crowdsec  | /var/lib/crowdsec/data was found in a volume
crowdsec  | Running hub upgrade
crowdsec  | Action plan:
crowdsec  | 🔄 check & update data files
crowdsec  |
crowdsec  | Running: cscli  parsers install "crowdsecurity/docker-logs"
crowdsec  | Nothing to do.
crowdsec  | Running: cscli  parsers install "crowdsecurity/cri-logs"
crowdsec  | Nothing to do.
crowdsec  | Running: cscli  collections install "crowdsecurity/traefik"
crowdsec  | Nothing to do.
crowdsec  | Running: cscli  collections install "crowdsecurity/appsec-virtual-pa                                                                             tching"
crowdsec  | Nothing to do.
crowdsec  | Running: cscli  collections install "crowdsecurity/appsec-generic-ru                                                                             les"
crowdsec  | Nothing to do.
crowdsec  | Object parsers/crowdsecurity/whitelists is tainted, skipping
crowdsec  | time="2025-03-22T02:09:19Z" level=info msg="Enabled feature flags: n                                                                             one"
crowdsec  | time="2025-03-22T02:09:19Z" level=info msg="Crowdsec v1.6.6-rc5-416e                                                                             b27f"
crowdsec  | time="2025-03-22T02:09:19Z" level=info msg="Loading prometheus colle                                                                             ctors"
crowdsec  | time="2025-03-22T02:09:19Z" level=info msg="Loading CAPI manager"
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="CAPI manager configured                                                                              successfully"
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="Start push to CrowdSec C                                                                             entral API (interval: 16s once, then 10s)"
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="CrowdSec Local API liste                                                                             ning on 0.0.0.0:8080"
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="Start sending metrics to                                                                              CrowdSec Central API (interval: 43m28s once, then 30m0s)"
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="capi metrics: sending"
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="last CAPI pull is newer                                                                              than 1h30, skip."
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="Start pull from CrowdSec                                                                              Central API (interval: 1h59m43s once, then 2h0m0s)"
crowdsec  | time="2025-03-22T02:09:20Z" level=info msg="Loading grok library /et                                                                             c/crowdsec/patterns"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loading enrich plugins"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Successfully registered                                                                              enricher 'GeoIpCity'"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Successfully registered                                                                              enricher 'GeoIpASN'"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Successfully registered                                                                              enricher 'IpToRange'"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Successfully registered                                                                              enricher 'reverse_dns'"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Successfully registered                                                                              enricher 'ParseDate'"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Successfully registered                                                                              enricher 'UnmarshalJSON'"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loading parsers from 5 f                                                                             iles"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loaded 1 parser nodes" f                                                                             ile=/etc/crowdsec/parsers/s00-raw/cri-logs.yaml stage=s00-raw
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loaded 1 parser nodes" f                                                                             ile=/etc/crowdsec/parsers/s00-raw/docker-logs.yaml stage=s00-raw
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loaded 1 parser nodes" f                                                                             ile=/etc/crowdsec/parsers/s01-parse/appsec-logs.yaml stage=s01-parse
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loaded 1 parser nodes" f                                                                             ile=/etc/crowdsec/parsers/s01-parse/traefik-logs.yaml stage=s01-parse
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loaded 1 parser nodes" f                                                                             ile=/etc/crowdsec/parsers/s02-enrich/http-logs.yaml stage=s02-enrich
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loaded 5 nodes from 3 st                                                                             ages"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="No postoverflow parsers                                                                              to load"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loading 46 scenario file                                                                             s"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=rough-violet name=crowdsecurity/CVE-2017-9841
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =shy-river name=crowdsecurity/http-sensitive-files
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=throbbing-forest name=crowdsecurity/pulse-secure-sslvpn-cve-2019-11510
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=morning-water name=crowdsecurity/CVE-2023-22515
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =restless-frog name=crowdsecurity/http-admin-interface-probing
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =bold-brook name=crowdsecurity/http-crawl-non_statics
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=weathered-star name=crowdsecurity/http-cve-2021-42013
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=silent-resonance name=crowdsecurity/CVE-2022-26134
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =solitary-smoke name=crowdsecurity/http-probing
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =dawn-flower name=crowdsecurity/http-xss-probbing
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=dawn-silence name=crowdsecurity/CVE-2022-41082
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=wild-leaf name=crowdsecurity/jira_cve-2021-26086
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=little-thunder name=crowdsecurity/fortinet-cve-2022-40684
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=misty-sunset name=crowdsecurity/CVE-2019-18935
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =spring-forest name=crowdsecurity/http-backdoors-attempts
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =aged-dawn name=crowdsecurity/http-bad-user-agent
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=fragrant-pine name=crowdsecurity/thinkphp-cve-2018-20062
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=divine-sky name=crowdsecurity/CVE-2022-42889
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=morning-meadow name=crowdsecurity/http-open-proxy
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=silent-butterfly name=crowdsecurity/apache_log4j2_cve-2021-44228
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =rough-darkness name=crowdsecurity/CVE-2022-41697
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=summer-voice name=crowdsecurity/http-cve-probing
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=black-voice name=crowdsecurity/CVE-2024-0012
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=fragrant-cloud name=crowdsecurity/netgear_rce
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=misty-leaf name=crowdsecurity/http-cve-2021-41773
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=fragrant-shadow name=crowdsecurity/fortinet-cve-2018-13379
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=divine-cloud name=crowdsecurity/CVE-2023-49103
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =still-voice name=crowdsecurity/http-sqli-probbing-detection
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =summer-firefly name=crowdsecurity/CVE-2022-46169-bf
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=twilight-flower name=crowdsecurity/CVE-2022-46169-cmd
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=holy-lake name=crowdsecurity/vmware-cve-2022-22954
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =floral-sea name=crowdsecurity/appsec-vpatch
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =red-shadow name=crowdsecurity/http-generic-bf
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =dark-wave name=LePresidente/http-generic-401-bf
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =crimson-tree name=LePresidente/http-generic-403-bf
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=young-snowflake name=crowdsecurity/vmware-vcenter-vmsa-2021-0027
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=purple-flower name=crowdsecurity/CVE-2022-37042
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=misty-glitter name=crowdsecurity/CVE-2022-35914
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=falling-pond name=crowdsecurity/CVE-2024-9474
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =wild-dawn name=crowdsecurity/CVE-2024-38475
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =patient-moon name=crowdsecurity/appsec-native
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =cool-leaf name=crowdsecurity/http-wordpress-scan
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=icy-dawn name=ltsich/http-w00tw00t
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=icy-sun name=crowdsecurity/CVE-2022-44877
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=wandering-breeze name=crowdsecurity/CVE-2023-22518
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=muddy-meadow name=crowdsecurity/grafana-cve-2021-43798
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=lively-forest name=crowdsecurity/f5-big-ip-cve-2020-5902
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding trigger bucket" c                                                                             fg=morning-frog name=crowdsecurity/spring4shell_cve-2022-22965
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding leaky bucket" cfg                                                                             =empty-butterfly name=crowdsecurity/http-path-traversal-probing
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Loaded 49 scenarios"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding crowdsecurity/vpa                                                                             tch-CVE-2018-1000861 to appsec rules"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding crowdsecurity/vpa                                                                             tch-CVE-2020-11738 to appsec rules"
crowdsec  | time="2025-03-22T02:09:21Z" level=info msg="Adding crowdsecurity/vpa                                                                             tch-CVE-
1 Like

i hope it solved it.

1 Like

Thanks, spelling can be my achilles heel sometimes.

1 Like

It seems the error was on the installer script it wasn’t generating the API key of crowsec bouncer and left by default the field crowdsecLapiKey.

I generated manually following a guide of yours and documentation of crowdsec and now finally it works by default from fossorial config.

Will do a backup and apply your guides.

Yes it was missing crowdsecLapiKey and some missing fields that still had your data and stuff.

So now is working. It feels good. A question is this enough rules or one of them is redundant?

1 Like

its case specific. you can add middleware or remove as per scenarios

1 Like

This looks promising as the base pangolin install doesnt come with much security on the traefik side. How does this work when pangolin gets updated? Do I need to set this all up again?

1 Like

It has nothing to do with pangolin. It all about traefik. We don’t touch pangolin directory or db so you can update the tags as usual.

Are there any changes to this guide if we have middleware manager setup?