Enhancing Your Pangolin Deployment with Middleware Manager

Enhancing Your Pangolin Deployment with Middleware Manager: A Complete Guide



Use Github Repo to open Issues

Please note that this was created for my personal use. I cannot accept responsibility if it affects your working environment. This has been tested with the applications I use and it works flawlessly.

:warning: Security Warning: Use Middlewares Responsibly

IMPORTANT: While middlewares can significantly enhance your security posture, they can also introduce serious vulnerabilities if misconfigured. Here are some critical warnings:

  1. Authentication Leakage: Improper forward authentication configurations can leak credentials or bypass authentication entirely.
  2. Rate Limiting Bypass: Incorrectly configured rate limiters might be bypassed using various techniques, leaving your services vulnerable to DDoS attacks.
  3. Header Injection: Poorly configured header middlewares can expose your applications to XSS or CSRF attacks.
  4. IP Whitelisting Risks: Relying solely on IP whitelisting can create a false sense of security, especially with sophisticated attackers who can spoof IP addresses.
  5. Testing Is Essential: Always test new middleware configurations in a staging environment before deploying to production.
  6. Provider Reference Confusion: Mixing up provider references (@http vs @file) can lead to middleware chains that don’t apply as expected.
  7. Performance Impact: Stacking too many middlewares can significantly impact performance, especially complex middlewares like authentication.

Always thoroughly understand any middleware before implementing it in production. Take time to read Traefik’s documentation on each middleware type before using it.

Pangolin has revolutionized how we handle tunneled reverse proxies with its elegant approach to identity and access management. But what if you need more advanced security features like rate limiting, IP filtering, or integration with third-party authentication systems? That’s where the Pangolin Middleware Manager comes in – a powerful extension that lets you harness the full capabilities of Traefik middlewares without modifying Pangolin itself.

What is Pangolin Middleware Manager?

Pangolin Middleware Manager is a specialized microservice that seamlessly integrates with your existing Pangolin deployment. It provides a simple web interface to attach additional Traefik middlewares to your resources, enabling advanced functionality that wasn’t previously possible without custom configuration.

With Middleware Manager, you can implement:

  • External authentication systems (Authelia, Authentik, JWT)
  • Security headers and content security policies
  • Geographic IP blocking
  • Rate limiting and DDoS protection
  • Custom redirect and path manipulation rules
  • Integration with security tools like CrowdSec

The real power comes from being able to apply these middlewares to specific resources without disrupting Pangolin’s core functionality.

Prerequisites

Before we begin, make sure you have:

  • A working Pangolin deployment (version 1.0.0 or later)
  • Docker and Docker Compose
  • Basic understanding of Traefik concepts
  • Administrative access to your Pangolin server

Deployment Guide

Adding Middleware Manager to your existing Pangolin setup is straightforward. Let’s break it down into simple steps:

Step 1: Update Your Docker Compose File

First, you’ll need to modify your docker-compose.yml file. Here’s what you need to add:

middleware-manager:
  image: hhftechnology/middleware-manager:latest
  container_name: middleware-manager
  restart: unless-stopped
  volumes:
    - ./data:/data
    - ./config/traefik/rules:/conf
    - ./config/middleware-manager/templates.yaml:/app/config/templates.yaml  # Optional for custom templates
  environment:
    - PANGOLIN_API_URL=http://pangolin:3001/api/v1
    - TRAEFIK_CONF_DIR=/conf
    - DB_PATH=/data/middleware.db
    - PORT=3456
  ports:
    - "3456:3456"

Step 2: Create Required Directories

You’ll need to create a directory for Traefik’s dynamic configuration files:

mkdir -p ./config/traefik/rules
mkdir -p ./config/middleware-manager

Then move your dynamic_config.yml in the rules folder

Step 3: Update Traefik Configuration

For the Middleware Manager to work correctly, Traefik needs to load configuration from the /rules directory. Make sure your Traefik service has this volume mounted:

traefik:
  # Other configuration...
  volumes:
    - ./config/traefik:/etc/traefik:ro
    - ./config/letsencrypt:/letsencrypt
    - ./config/traefik/logs:/var/log/traefik
    - ./config/traefik/rules:/rules  # Add this line

Also, you need to update your Traefik static configuration (traefik_config.yml) to include the file provider for the rules directory. Add the following to your providers section:

providers:
  # Existing providers...
  file:
    directory: "/rules"
    watch: true

Step 4: Add Badger Plugin Configuration

The Middleware Manager relies on Traefik’s ability to use the Badger plugin. Make sure your Traefik configuration includes:

experimental:
  plugins:
    badger:
      moduleName: "github.com/fosrl/badger"
      version: "v1.0.0"

Step 5: Start the Services

Now you can start your updated stack:

docker compose up -d

Step 6: Access the Middleware Manager UI

Once everything is running, you can access the Middleware Manager UI at:

http://your-server:3456

Use Template as much as possible dont use dashboard unless you know how to use json format for the same.

Understanding How It Works

The Middleware Manager operates by:

  1. Monitoring resources created in Pangolin through its API
  2. Allowing you to attach middlewares to specific resources via the web interface
  3. Generating Traefik configuration files that properly reference services defined by Pangolin
  4. Ensuring middlewares are applied with the correct provider references

This approach keeps Pangolin’s configuration clean while extending its capabilities through Traefik’s powerful middleware system.

Common Middleware Configurations

Authentication with Authelia

middlewares:
  - id: "authelia"
    name: "Authelia Authentication"
    type: "forwardAuth"
    config:
      address: "http://authelia:9091/api/verify?rd=https://auth.yourdomain.com"
      trustForwardHeader: true
      authResponseHeaders:
        - "Remote-User"
        - "Remote-Groups"
        - "Remote-Name"
        - "Remote-Email"

Basic Rate Limiting

middlewares:
  - id: "rate-limit"
    name: "Basic Rate Limiting"
    type: "rateLimit"
    config:
      average: 100
      burst: 50

Security Headers

middlewares:
  - id: "security-headers"
    name: "Strong Security Headers"
    type: "headers"
    config:
      customResponseHeaders:
        Server: ""
        X-Powered-By: ""
      browserXSSFilter: true
      contentTypeNosniff: true
      customFrameOptionsValue: "SAMEORIGIN"
      forceSTSHeader: true
      stsIncludeSubdomains: true
      stsSeconds: 63072000

Debugging Common Issues

When using Middleware Manager, you might encounter some challenges. Here’s how to troubleshoot them:

1. “The service does not exist” error in Traefik logs

This usually means the cross-provider reference isn’t working correctly.

Solution:

  1. Verify the configuration file was generated correctly: cat ./config/traefik/rules/resource-overrides.yml
  2. Check if service references include the @http suffix
  3. Restart the Middleware Manager: docker restart middleware-manager

2. “The middleware does not exist” error in Traefik logs

Similar to the service error, but for middlewares:

Solution:

  1. Verify the middleware is properly defined in the configuration
  2. Ensure Pangolin-defined middlewares have an @http suffix when referenced
  3. Check if the middleware requires a Traefik plugin that isn’t installed

3. No changes appearing after adding middleware

If you’ve added a middleware but don’t see changes applied:

Solution:

  1. Check Traefik’s dashboard (if enabled) to see if the routes are correctly defined
  2. Look at the Middleware Manager logs: docker logs middleware-manager
  3. Verify the middleware priority (lower numbers have higher precedence)
  4. Restart Traefik: docker restart traefik

4. Middleware Manager UI not showing resources

If the UI doesn’t display your Pangolin resources:

Solution:

  1. Verify connectivity between Middleware Manager and Pangolin: docker exec middleware-manager curl -f http://pangolin:3001/api/v1
  2. Check if the Pangolin API URL is correctly set in the environment variables
  3. Restart the Middleware Manager container

5. Database Errors

If you encounter database-related errors:

Solution:

  1. Check permissions on the data directory: ls -la ./data
  2. Ensure the database is properly initialized: docker logs middleware-manager | grep -i database
  3. As a last resort, you can reset the database: rm ./data/middleware.db (this will remove all your middleware configurations)

Final Thoughts

The Pangolin Middleware Manager bridges the gap between Pangolin’s elegant simplicity and Traefik’s powerful middleware capabilities. By installing this extension, you gain a flexible security layer that can adapt to your specific needs without sacrificing the user-friendly experience that makes Pangolin so valuable.

Middlewares are powerful tools that require careful configuration and testing. Start simple, test thoroughly, and gradually expand your security posture as you become more comfortable with the system.

Happy proxying!

Create A Resource for Middleware-manager and keep the protection on

5 Likes

This looks great. I love the Security warning before introduction to anything else. I will test it out this week or maybe tonight and report back.

1 Like

sure.. will love to hear the feedback. was working on this past month

1 Like

This looks really useful. I’ll test it when I could find some time next week. Thanks @hhf.technoloy

1 Like

Pangolin Middleware Manager Wiki

Advanced Middleware Configurations and Use Cases

Understanding Middleware Configuration Structure

Before diving into specific examples, it’s important to understand how middleware configurations are structured in Traefik. Each middleware type requires a specific configuration format:

  • Configurations are JSON objects with properties specific to the middleware type
  • For complex middlewares, the structure can be nested with arrays and sub-objects
  • Some middleware types (like headers) have a large number of optional properties
  • Special attention must be paid to types (booleans, strings, numbers, arrays)

The Middleware Manager’s web interface provides template configurations for each middleware type, but understanding the underlying structure helps you customize them effectively.

Authentication Middleware Configurations

Forward Authentication with Authelia

Authelia is a popular authentication and authorization server. Here’s how to configure it with the Middleware Manager:

{
  "address": "http://authelia:9091/api/verify?rd=https://auth.yourdomain.com",
  "trustForwardHeader": true,
  "authResponseHeaders": [
    "Remote-User",
    "Remote-Groups",
    "Remote-Name",
    "Remote-Email"
  ]
}

This configuration:

  • Forwards authentication requests to the Authelia server
  • Includes a redirect URL for failed authentication
  • Trusts X-Forwarded-* headers for proper client IP detection
  • Specifies which headers from Authelia should be passed to the backend service

Forward Authentication with Authentik

Similar to Authelia, Authentik is a flexible authentication platform:

{
  "address": "http://authentik:9000/outpost.goauthentik.io/auth/traefik",
  "trustForwardHeader": true,
  "authResponseHeaders": [
    "X-authentik-username",
    "X-authentik-groups",
    "X-authentik-email",
    "X-authentik-name",
    "X-authentik-uid"
  ]
}

The key differences are the endpoint URL and the specific header names that Authentik uses.

Basic Authentication

For simpler needs, basic authentication provides username/password protection:

{
  "users": [
    "admin:$apr1$H6uskkkW$IgXLP6ewTrSuBkTrqE8wj/"
  ]
}

The passwords must be hashed. You can generate these hashed passwords using htpasswd:

htpasswd -nbm username password

Security Headers Configuration

Security headers protect your applications from various web vulnerabilities:

{
  "browserXssFilter": true,
  "contentTypeNosniff": true,
  "customFrameOptionsValue": "SAMEORIGIN",
  "customResponseHeaders": {
    "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
    "Referrer-Policy": "no-referrer",
    "X-Content-Type-Options": "nosniff",
    "X-Download-Options": "noopen",
    "X-Permitted-Cross-Domain-Policies": "none",
    "X-Robots-Tag": "none",
    "X-XSS-Protection": "1; mode=block"
  }
}

This configuration:

  • Enables browser XSS protection
  • Prevents MIME type sniffing
  • Restricts iframe embedding to the same origin
  • Sets a strong HSTS policy
  • Configures various security headers for robust protection

Rate Limiting

Rate limiting helps prevent abuse and DoS attacks:

{
  "average": 100,
  "burst": 200
}

This configuration:

  • Allows an average of 100 requests per second
  • Allows bursts of up to 200 requests
  • Automatically rejects requests that exceed these limits

IP Whitelist

IP whitelisting restricts access to specified IP ranges:

{
  "sourceRange": [
    "10.0.0.0/8",
    "192.168.1.0/24",
    "127.0.0.1/32"
  ]
}

This configuration allows access only from the specified private IP ranges and localhost.

RegEx Path Replacement

RegEx path replacement enables powerful URL transformations:

{
  "regex": "^/catalog/([0-9]+)/details",
  "replacement": "/product.php?id=$1"
}

This configuration transforms URLs like /catalog/12345/details to /product.php?id=12345, which is especially useful for legacy applications or clean URL structures.

Full Application Examples

Let’s explore complete middleware configurations for common applications:

Nextcloud Configuration

Nextcloud requires several middlewares for optimal functionality:

  1. WebDAV Redirect Middleware (replacepathregex):
{
  "regex": "^/.well-known/ca(l|rd)dav",
  "replacement": "/remote.php/dav/"
}
  1. Security Headers for Nextcloud:
{
  "browserXssFilter": true,
  "contentTypeNosniff": true,
  "customFrameOptionsValue": "SAMEORIGIN",
  "customResponseHeaders": {
    "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
    "Referrer-Policy": "no-referrer",
    "X-Content-Type-Options": "nosniff",
    "X-Download-Options": "noopen",
    "X-Permitted-Cross-Domain-Policies": "none",
    "X-Robots-Tag": "none",
    "X-XSS-Protection": "1; mode=block"
  }
}
  1. Rate Limiting:
{
  "average": 100,
  "burst": 200
}

You can combine these into a chain middleware for complete Nextcloud protection.

WordPress Configuration

WordPress benefits from custom headers and path handling:

  1. Headers Middleware:
{
  "browserXssFilter": true,
  "contentTypeNosniff": true,
  "customFrameOptionsValue": "SAMEORIGIN",
  "customResponseHeaders": {
    "X-Robots-Tag": "index, follow",
    "Referrer-Policy": "strict-origin-when-cross-origin"
  }
}
  1. Path Regex Redirect for WP Admin:
{
  "regex": "^/wp-admin$",
  "replacement": "/wp-admin/",
  "permanent": true
}

This configuration ensures proper SEO handling and fixes a common WordPress issue where accessing /wp-admin without a trailing slash causes problems.

Authentication Chain Example

For advanced security, you can create an authentication chain that combines multiple middleware types:

  1. IP Whitelist Middleware:
{
  "sourceRange": [
    "10.0.0.0/8",
    "192.168.1.0/24",
    "127.0.0.1/32"
  ]
}
  1. Forward Auth Middleware (Authelia):
{
  "address": "http://authelia:9091/api/verify?rd=https://auth.example.com",
  "trustForwardHeader": true,
  "authResponseHeaders": [
    "Remote-User",
    "Remote-Groups",
    "Remote-Email"
  ]
}
  1. Chain Middleware:
    • Create a middleware with type “Middleware Chain”
    • Select both middlewares in order: IP Whitelist first, then Authelia

This chain will first check if the IP is allowed, and only then authenticate the user, combining network-level and user-level security.

Advanced RegEx Patterns for URL Transformation

The replacepathregex middleware is incredibly powerful. Here are some advanced patterns:

Capturing Groups

{
  "regex": "^/api/v1/(.*)",
  "replacement": "/api/v2/$1"
}

This captures everything after /api/v1/ and passes it to the replacement.

OR Conditions

{
  "regex": "^/.well-known/ca(l|rd)dav",
  "replacement": "/remote.php/dav/"
}

This matches either caldav or carddav.

Multiple Capture Groups

{
  "regex": "^/products/([0-9]+)/reviews/([0-9]+)",
  "replacement": "/api/products/$1/reviews/$2"
}

This extracts both product ID and review ID from the URL path.

Path Transformations with Query Parameters

{
  "regex": "^/legacy/([^/]+)/(.*)",
  "replacement": "/new/$2?category=$1"
}

This converts a path structure to include query parameters.

Implementation Best Practices

When implementing these advanced configurations, keep these best practices in mind:

  1. Start simple and test: Begin with basic middleware configurations and add complexity gradually, testing at each step.

  2. Use middleware chains strategically: Chains allow you to apply multiple middlewares in a specific order and reuse the combination.

  3. Consider middleware priority: Remember that lower priority numbers run first, so authentication should generally have a lower number than other middlewares.

  4. Be careful with regex patterns: Test regex patterns thoroughly to ensure they match exactly what you intend and don’t cause unexpected behavior.

  5. Document your configurations: For complex setups, maintain documentation explaining the purpose and behavior of each middleware.

  6. Monitor performance impact: Some middlewares (especially those with complex regex or external authentication) can impact performance, so monitor your system after implementing them.

1 Like

Okay this is suuuuper good, im going to install it :laughing: (Yes, i am the girl that loves UI)

1 Like

Perhaps a dumb question…but any reason this won’t / shouldn’t be built into the core Pangolin dashboard?

5 Likes

question with this will i be able to replicate the middlewares for headscale,

CORS and pathprefix?

1 Like

i didn’t have an issue with headscale deployment at all. Can you share a bit more details in what going on at your end with some logs or console screenshots from the browser so that i can help on the same.

This is a great project. Something I’ve been waiting for!

I would have some questions (for confirming if I have misconfigured something):

  1. I would like you to clarify the role of the templates.yaml file. If I understand correctly, Middleware Manager will only load the middleware from this file if id does not exist in its database? After the middleware appears in the webUI, can I still edit the templates.yaml or should I edit the json in the UI? And, if I edit the middleware in the UI, will (should) it be synced back to the templates.yaml?

  2. Does Middleware Manager detect existing middlewares in dynamic configurations?

  3. The status of each Resouce is “Not Protected” in Middleware Manager webUI. Does this mean only that no middleware was assigned? And nothing to do with Pangolin’s protection?

Also, a feature request, if possible: adding a middleware to multiple Resources at the same time. Now, that this application made traefik plugin testing easier by a mile, it’s became tedious to add, for example, GeoBlock to each Resource manually.

1 Like

I will always recommend using templates for major additions. for minor additions like key and few number tweaks you can use in ui edit.
Once you edit the template restart is must for the middleware docker container.
And, if I edit the middleware in the UI, will (should) it be synced back to the templates.yaml? NO. templates have nothing to do with ui changes. UI changes are recorded in database.

No. It doesn’t mess with the default configs. so, one fine day if you just want to remove it all your resources will work as default.

It has nothing to do with pangolin. neither we touch the database, nor we edit any files in your pangolin stack, it’s a separate entity which deals with traefik and gives you ability to add and remove different functions.

new update coming soon, if you are on a test bench then see all the features here hhftechnology/middleware-manager:sni-support

Thank you! I will check it.

1 Like

After following the guide through creating a Pangolin resource for the Middleware manager, should we revise the compose file to no longer publish port 3456?

1 Like

If you wish. It’s just for UI. For production I don’t have a domain attached to it, is just accessible via private network. You can take all measures to make it secure

how would you add sablier through this? that’s one plugin i see as valuable having a service-based switch to activate

I’m currently running 30 containers on a rk3588 board and the only bottleneck is ram, load avg = 0.5

1 Like

Hi. New to the forum. Thank you for all your hard work. I am having some errors whenever I follow this guide to install MW manager then try to run the crowdsec management script. It appears the script is looking for the dynamic config to be in traefik folder and not the rules subfolder. Is there anyway for me to fix this?

1 Like

no, not correct.



In rules folder you can have any permutation combination but you can’t have duplicate middlewares. Either you use MM

@hhf.technoloy Thanks for this.

How can I add this in a separate docker-compose and in its own dedicated directory to keep the Pangolin directory clean and tidy?

I could add this into the same docker network …

1 Like