CrowdSec Manager for Pangolin: User Guide

CrowdSec Manager for Pangolin: Complete User Guide

With traefik logs Update and Back Integrated Version.02

Tested on Debian, Alma, and Ubuntu 24.04(Prefered)

Run this command if you are using Debian or Ubuntu before script execution

sudo apt install grep gawk sed tar

CrowdSec Manager is an all-in-one tool designed to simplify the management of your CrowdSec installation within a Docker environment tailored for Pangolin. This comprehensive guide walks you through utilizing the script to oversee various facets of your CrowdSec setup, from basic health diagnostics to advanced configurations such as captcha protection, custom security scenarios, and IP management.

Prerequisites

Before diving into the script, ensure the following requirements are met:

  • Docker Compose Stack of Pangolin: Installed and operational.
  • Working Setup: Includes CrowdSec, Traefik, Gerbil, and Pangolin components.
  • Bash Shell Environment: Available on your system.
  • Basic Terminal Knowledge: Familiarity with command-line operations.

Getting Started

1. Installing the Script

To begin using the CrowdSec Manager script:

  1. Download the Script:

    curl -o setup_crowdsec_manager.sh https://gist.githubusercontent.com/hhftechnology/aadadf48ac906fc38cfd0d7088980475/raw/0a384d518e74c9963a51fcfb60d5ef5bccf9f645/setup_crowdsec_manager.sh
    
  2. Make it Executable:

    chmod +x setup_crowdsec_manager.sh
    
  3. Run the Script:
    Execute the script in the directory containing your docker-compose.yml file for Pangolin:

    ./setup_crowdsec_manager.sh
    

    Note: If you encounter Docker permission issues, prepend sudo:

    sudo ./setup_crowdsec_manager.sh
    

2. Initial Health Check

Upon launching, the script automatically verifies the status of essential containers (CrowdSec, Traefik, Pangolin, Gerbil):

  • Success: Displays the main menu if all containers are running.
  • Failure: Alerts you if any container is missing or stopped, potentially exiting to prompt corrective action.

Main Features

The script’s main menu organizes functionalities into four categories: System Health & Diagnostics, IP Management, Advanced Configuration, and Logs & Monitoring.

System Health & Diagnostics

1. Check System Health (Option 1)

Quickly assess the health of your CrowdSec stack:

  1. Select 1 from the main menu.
  2. The script checks the status of CrowdSec, Traefik, Pangolin, and Gerbil containers.
  3. Output:
    • Green [+] indicates a running container.
    • Red [-] signals a non-running container.
  4. Press Enter to return to the main menu.

2. Run Complete Diagnostic Check (Option 2)

Perform an in-depth diagnostic of your CrowdSec setup:

  1. Choose 2 from the main menu.
  2. The script executes a series of checks:
    • Container health
    • Bouncer registration
    • Active decisions
    • Metrics availability
    • Traefik integration
    • Key configuration settings
  3. Review detailed outputs for each check.
  4. A summary verdict confirms if CrowdSec is functioning correctly.
  5. Press Enter to return to the main menu.

3. Check CrowdSec Bouncers (Option 3)

Verify registered bouncers enforcing CrowdSec decisions:

  1. Select 3 from the main menu.
  2. Lists all registered bouncers.
  3. Confirms Traefik bouncer registration specifically.
  4. Press Enter to return to the main menu.

4. Check CrowdSec Metrics (Option 4)

Examine CrowdSec’s performance metrics:

  1. Choose 4 from the main menu.
  2. Displays:
    • Prometheus metrics (first 10 CrowdSec-related lines)
    • AppSec-specific metrics
    • Internal CrowdSec metrics
  3. Ensures data collection is active.
  4. Press Enter to return to the main menu.

5. Check Traefik CrowdSec Integration (Option 5)

Validate Traefik’s integration with CrowdSec:

  1. Select 5 from the main menu.
  2. Searches for CrowdSec middleware in various Traefik config files.
  3. Reports middleware presence.
  4. Press Enter to return to the main menu.

IP Management

6. Check CrowdSec Decisions (List Blocked IPs) (Option 6)

View active security decisions:

  1. Choose 6 from the main menu.
  2. Lists blocked IPs and reasons.
  3. Note: An empty list indicates no detected malicious activity yet.
  4. Press Enter to return to the main menu.

7. IP Whitelisting Management (Option 7)

Manage IP whitelists to exempt trusted sources:

  1. Select 7 from the main menu.
  2. Access a submenu:
    • 1. Whitelist current public IP
    • 2. Whitelist a specific IP
    • 3. Set up comprehensive whitelist with standard private networks
    • 4. View currently whitelisted IPs
    • 0. Return to main menu
  3. Options:
    • 1: Detects and whitelists your public IP in CrowdSec and/or Traefik.
    • 2: Prompts for an IP to whitelist in CrowdSec and/or Traefik.
    • 3: Configures a whitelist with defaults:
      - "192.168.0.0/16"    # Common internal network
      - "10.0.0.0/8"        # Private range
      - "172.16.0.0/12"     # Docker network range
      - "100.89.137.0/20"   # From Pangolin configuration
      
      Allows additional IPs via prompt (one per line, Ctrl+D to finish).
    • 4: Displays whitelisted IPs in CrowdSec and Traefik configs.
  4. Follow prompts; CrowdSec restarts if changes are made.
  5. Press Enter to return to the main menu.

8. Unban an IP (Option 8)

Remove bans from specific IPs:

  1. Choose 8 from the main menu.
  2. Enter the IP to unban.
  3. If banned, the script removes the decision; if not, it notifies you.
  4. Press Enter to return to the main menu.

9. Check IP Security Status (Option 9)

Analyze an IP’s security status:

  1. Select 9 from the main menu.
  2. Enter the IP to check.
  3. Reports:
    • Block status in CrowdSec
    • Whitelist status in CrowdSec (including subnets)
    • Whitelist status in Traefik (including subnets)
  4. Press Enter to return to the main menu.

Advanced Configuration

10. Enroll with CrowdSec Console (Option 10)

Link your CrowdSec instance to the Console:

  1. Visit CrowdSec Console, log in, and get your enrollment key.

  2. Choose 10 from the main menu.
  3. Paste the enrollment key.
  4. The script enrolls and restarts CrowdSec.
  5. Accept the ‘pangolin-crowdsec’ instance in the Console.
  6. Important: After enrollment, go back to the CrowdSec Console and accept the instance named ‘pangolin-crowdsec’
  7. Press Enter to enable additional features and check status.
  8. Press Enter again to return to the main menu.

11. Set Up Custom Scenarios (Option 11)

Define custom security rules:

  1. Select 11 from the main menu.
  2. Confirm installation.
  3. Installs scenarios for:
    • Authentication brute force
    • API abuse
    • Resource scanning
    • HTTP flooding
  4. CrowdSec restarts; verify scenario loading.
  5. Caution: Monitor decisions to avoid false positives.
  6. Press Enter to return to the main menu.

12. Set Up Captcha Protection (Option 12)

Implement Cloudflare Turnstile captcha:

  1. Create a Non-Interactive Turnstile site at Cloudflare Dashboard.
  2. Choose 12 from the main menu.
  3. If configured, decide to overwrite.

–very imp Select Non-Interactive only.


4. Enter Site Key and Secret Key.
5. The script:

  • Sets up a captcha profile
  • Creates an HTML template
  • Updates Traefik middleware
  1. Restarts Traefik and CrowdSec.
  2. Test with:
    docker exec crowdsec cscli decisions add --ip <test-ip> --type captcha -d 1h
    
  3. Press Enter to return to the main menu.

Logs & Monitoring

13. View Recent CrowdSec Logs (Option 13)

Examine recent CrowdSec logs:

  1. Select 13 from the main menu.
  2. Displays the last 50 lines.
  3. Press Enter to return to the main menu.

14. View Recent Traefik Logs (Option 14)

Check recent Traefik logs:

  1. Choose 14 from the main menu.
  2. Shows the last 50 lines.
  3. Press Enter to return to the main menu.

15. Follow CrowdSec Logs (Live) (Option 15)

Monitor CrowdSec logs in real-time:

  1. Select 15 from the main menu.
  2. View live logs.
  3. Press Ctrl+C to exit and return to the main menu.

Common Issues and Troubleshooting

CrowdSec Container Won’t Start

  • Check: Use Option 13 for logs.
  • Fix: Look for config errors in scenarios/profiles; ensure whitelist formatting.

Captcha Not Working

  • Verify: Correct Turnstile keys, template at ./config/traefik/conf/captcha.html, middleware (Option 5).
  • Test: Apply a manual captcha decision.

Traefik Not Blocking Attacks

  • Check: Bouncer registration (Option 3), API key match, middleware config (Option 5).

False Positives in Decisions

  • Steps:
    1. Check decisions (Option 6).
    2. Unban IP (Option 8) if needed.
    3. Add to whitelist (Option 7).
    4. Adjust custom scenarios.

Best Practices

  1. Regular Monitoring: Use Option 6 to track decisions.
  2. Testing Changes: Run Option 2 post-configuration.
  3. Whitelist Management: Update via Option 7 to avoid false positives.
  4. Console Integration: Enroll with Option 10 for enhanced oversight.
  5. Backup Configurations: Save configs before changes; note script backups.
  6. Update Regularly: Keep CrowdSec and bouncers current.
  7. Log Review: Use Options 13-15 to monitor events and health.

Conclusion

The CrowdSec Manager script streamlines the administration of your CrowdSec security stack, enabling robust protection for your Pangolin environment. This guide equips you to leverage its full feature set, ensuring your infrastructure remains secure against cyber threats. Security is an ongoing effort—consistently review settings, monitor logs, and stay updated on threats for optimal defense.

8 Likes

Will update more features to this script later this month. It’s an evolved version of the trouble shooting script.

2 Likes

When I run this directly after a sudo docker compose up -d command on my Pangolin VPS, I get a notice saying that traefik container is NOT running…

user@vps:~# sudo docker compose up -d
[+] Running 5/5
:check_mark: Network pangolin Created
:check_mark: Container crowdsec Started
:check_mark: Container pangolin Healthy
:check_mark: Container gerbil Started
:check_mark: Container traefik Started

user@vps:~# sudo ./crowdsec-manager.sh
/======================================================================
CHECKING PREREQUISITES
/======================================================================

[+] Docker is running
[+] crowdsec container is running
[-] traefik container is NOT running
[*] Temporary files cleaned up.

I am working on a fix. Please wait

1 Like

error resolved please try and let me know.

Working now, thank you for all you do!

1 Like

Polished few bugs today also.

I’m being banned 100% of the time:

crowdsec  | time="2025-03-08T00:05:57Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:57 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 1.796826ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""
crowdsec  | time="2025-03-08T00:05:57Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:57 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 2.029658ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""
crowdsec  | time="2025-03-08T00:05:58Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:58 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 2.578648ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""
crowdsec  | time="2025-03-08T00:05:58Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:58 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 2.18327ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""
crowdsec  | time="2025-03-08T00:05:58Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:58 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 3.558341ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""
crowdsec  | time="2025-03-08T00:05:58Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:58 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 1.545924ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""
crowdsec  | time="2025-03-08T00:05:58Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:58 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 4.6205ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""
crowdsec  | time="2025-03-08T00:05:59Z" level=info msg="172.18.0.6 - [Sat, 08 Mar 2025 00:05:59 UTC] \"GET /v1/decisions?ip=<my-ip>&banned=true HTTP/1.1 403 1.282028ms \"Crowdsec-Bouncer-Traefik-Plugin/1.X.X\" \""

I’ve made sure that I’m whitelisted:

[+] crowdsec container is running
[+] IP <my-ip> is NOT currently blocked by CrowdSec.
[+] IP <my-ip> is whitelisted in Traefik configuration (likely part of a subnet).

Thoughts?

I recently followed your iptables advice (in another thread)… that shouldn’t have any impact here since I’m seeing it logged in Crowdsec, but figured I’d point it out anyhow…

Your bounce is not registered or keys don’t match.

docker exec crowdsec cscli bouncers list
docker exec crowdsec cscli bouncers delete traefik-bouncer
docker exec crowdsec cscli bouncers add traefik-bouncer

It does say “valid” and I had already tried deleting and re-adding the bouncer. Tried again now and no dice.

Could it be an iptables issue with fail2ban or geoblocking?

I mostly followed your guide here: Discord < could this be an issue at all?

INPUT DROP [4571:309566]
-A INPUT -i lo -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 42522 -j ACCEPT
-A INPUT -p udp -m udp --dport 51820 -j ACCEPT # Wireguard
-A INPUT -p udp -m udp --dport 17072 -j ACCEPT # Wireguard
-A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT # Crowdsec
-A INPUT -p tcp -m tcp --dport 6060 -j ACCEPT # Crowdsec metrics
-A INPUT -p tcp -m tcp --dport 7422 -j ACCEPT # Crowdsec AppSec endpoint
-A INPUT -i docker0 -j ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -m conntrack --ctstate INVALID -j DROP
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o eth0 -j ACCEPT
-A FORWARD -i eth0 -o docker0 -j ACCEPT
-A OUTPUT -m conntrack --ctstate INVALID -j DROP

Fail2ban for sure. Please remove it.

"If you encounter issues adding IP addresses to your CrowdSec whitelist through the usual methods (e.g., using a management interface), you can manually edit the whitelist configuration file and restart CrowdSec. Here’s an example configuration:

name: crowdsecurity/my-whitelists
description: "Whitelist events from my IPv4 addresses"
whitelist:
  reason: "My IPv4 ranges"
  ip:
    - "51.20.140.83"  # Added by IP Shield
    - "10.0.0.88"  # Added by IP Shield
    - "127.0.0.1"
  cidr:
    - "65.36.22.25/32"  # Added by IP Shield
    - "192.168.0.0/16"
    - "10.0.0.0/8"
    - "172.16.0.0/12"
  expression:
    - evt.Parsed.source_ip == '127.0.0.1'
    - evt.Parsed.source_ip contains '172.17.'

Remember to restart the CrowdSec service after making changes to the configuration file for them to take effect."

docker compose restart crowdsec

Crowdsec is running, yet the script does not detect it. What could I do to have the container recognized?

[opc@debian ~]$ docker compose up -d --remove-orphans
[+] Running 5/5
 ✔ Container backrest  Running                                                                                     0.0s
 ✔ Container crowdsec  Running                                                                                     0.0s
 ✔ Container pangolin  Healthy                                                                                     1.0s
 ✔ Container gerbil    Running                                                                                     0.0s
 ✔ Container traefik   Running                                                                                     0.0s
======================================================================
   CHECKING PREREQUISITES
======================================================================

[+] Docker is running
[-] CrowdSec container does not exist. Please ensure it's created.
[!] You may need to run your docker-compose first.
[*] Temporary files cleaned up.
1 Like

sudo apt install grep gawk sed tar
Run the above command and try.
Which is your OS Version?

Thanks for the quick answer - it’s still the same. But this is my mistake, I’m on Oracle Linux and I read too quickly and missed the Debian & derivatives prerequisite… :slight_smile:

1 Like

I don’t have oracle linux. so didn’t test on it. No issues.

Use the bot instead
Pangolin Discord bot to Manage the stack

If you just need crowdsec management then use hhftechnology/pangolin-discord-bot:crowdsec

1 Like

I have the same issue running on Debian 12.
Not detecting crowdsec. apt install grep gawk sed tar has been run

1 Like

share your docker ps and did you run sudo apt update && sudo apt upgrade -y ?

sudo apt update && sudo apt upgrade -y has been performed

6f2c4b4d3f47   traefik:latest                    "/entrypoint.sh --co…"   About a minute ago   Up About a minute                                                                                                                                             traefik
8f49d4f5074c   fosrl/gerbil:latest               "/entrypoint.sh --re…"   About a minute ago   Up About a minute             0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp, 0.0.0.0:51820->51820/udp, [::]:51820->51820/udp   gerbil
bef78c75814b   fosrl/pangolin:latest             "docker-entrypoint.s…"   About a minute ago   Up About a minute (healthy)                                                                                                                                   pangolin
5a06ba1d80c2   crowdsecurity/crowdsec:latest     "/bin/bash /docker_s…"   About a minute ago   Up About a minute (healthy)   0.0.0.0:6060->6060/tcp, [::]:6060->6060/tcp                                                                                     crowdsec
b5ebb601e9ec   portainer/portainer-ce:latest     "/portainer"             33 hours ago         Up 2 hours                    8000/tcp, 9000/tcp, 0.0.0.0:9443->9443/tcp, [::]:9443->9443/tcp                                                                 portainer
b79bdc4f4d1c   portainer/agent:latest            "./agent"                33 hours ago         Up 2 hours                    0.0.0.0:9001->9001/tcp, [::]:9001->9001/tcp                                                                                     portainer_agent
1c75e4361c66   henrygd/beszel-agent              "/agent"                 4 days ago           Up 2 hours                                                                                                                                                    beszel-agent
72e965180b63   louislam/dockge:1                 "/usr/bin/dumb-init …"   2 weeks ago          Up 2 hours (healthy)          0.0.0.0:5001->5001/tcp, [::]:5001->5001/tcp                                                                                     dockge
234265b03979   rustdesk/rustdesk-server:latest   "hbbs -r rust.manilx…"   2 weeks ago          Up 2 hours                    0.0.0.0:21115-21116->21115-21116/tcp, [::]:21115-21116->21115-21116/tcp, 0.0.0.0:21116->21116/udp, [::]:21116->21116/udp        hbbs
dc7b605b61d8   rustdesk/rustdesk-server:latest   "hbbr -k a37AQsWJwuL…"   2 weeks ago          Up 2 hours                    0.0.0.0:21117->21117/tcp, [::]:21117->21117/tcp                                                                                 hbbr
0bf348f470e6   containrrr/watchtower             "/watchtower"            4 weeks ago          Up 2 hours (healthy)          8080/tcp                                                                                                                        watchtower
1 Like
└─# ./setup_crowdsec_manager.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  128k  100  128k    0     0   377k      0 --:--:-- --:--:-- --:--:--  376k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4066  100  4066    0     0  12722      0 --:--:-- --:--:-- --:--:-- 12746
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1860  100  1860    0     0   6627      0 --:--:-- --:--:-- --:--:--  6642
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2842  100  2842    0     0   9956      0 --:--:-- --:--:-- --:--:--  9937
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11989  100 11989    0     0  45960      0 --:--:-- --:--:-- --:--:-- 46111
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3526  100  3526    0     0  14055      0 --:--:-- --:--:-- --:--:-- 14104
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15554  100 15554    0     0  59171      0 --:--:-- --:--:-- --:--:-- 59140
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  8864  100  8864    0     0   9170      0 --:--:-- --:--:-- --:--:--  9175
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2315  100  2315    0     0   1930      0  0:00:01  0:00:01 --:--:--  1932
======================================================================
   CHECKING PREREQUISITES
======================================================================

[+] Docker is running
[-] CrowdSec container does not exist. Please ensure it's created.
[!] You may need to run your docker-compose first.
[*] Temporary files cleaned up.
1 Like