Advance Guide- Protecting Your Pangolin Stack from SYN Flood Attacks

Networking Firewalls & Security
1

USE THIS GUIDE WITH CATION! Performance impact if not deployed correctly.

Pangolin provides an excellent solution for secure tunneled reverse proxying, but like any internet-facing service, it can be vulnerable to various attacks. One of the most common and dangerous is the SYN flood attack, which can overwhelm your server by flooding it with connection requests.

This guide will walk you through implementing robust protections for your Pangolin infrastructure against SYN flood attacks and other security threats.

Understanding the Pangolin Stack

Before diving into security measures, let’s understand the key components of the Pangolin stack:

  1. Pangolin: The core management server that provides the web UI, API, and authentication
  2. Gerbil: WireGuard interface management server that creates and manages tunnels
  3. Traefik: The reverse proxy that handles incoming connections and routes traffic
  4. Badger: Traefik middleware that validates authentication
  5. CrowdSec: Security automation tool already included in the stack for threat detection

The most vulnerable parts of this stack are the publicly exposed services, particularly Traefik and Gerbil, which handle incoming connections on ports 80, 443, and 51820 (WireGuard).

What is a SYN Flood Attack?

A SYN flood attack exploits the TCP three-way handshake process:

  1. Client sends a SYN packet to the server
  2. Server responds with a SYN-ACK packet
  3. Client completes the handshake with an ACK packet

In a SYN flood attack, attackers send numerous SYN packets without completing the handshake. This leaves connections half-open, consuming server resources until they time out. When these half-open connections exhaust the server’s capacity, legitimate connections can’t be established.

Security Measures Already in Place

Your Pangolin stack already includes some security components:

  • CrowdSec integration: Detects and blocks various attack patterns
  • Traefik secure headers: Configured in the middlewares-secure-headers.yaml
  • TLS configuration: Secure SSL/TLS settings in tls-opts.yaml

Let’s enhance these with additional protections specifically targeting SYN flood attacks.

Step-by-Step Implementation Guide

Step 1: Optimize Kernel Parameters

SSH into your server and modify the kernel parameters to better handle SYN flood attempts:

sudo nano /etc/sysctl.conf

Add or uncomment these lines:

# Protection against IP spoofing
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.all.rp_filter=1

# SYN flood protection
net.ipv4.tcp_syncookies=1
net.ipv4.tcp_max_syn_backlog=4096
net.ipv4.tcp_synack_retries=3

# Additional network hardening
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_sack=0
net.ipv4.tcp_dsack=0

Apply the changes:

sudo sysctl -p

Step 2: Create SYN Flood Protection Firewall Rules

Create a dedicated file for SYN flood protection rules:

sudo mkdir -p ./config/traefik/rules/security
sudo nano ./config/traefik/rules/security/syn-flood-protection.yaml

Now, we need to modify the UFW firewall rules to implement SYN flood protection. First, check if UFW is active:

sudo ufw status

If not active, enable it:

sudo ufw allow 22/tcp
sudo ufw enable

Now, edit the UFW before rules:

sudo nano /etc/ufw/before.rules

Add these lines before the final COMMIT line:

# SYN flood protection chains
:SYN_FLOOD_PROTECTION - [0:0]
:fail2ban - [0:0]

# Check fail2ban chain first
-A ufw-before-input -j fail2ban
-A fail2ban -j RETURN

# Apply SYN flood protection to HTTP and HTTPS ports
-A ufw-before-input -p tcp --syn --dport 80 -j SYN_FLOOD_PROTECTION
-A ufw-before-input -p tcp --syn --dport 443 -j SYN_FLOOD_PROTECTION

# Drop excessive SYN packets and log them
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 80 -m conntrack --ctstate NEW -m hashlimit --hashlimit-name http_limit --hashlimit-above 10/second --hashlimit-burst 20 --hashlimit-mode srcip --hashlimit-srcmask 32 -j DROP
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 80 -m conntrack --ctstate NEW -m hashlimit --hashlimit-name http_limit --hashlimit-above 10/second --hashlimit-burst 20 --hashlimit-mode srcip --hashlimit-srcmask 32 -j LOG --log-prefix "[UFW SYN Flood Detected] "
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 443 -m conntrack --ctstate NEW -m hashlimit --hashlimit-name https_limit --hashlimit-above 10/second --hashlimit-burst 20 --hashlimit-mode srcip --hashlimit-srcmask 32 -j DROP
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 443 -m conntrack --ctstate NEW -m hashlimit --hashlimit-name https_limit --hashlimit-above 10/second --hashlimit-burst 20 --hashlimit-mode srcip --hashlimit-srcmask 32 -j LOG --log-prefix "[UFW SYN Flood Detected] "

# Allow normal SYN packets after checks
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 80 -m conntrack --ctstate NEW -j ACCEPT
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 443 -m conntrack --ctstate NEW -j ACCEPT

# Block invalid packets
*mangle
:PREROUTING ACCEPT [0:0]
-A PREROUTING -m conntrack --ctstate INVALID -j DROP
-A PREROUTING -p tcp -m tcp ! --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j DROP
COMMIT

Do the same for IPv6 by editing /etc/ufw/before6.rules with similar rules, adjusting the chain name to ufw6-before-input.

Reload UFW:

sudo ufw reload

Step 3: Configure Fail2ban for Persistent Protection

Install Fail2ban if not already installed:

sudo apt install fail2ban -y

Create a custom filter for SYN flood detection:

sudo nano /etc/fail2ban/filter.d/synflood.conf

Add this content:

[Definition]
failregex = .*UFW SYN Flood Detected.*SRC=<HOST>.*DPT=\d+.*
ignoreregex =

Create a local jail configuration:

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

Add this section under the # JAILS heading:

[synflood]
enabled  = true
filter   = synflood
action   = iptables[type=allports, name=synflood, chain=fail2ban, protocol=tcp]
logpath  = /var/log/syslog
maxretry = 5
findtime = 600
bantime  = 86400

Start Fail2ban:

sudo systemctl enable fail2ban
sudo systemctl restart fail2ban

Step 4: Enhance Traefik Security Configuration

Add rate limiting by creating a file:

sudo nano ./config/traefik/rules/security/rate-limit.yaml

With this content:

http:
  middlewares:
    global-rate-limit:
      rateLimit:
        average: 100
        burst: 50

Update your Traefik entry points configuration to apply these middlewares. Edit the traefik_config.yml file:

sudo nano ./config/traefik/traefik_config.yml

Find the entryPoints section and add the middleware to the websecure entry point:

entryPoints:
  websecure:
    address: ":443"
    transport:
      respondingTimeouts:
        readTimeout: "30m"
    http:
      middlewares:
        - crowdsec@file
        - middlewares-compress@file
        - middlewares-secure-headers@file
        - global-rate-limit@file  # Add this line

Step 5: Configure CrowdSec for Improved Attack Detection

CrowdSec is already included in your stack. Let’s make sure it’s properly configured to detect SYN flood attacks.

First, add a custom CrowdSec scenario:

sudo mkdir -p ./config/crowdsec/scenarios
sudo nano ./config/crowdsec/scenarios/syn-flood.yaml

Add this content:

type: leaky
name: pangolin-syn-flood
description: "Detect SYN flood attacks"
filter: "evt.Parsed.program == 'kernel' && evt.Parsed.message contains '[UFW SYN Flood Detected]'"
leakspeed: "10s"
capacity: 5
labels:
  service: pangolin
  type: syn-flood
  remediation: true
groupby: evt.Meta.source_ip
blackhole: 1m
reprocess: false

Next, create an appropriate CrowdSec profile:

sudo nano ./config/crowdsec/profiles/syn-flood.yaml

With this content:

name: syn-flood-ban
filters:
 - 'decision.labels.type == "syn-flood"'
decisions:
 - type: ban
   duration: 1h
on_success: break

Step 6: Restart the Stack

After making all these changes, restart the Pangolin stack:

sudo docker compose down
sudo docker compose up -d

Step 7: Monitoring and Testing

Monitor your logs for any signs of attacks:

# Check UFW logs
sudo grep "SYN Flood Detected" /var/log/syslog

# Check Fail2ban status
sudo fail2ban-client status synflood

# Check CrowdSec decisions
sudo docker exec -it crowdsec cscli decisions list

Additional Security Enhancements

1. Secure WireGuard (Gerbil)

WireGuard, managed by Gerbil, uses port 51820 by default. Protect it by adding:

sudo nano /etc/ufw/before.rules

Add before the final COMMIT:

# WireGuard rate limiting 
-A ufw-before-input -p udp --dport 51820 -m hashlimit --hashlimit-name wireguard_limit --hashlimit-above 5/second --hashlimit-burst 10 --hashlimit-mode srcip --hashlimit-srcmask 32 -j DROP
-A ufw-before-input -p udp --dport 51820 -m hashlimit --hashlimit-name wireguard_limit --hashlimit-above 5/second --hashlimit-burst 10 --hashlimit-mode srcip --hashlimit-srcmask 32 -j LOG --log-prefix "[UFW WireGuard Flood] "

2. Add Cloudflare as a Protective Layer

If possible, consider putting your Pangolin instance behind Cloudflare for additional DDoS protection:

  1. Set up Cloudflare for your domain
  2. Configure Traefik to trust Cloudflare IP addresses (already configured in your traefik_config.yml)
  3. Enable Cloudflare’s security features like “I’m Under Attack” mode if needed

3. Secure Database and Logs

Ensure your database and logs are properly secured:

sudo chmod 700 ./config/db
sudo chmod 600 ./config/db/db.sqlite
sudo chmod 700 ./config/logs

Monitoring for Attacks

Set up monitoring for potential attacks:

  1. Regular log checking: Create a cron job to check logs for patterns of attacks
sudo crontab -e

Add:

0 * * * * grep "SYN Flood Detected" /var/log/syslog | wc -l > /tmp/syn_flood_count.txt
  1. Email alerts: Configure Fail2ban to send email notifications when IPs are banned

Edit /etc/fail2ban/jail.local:

[DEFAULT]
destemail = your-email@example.com
sender = fail2ban@your-server.com
mta = sendmail
action = %(action_mwl)s

How to Unban IP Addresses

If a legitimate IP gets banned, you can unban it using:

# For Fail2ban
sudo fail2ban-client set synflood unbanip [IP_ADDRESS]

# For CrowdSec
sudo docker exec -it crowdsec cscli decisions delete --ip [IP_ADDRESS]

Conclusion

By implementing these measures, your Pangolin stack should now be well-protected against SYN flood attacks and other common security threats. The combination of kernel optimizations, firewall rules, Fail2ban, and CrowdSec provides multiple layers of defense, ensuring your reverse proxy remains available even during attack attempts.

Remember to regularly:

  • Monitor system logs for suspicious activity
  • Keep all components updated to the latest versions
  • Review and adjust rate limits based on your specific traffic patterns

This security setup balances protection against attacks with allowing legitimate traffic through, ensuring your Pangolin deployment remains both secure and functional.

Adding Additional Open Ports with Protection to Your Pangolin Stack

When expanding your Pangolin setup to accommodate additional services, you’ll need to open more ports while ensuring they remain protected from attacks. Here’s how to add new TCP and UDP ports with proper security measures in place:

Opening New TCP Ports with SYN Flood Protection

Step 1: Add the Port to UFW’s Allowed List

First, allow the new port through UFW. For example, to open TCP port 8080:

sudo ufw allow 8080/tcp

Step 2: Apply SYN Flood Protection to the New TCP Port

Edit your UFW before rules file:

sudo nano /etc/ufw/before.rules

You need to add two types of rules for each new TCP port:

  1. A rule to redirect traffic to your SYN_FLOOD_PROTECTION chain
  2. Rules within the SYN_FLOOD_PROTECTION chain to handle the new port

Find your existing SYN_FLOOD_PROTECTION section and add these lines:

# Redirect the new port to SYN_FLOOD_PROTECTION chain
-A ufw-before-input -p tcp --syn --dport 8080 -j SYN_FLOOD_PROTECTION

# Then inside the SYN_FLOOD_PROTECTION chain section, add:
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 8080 -m conntrack --ctstate NEW -m hashlimit --hashlimit-name tcp8080_limit --hashlimit-above 10/second --hashlimit-burst 20 --hashlimit-mode srcip --hashlimit-srcmask 32 -j DROP
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 8080 -m conntrack --ctstate NEW -m hashlimit --hashlimit-name tcp8080_limit --hashlimit-above 10/second --hashlimit-burst 20 --hashlimit-mode srcip --hashlimit-srcmask 32 -j LOG --log-prefix "[UFW SYN Flood Detected] "
-A SYN_FLOOD_PROTECTION -p tcp --syn --dport 8080 -m conntrack --ctstate NEW -j ACCEPT

Make sure to give each port a unique hashlimit name (like tcp8080_limit in this example).

Step 3: Apply the Same Rules to IPv6 (if used)

If you’re using IPv6, edit the IPv6 rules file:

sudo nano /etc/ufw/before6.rules

Add similar rules, changing the chain name to ufw6-before-input.

Opening New UDP Ports with Rate Limiting

UDP doesn’t use SYN packets like TCP, so SYN flood protection isn’t relevant. However, you should still apply rate limiting to prevent UDP flood attacks.

Step 1: Allow the UDP Port in UFW

For example, to open UDP port 5000:

sudo ufw allow 5000/udp

Step 2: Add Rate Limiting for the UDP Port

Edit the before.rules file again:

sudo nano /etc/ufw/before.rules

Add these lines before the final COMMIT:

# UDP rate limiting for port 5000
-A ufw-before-input -p udp --dport 5000 -m hashlimit --hashlimit-name udp5000_limit --hashlimit-above 15/second --hashlimit-burst 25 --hashlimit-mode srcip --hashlimit-srcmask 32 -j DROP
-A ufw-before-input -p udp --dport 5000 -m hashlimit --hashlimit-name udp5000_limit --hashlimit-above 15/second --hashlimit-burst 25 --hashlimit-mode srcip --hashlimit-srcmask 32 -j LOG --log-prefix "[UFW UDP Flood Detected] "

Note that UDP typically needs higher limits than TCP because it’s a connectionless protocol and legitimate applications often send more packets.

Step 3: Again, Apply to IPv6 if Needed

Apply the same rules to the before6.rules file if using IPv6.

Step 4: Update Fail2ban Configuration (Optional)

If you want Fail2ban to monitor and block IPs attempting to flood your new ports, edit the synflood filter:

sudo nano /etc/fail2ban/filter.d/synflood.conf

The existing filter should work as-is since it looks for the log prefix “[UFW SYN Flood Detected]” which we’re using for all ports.

Step 5: Configure Docker to Expose the Ports

If your services are running in Docker containers, update your docker-compose.yml file to expose the new ports. For the Gerbil service, which handles networking for Traefik:

gerbil:
  # Existing configuration...
  ports:
    # Existing ports...
    - 51820:51820/udp # WireGuard
    - 443:443 # HTTPS
    - 80:80 # HTTP
    
    # Add your new ports
    - 8080:8080 # New TCP port
    - 5000:5000/udp # New UDP port

Step 6: Apply the Changes

Reload UFW to apply the new firewall rules:

sudo ufw reload

Restart Fail2ban to ensure it picks up any changes:

sudo systemctl restart fail2ban

If you modified your docker-compose.yml file, restart the affected containers:

sudo docker compose down
sudo docker compose up -d

Step 7: Test the New Ports

Verify that your ports are open and working correctly:

# Check if the port is listening
sudo netstat -tuln | grep 8080

# Test TCP connection
nc -zv your-server-ip 8080

# For UDP ports
nc -zuv your-server-ip 5000

Special Considerations for Raw TCP/UDP Resources in Pangolin

If you’re using Pangolin’s raw TCP/UDP resource feature (as described in the documentation you provided), you need a few additional steps:

  1. Configure the EntryPoint in Traefik for each port:
sudo nano ./config/traefik/traefik_config.yml

Add your new EntryPoints in the format described in the Pangolin documentation:

entryPoints:
  # Existing entrypoints...
  
  # Add TCP entrypoint
  tcp-8080:
    address: ":8080/tcp"
    
  # Add UDP entrypoint
  udp-5000:
    address: ":5000/udp"
  1. Restart Traefik to apply the configuration changes:
sudo docker compose restart traefik

Monitoring Your New Ports

Keep an eye on traffic to your new ports:

# Monitor connections to a specific port
sudo netstat -anp | grep :8080

# Check logs for potential attacks on the new ports
sudo grep "UFW SYN Flood Detected" /var/log/syslog | grep "DPT=8080"
sudo grep "UFW UDP Flood Detected" /var/log/syslog | grep "DPT=5000"

By following these steps, you’ll have properly opened and protected new TCP and UDP ports in your Pangolin stack, maintaining the security measures we established earlier against potential flood attacks.

1 Like