Problem with 403 denied access when uploading multiple or big files

Help
,
1

Hi! So far im in love with Pangolin, i used a few of the guides of this forums for personalize it (like adding geoblock,etc) but seems Crowdsec is blocking me or my family members when we are uploading any type of big files or multiple files(Acess Denied Error 403), lets say on Nextcloud or our Gitea instance…

I read somewhere that i should run the command " 1. docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' pangolin crowdsec" but not sure what i should do with the output? is this:
putty_QOcvSXYJTY

So if someone’s could help me it will be amazing since my family been complaining about getting access denied because of this.

2

Hi there, sorry for the delayed response.

Why CrowdSec Might Block Uploads:

  1. Rate Limiting: Uploading many files quickly can trigger scenarios designed to detect brute-force attacks or content scanning, exceeding requests-per-second thresholds.

  2. *Large Request Size: CrowdSec block excessively large HTTP request bodies, potentially flagging large file uploads.
    try this to set in the middleware.

  middlewares:
    limit:
      buffering:
        maxRequestBodyBytes: 4000000000
        maxResponseBodyBytes: 4000000000
        memResponseBodyBytes: 2000000
        memRequestBodyBytes: 2000000

If the request exceeds the allowed size, it is not forwarded to the service, and the client gets a 413 and thats the reason crowdsec flags you and a block is issued.

  1. Long Request Duration: Large uploads take time. Scenarios looking for slow attacks (like Slowloris) might mistakenly trigger if the connection stays open for a long time processing the upload.

  2. Specific Scenarios: There might be a specific scenario (either default or one you added) that is overly sensitive to the patterns generated by your upload activity.

    • Check CrowdSec Decisions: Use the CrowdSec command-line tool (cscli) on your server to see why the IP address was blocked.
      • Find active decisions: docker exec crowdsec cscli decisions list (Look for your IP or your family members’ IPs). Note the Reason (scenario name) and ID.
      • Inspect the alert that led to the decision: sudo cscli alerts list (Find the alert corresponding to the decision/IP) then docker exec crowdsec cscli alerts inspect <alert_ID> This gives detailed information about the trigger.
      • Traefik Logs: (e.g., /root/config/traefik/logs/access.log). Will show the 403 error and potentially which service generated it.

  3. Whitelist Trusted IPs (Simple Fix, Use Cautiously):

    • If the IPs being blocked are your static home IP or specific IPs your family uses, you can add them to a CrowdSec whitelist. This tells CrowdSec to ignore all events from these IPs.
    • Command: docker exec crowdsec cscli allowlists create my_allowlist --description "family_allowlist"
    • docker exec crowdsec cscli allowlists add my_allowlist 1.2.3.4 -e 7d
    • Allowlists | CrowdSec
    • Pros: Quick fix for trusted sources.
    • Cons: Doesn’t fix the underlying false positive. If the IPs change (dynamic IPs), this won’t work long-term. Whitelisting bypasses all checks, slightly reducing security for that IP.

  4. Create a Specific Whitelist for the Scenario/Event:

    • This is more precise than whitelisting the entire IP. You can tell CrowdSec to ignore events matching certain criteria before they contribute to triggering a scenario.
    • Create or edit a whitelist configuration file (e.g., /etc/crowdsec/parsers/s02-parse/my_upload_whitelist.yaml).
    • Inside, you can specify conditions, for example:
      • Ignore events from specific IPs for a particular scenario.
      • Ignore events targeting a specific URL path (e.g., your upload endpoint).
      • Ignore events based on log content patterns.
    • Example structure (conceptual):
      name: mycompany/my-upload-whitelist
description: "Whitelist events related to file uploads for specific IPs"
whitelist:
  reason: "Whitelist family uploads for scenario X"
  ip:
    - "YOUR_IP_ADDRESS"
    - "FAMILY_MEMBER_IP_ADDRESS"
  expression: # Optional: Add more conditions like target URL
    - "evt.Meta.target_uri contains '/upload-endpoint'"
  # Apply only to a specific scenario:
  # scenario:
  #   - "crowdsecurity/http-bf" # Replace with the actual scenario name
    • Refer to the CrowdSec Whitelisting Documentation for the correct syntax. Reload CrowdSec after creating/editing or just search https://app.crowdsec.net/hub/author/crowdsecurity/log-parsers/nextcloud-whitelist and add

  5. Adjust traefik middleware buffers:

  middlewares:
    limit:
      buffering:
        maxRequestBodyBytes: 4000000000
        maxResponseBodyBytes: 4000000000
        memResponseBodyBytes: 2000000
        memRequestBodyBytes: 2000000

If the request exceeds the allowed size, it is not forwarded to the service, and the client gets a 413 and thats the reason crowdsec flags you and a block is issued.

there was an issue if your file is large than 2gb and it was recently closed on april 11
Can’t upload docker images larger than 2GB via traefik 3.0 proxy · Issue #10741 · traefik/traefik
Set default ReadTimeout value to 60s by rtribotte · Pull Request #10602 · traefik/traefik

Last and final, use middleware manager to add crowdsec on individual resources so that you get more control and keep your logs separated with log processor, you can feed crowdsec only those logs necessary.

1 Like
3

This fix does WORK!

By adding as you said in the dynamic config at the middlewares:

        buffering:
            buffering:
                maxRequestBodyBytes: 4000000000
                maxResponseBodyBytes: 4000000000
                memResponseBodyBytes: 2000000
                memRequestBodyBytes: 2000000

And then in the traefik config under middlewares before crowdsec buffering@file

It does the job, now me and my family members are not getting 403 errors anymore!

1 Like
4

@hhf.technoloy Seems is creating some troubles with Jellyfin when used on SmartTVs, i will need to incresease the bodybytes for fix it?

1 Like
5

You have to balance it. What is the issue. Any logs ?

6

So thats the problem, i couldnt find any logs, but when they are trying to load a movie the screen just remains black, after disabling a bit crowdsec and the buffering middleware the movie starts without any problems, i will give it a bit more testing

1 Like
7

What is the media streaming at? 1080p?

8

Hi! sorry for the late response, 1080p bluray yes

1 Like
9

i think the issue is at the source. please check jellyfin logs

1 Like
10

Crowdsec Manager