Last Updated: 2024-11-23

We had a deploy script with something like this:

scp ./public/css/app.css $server:$dir/public/css/
scp ./public/css/app.css.gz $server:$dir/public/css/
scp ./public/js/app.js $server:$dir/public/js/
scp ./public/js/app.js.gz $server:$dir/public/js/
scp ./public/mix-manifest.json $server:$dir/public/mix-manifest.json

Sporadically, it would stop, giving an error about port 22 and the connection being broken.

Further investigation revealed that our deploy server was rate-limiting all these SSH requests of scp

This was done via the ufw firewall, which is "an interface to iptables that is geared towards simplifying the process of configuring a firewall."

To see what was configured:

$ ufw status verbose

Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), allow (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
# LIMIT means the port is firewalled
22/tcp                     LIMIT IN    Anywhere
2375/tcp                   ALLOW IN    Anywhere
2376/tcp                   ALLOW IN    Anywhere
22/tcp (v6)                LIMIT IN    Anywhere (v6)
2375/tcp (v6)              ALLOW IN    Anywhere (v6)
2376/tcp (v6)              ALLOW IN    Anywhere (v6)

For a more portable look at the exact rules, use iptables: (ufw is a wrapper about iptables)

$ iptables -S
-A ufw-user-input -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --set --name DEFAULT --mask 255.255.255.255 --rsource
-A ufw-user-input -p tcp -m tcp --dport 22 -m conntrack --ctstate NEW -m recent --update --seconds 30 --hitcount 6 --name DEFAULT --mask 255.255.255.255 --rsource -j ufw-user-limit
-A ufw-user-input -p tcp -m tcp --dport 22 -j ufw-user-limit-accept

ufw logs to the kernel and I can see the blocked entries:

$ cat /var/log/kern.log
Oct  8 17:07:08 project_s-production kernel: [15833226.781556] [UFW BLOCK]
IN=eth0 OUT= MAC=be:85:d9:e1:5d:26:40:a6:77:34:67:f0:08:00 SRC=85.99.91.123
DST=138.197.188.15 LEN=52 TOS=0x00 PREC=0x00 TTL=116 ID=58492 DF PROTO=TCP
SPT=58190 DPT=21 WINDOW=64240 RES=0x00 SYN URGP=0

In other code-bases, the rate-limiting might be applied with ssh

$ cat /etc/ssh/sshd_config

# Authentication:

LoginGraceTime 2m
PermitRootLogin yes
StrictModes yes
MaxAuthTries 6
MaxSessions 10

References

• https://www.digitalocean.com/community/tutorials/how-to-set-up-a-firewall-with-ufw-on-ubuntu-18-04