How AWS Security Groups and NACLs block TCP ports differently
Several years ago at an AWS training course, the trainer said that EC2 security groups are "stateful," whereas EC2 NACLs (Network Access Control Lists) are not. The consequence was that you could easily set up "outgoing only" connections on a security group by adding an outbound rule, but on a NACL, you'd have to configure both an outbound and an inbound rule to allow even outbound connections. The NACL had to be configured this way or it wouldn't let the responses come back from whatever network location your system was trying to connect to. But I've never really known what he meant.
In reading the nmap Book section on Avoiding Intrusion Detection and Prevention Systems when port scanning, I found an explanation of nmap's ACK scanning feature that helped me understand what "stateless" really means in the above context.
Here's the excerpt from the nmap book:
Many networks allow nearly unrestricted outbound connections, but wish to block Internet hosts from initiating connections back to them. Blocking incoming
SYNpackets (without theACKbit set) is an easy way to do this, but it still allows anyACKpackets through. Blocking thoseACKpackets is more difficult, because they do not tell which side started the connection. To block unsolicitedACKpackets (as sent by the NmapACKscan), while allowingACKpackets belonging to legitimate connections, firewalls must statefully watch every established connection to determine whether a givenACKis appropriate.
Let's do two sets of tests: one focused on NACLs and one focused on Security Groups.
In these first tests (focused on the NACL), the EC2 instance has a Security Group, but all traffic is allowed (both inbound and outbound) on that Security Group. I did this because every EC2 instance must have a security group in AWS, and I wanted to focus on only the NACL for now, so the Security Group is just passing traffic through.
NACL (Stateless)
First we'll put the EC2 instance behind a NACL that has all outbound ports allowed but all inbound ports disallowed. We should expect all ports to show as filtered in an nmap ACK scan. More important to understand right now is that any outgoing TCP connections, which require packets to be sent back to establish the connection and transfer data, will not work.
(Diagrams generated with ChatGPT)
+-----------------------+
| Network ACL |
| |
| |
| Outbound ports: |
| 0-65535 |
+-----------------------+
^
|
v
+-----------------------+
| Subnet |
| |
| +-------------------+ |
| | AWS EC2 Instance | |
| | | |
| | netcat listening| |
| | on port 80 | |
| +-------------------+ |
| |
+-----------------------+
As promised, no TCP connections to network locations outside the instance's local subnet are possible in this configuration:
[ec2-user@ip-10-0-12-197 ~]$ nc -w 1 -v google.com 80
Ncat: Version 7.93 ( https://nmap.org/ncat )
Ncat: Connection to 142.250.31.102 failed: TIMEOUT.
Ncat: Trying next address...
Ncat: Connection to 142.250.31.113 failed: TIMEOUT.
Ncat: Trying next address...
Ncat: Connection to 142.250.31.138 failed: TIMEOUT.
Ncat: Trying next address...
Ncat: Connection to 142.250.31.139 failed: TIMEOUT.
Ncat: Trying next address...
Ncat: Connection to 142.250.31.100 failed: TIMEOUT.
Ncat: Trying next address...
Ncat: Connection to 142.250.31.101 failed: TIMEOUT.
Ncat: Trying next address...
Ncat: Connection to 2607:f8b0:4004:c08::66 failed: Network is unreachable.
Ncat: Trying next address...
Ncat: Connection to 2607:f8b0:4004:c08::71 failed: Network is unreachable.
Ncat: Trying next address...
Ncat: Connection to 2607:f8b0:4004:c08::64 failed: Network is unreachable.
Ncat: Trying next address...
Ncat: Network is unreachable.
In this configuration, nmap finds that all ports (port 53 notwithstanding) are filtered:
eddie@pop-os:~$ sudo nmap -T4 -Pn -sA 54.221.77.100
Starting Nmap 7.80 ( https://nmap.org ) at 2024-01-26 14:59 EST
Nmap scan report for ec2-54-221-77-100.compute-1.amazonaws.com (54.221.77.100)
Host is up (0.0020s latency).
Not shown: 999 filtered ports
PORT STATE SERVICE
53/tcp unfiltered domain
Nmap done: 1 IP address (1 host up) scanned in 4.31 seconds
Side note: I'm actually not sure why port 53 is unfiltered here, so I'll try to figure that out separately. Regardless, all other scanned ports are filtered. We could try all 65536 ports, but there's no need at least for our demonstration.
Next, we'll add an inbound port range rule to the NACL. This will allow inbound ephemeral ports 1024-65535, which are also called "local" ports in Linux. These are the ports that could be used for connection back to the EC2 instance from whatever outgoing connections it originates.
+-----------------------+
| Network ACL |
| |
| Inbound ports: |
| 1024-65535 |
| |
| Outbound ports: |
| 0-65535 |
+-----------------------+
^
|
v
+-----------------------+
| Subnet |
| |
| +-------------------+ |
| | AWS EC2 Instance | |
| +-------------------+ |
| |
+-----------------------+
Outbound connections are now possible:
[ec2-user@ip-10-0-12-197 ~]$ nc -w 1 -v google.com 80
Ncat: Version 7.93 ( https://nmap.org/ncat )
Ncat: Connected to 142.250.31.101:80.
When we perform an nmap ACK scan, the scan should show that a RST response was returned from the EC2 instance.
eddie@pop-os:~$ sudo nmap -ddd -T4 -Pn -sA 54.221.77.100
...OUTPUT TRUNCATED BY ME...
1001/tcp filtered webpush no-response
1002/tcp filtered windows-icfw no-response
1007/tcp filtered unknown no-response
1009/tcp filtered unknown no-response
1010/tcp filtered surf no-response
1011/tcp filtered unknown no-response
1021/tcp filtered exp1 no-response
1022/tcp filtered exp2 no-response
1023/tcp filtered netvenuechat no-response
1024/tcp unfiltered kdm reset ttl 115
1025/tcp unfiltered NFS-or-IIS reset ttl 115
1026/tcp unfiltered LSA-or-nterm reset ttl 113
1027/tcp unfiltered IIS reset ttl 114
1028/tcp unfiltered unknown reset ttl 113
1029/tcp unfiltered ms-lsa reset ttl 115
1030/tcp unfiltered iad1 reset ttl 114
1031/tcp unfiltered iad2 reset ttl 113
1032/tcp unfiltered iad3 reset ttl 115
...OUTPUT TRUNCATED BY ME...
What happened here?
The nmap scan I performed sent ACK packets to the top 1000 most interesting ports (as defined by nmap), just like in the previous scan, but during this scan, some ports showed as "unfiltered" rather than "filtered."
Notice that the return status switched from filtered to unfiltered immediately when we started scanning port 1024. This is not an accident - our firewall (NACL) has ports 1024-65535 allowed for inbound connections. This doesn't tell us that those ports are open in nmap terms - there may be (in this case there is not) something listening on those ports on the target host, but all we can tell from this that the Firewall allowed the ACK packet to reach the EC2 instance, and the instance returned a RST packet.
For more information about nmap's definitions of open, closed, filtered, unfiltered, etc., see the nmap docs.
Security Group (Stateful)
Now, if we allow all inbound and outbound ports on the NACL so that all our traffic is handled by the Security Group attached to the EC2 instance, we can shift our test to the Security Group. We'll allow all outbound ports on the Security Group, but we won't allow any inbound ports.
+-----------------------+
| Network ACL |
| |
| |
| All inbound and |
| outbound traffic |
| allowed |
+-----------------------+
^
|
v
+-----------------------+
| Subnet |
| |
| +-------------------+ |
| | Security Group | |
| | | |
| | Inbound ports: | |
| | None allowed | |
| | Outbound ports: | |
| | 0-65535 | |
| +-------------------+ |
| | AWS EC2 Instance | |
| | | |
| +-------------------+ |
| |
+-----------------------+
When we ACK scan the host this time, we see that all the ports are filtered:
eddie@pop-os:~$ sudo nmap -T4 -Pn -sA 54.221.77.100
[sudo] password for eddie:
Starting Nmap 7.80 ( https://nmap.org ) at 2024-01-26 15:32 EST
Nmap scan report for ec2-54-221-77-100.compute-1.amazonaws.com (54.221.77.100)
Host is up (0.00064s latency).
Not shown: 999 filtered ports
PORT STATE SERVICE
53/tcp unfiltered domain
Nmap done: 1 IP address (1 host up) scanned in 6.30 seconds
And yet, outbound connections from the host still work:
[ec2-user@ip-10-0-12-197 ~]$ nc -w 1 -v google.com 80
Ncat: Version 7.93 ( https://nmap.org/ncat )
Ncat: Connected to 142.250.31.101:80.
Curling google.com:
[ec2-user@ip-10-0-12-197 ~]$ curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>