Checking a Server's Outgoing Connections

In short: You check a server's outgoing connections with ss -tunap or lsof -i, which assign every connection to a process. Connections to the database, mail server, update servers and known interfaces are normal. Connections from PHP or web server processes to unknown destinations, unusual ports and persistent connections for no reason are warning signs of remote control or data leakage. In the long term, egress filtering that allows only the destinations needed helps.

Why looking outward reveals so much

When searching for a break-in, most people look at files. But an attacker wants to do something with his access: download tools, receive commands, get data out or use the server for spam or attacks on others. All of that needs connections to the outside.

A web server opens only a few connections on its own, and most of them are predictable: it queries the database, hands over mail, calls payment or shipping interfaces and fetches updates. That is why foreign connections stand out here in particular. According to the reports available to us, outgoing connections are among the general warning signs after a break-in; there are no confirmed indicators of compromise specifically for PHP2Shell. Further signs are summarized in Am I already compromised?.

Listing outgoing connections and assigning them to processes

The following commands only read. Run them as root, otherwise the process details are missing.

# All established TCP connections with process
ss -tnp state established

# TCP and UDP including listening services
ss -tunap

# Connections per program and user, without name resolution
lsof -i -n -P | grep -v LISTEN

# Only connections of processes of the web server users
lsof -i -n -P -u www-data 2>/dev/null

In the output, watch three columns: the process, the destination address with port and the user. A connection from php-fpm to the database on port 3306 is normal. A connection from a program with an unknown name out of /tmp to a foreign address on an unusual port is not. Which user PHP runs under depends on the distribution and panel.

Normal or suspicious? A classification

Many connections on port 25 point to spam sending; there is a separate article on this, The server is sending spam.

Assessing an unknown destination

A foreign address in the list is not yet proof. Before you decide, clarify where the connection leads and who opened it. These steps, too, change nothing:

1. Determine the process: Take the PID from ss and check with ls -l /proc/PID/exe and the command line which program is behind it.
2. Look up the destination: Use dig -x <IP> to find the name and whois <IP> to find the network operator.
3. Compare with the application: Ask the developer or service provider whether your application uses this service.
4. Check the history: Firewall and network logs show since when the connection has existed and how much data has flowed.
5. Secure the results: Save outputs with the time before you terminate or block anything.

Expect ambiguous results. Many legitimate services run at large cloud and CDN providers whose addresses allow no inference about the customer. Conversely, attackers also use rented cloud servers. A well-known network operator is therefore neither all-clear nor proof; what matters is whether the process and the purpose fit your application.

A snapshot with ss shows only what is open right now. Short connections that are opened every minute you see only in logs. That is why firewall or flow logs are often more important than a look at the server itself for the question of data leakage.

Do not forget DNS and containers

Not every data leak needs a visible connection to a foreign address. Name resolutions via DNS go to your data center's resolver and hardly stand out in the ss list. Attackers can pack data into long, random-looking subdomains and thus get it out via DNS queries. Hints of this are provided by the resolver's logs: unusually many requests to the same foreign domain, very long names or requests at times when nobody is using the website.

If your applications run in containers, the host often sees the containers' connections only as traffic of the container network. In that case, check each container individually, as far as the tools are available there, or evaluate the connections at the level of the host and the firewall.

# Connections inside a container (if ss is included in the image)
docker exec <container> ss -tnp 2>/dev/null

# On the host: connections tracked by the kernel, if conntrack is installed
sudo conntrack -L 2>/dev/null | head -50

Limiting outgoing traffic permanently

The most effective prevention is egress filtering: the server may reach only the destinations it really needs, and everything else is blocked and logged. This makes it considerably harder for attackers to download tools or pull out data, even if they can execute code.

The path there begins with a list of the destinations needed, which you derive from the connections collected above. A phase follows in which blocked connections are only logged before the block is activated. Payment services, update servers and interfaces with changing addresses need special care here. How a firewall for web servers is built overall is described in Firewall for web servers: open only necessary ports.

How does PHP SOS help?

• Check connections: We assign all active connections to processes and purposes and assess unknown destinations.
• Assess data leakage: We evaluate firewall and network logs and clarify whether and when data was transferred.
• Limit egress: Together with you, we draw up a list of permitted destinations and set up egress filtering step by step.
• Monitor permanently: With security monitoring, we report new or unusual connections.

Frequently asked questions

Sources

• BSI: German Federal Office for Information Security
• GDPR Art. 33: Notification to the supervisory authority
• OWASP Cheat Sheet Series