Emergency response under Linux (Basic knowledge record)

Check CPU Usage#

The purpose of checking CPU usage is to determine which process is using a high amount of CPU (specifically for mining).

top -c -o %CPU
htop -t

# View the top five processes by CPU usage
ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%cpu | head -n 5

Memory Usage#

top -c -o %MEM
htop -t

# View the top five processes by memory usage
ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%mem | head -n 5

Network Usage#

iftop # Requires installation and root privileges, does not display process ID
nethogs # Requires installation and root privileges, displays process ID
ss -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr # View the number of local network card connections by source IP
ss -ntu | awk '{print $6}' | cut -d ":" -f1 | sort | uniq -c | sort -nr # View the number of destination IP connections

Outbound IP#

# Search by destination IP
netstat -pantu | grep 1.1.1.1
netstat -pantu | grep 3389
lsof -i:3389 # Requires root privileges

# Search by local IP
netstat -pantu | grep 3389
lsof -i:3389

Finding Malicious Samples#

Get PID - Find the path of the malicious file
Found malicious file - PID

Get PID based on process name or partial string

pidof "name"
ps -aux | grep "name"
ps -ef | grep "name" | grep -v grep | awk '{print $2}'
pgrep -f "name"

Get detailed information of a process based on PID

lsof -p PID # Requires root privileges
pwdx PID # Requires root privileges, get the directory where the PID process was started, which is the path of the malicious file
systemctl status PID # Get the status information of the process
cat /proc/PID/maps # Output the memory mapping information of the specified process
ls -al /proc/1505945/exe # Output the absolute path of the program executed by the specified process
cat /proc/$$/mountinfo # View the file system information mounted by the current process, where $$ represents the process ID (PID) of the current process.

View threads

ps H -T -p PID # PID is the process ID, spid is the thread ID, CMD represents the command line of the process/thread
ps -Lf PID # Display information of each thread in the specified process
pstree -agplU # Display the relationship between all processes in the system ✅

Get PID based on file

lsof | grep FILENAME
lsof FILENAME
fuser FILENAME # Command to find the process that uses a specific file or socket

Determine Program Running Time#

ps -eo pid,lstart,etime,cmd | grep PID

# Compare the creation time of the malicious file
stat FILENAME
ls -al FILENAME

Handling Abnormal Processes#

Download samples from the server
Online virus analysis
Process termination

1. Check if there are any child processes
ps ajfx ✅
systemctl status

2. If there are no child processes
kill -9 PID

3. If there are child processes
kill -9 -PID

Delete Malicious Files#

Check if the process is occupying the file

lsof FILENAME

If the file cannot be deleted due to the "a" and "i" attributes

chattr -a
chattr -i

If the file cannot be deleted due to strange file names

Use inode to delete

ls -li FILENAME # View inode

Delete the file

find ./* -inum INODE -delete
find ./ -inum INODE -exec rm {} \;
find ./* -inum INODE -exec rm -i {} \; # Prompt for confirmation before deleting
find ./* -inum INODE -exec rm -f {} \; # Force delete
find ./* -inum INODE | xargs rm -rf
rm `find ./* -inum INODE`

These are the commands needed for most emergency incidents. There may be some differences for different events:

Extension 1: TCP Connection States in netstat#

State	Type	Description
LISTEN	TCP Listening Port	Listening state. Indicates that the port is waiting for a connection from the other end for communication.
SYN_SENT	TCP Transmission Control Protocol State	Connection request has been sent. Indicates that the TCP connection has been initiated but not yet acknowledged.
SYN_RECV	TCP Transmission Control Protocol State	Connection request is being received. Indicates that the TCP connection has been received and is waiting for acknowledgment. Usually only appears on servers, indicating that a request has been received from a client.
ESTABLISHED	TCP Transmission Control Protocol State	Indicates that the TCP connection has been established and is in communication.
FIN_WAIT1	TCP Transmission Control Protocol State	Indicates that the TCP connection has been closed and is waiting for a close request from the other end.
FIN_WAIT2	TCP Transmission Control Protocol State	Indicates that the TCP connection has been closed and is waiting for a close request from the other end, or is receiving final confirmation from the other end.
TIME_WAIT	TCP Transmission Control Protocol State	Indicates that the TCP connection has been closed and all data has been transmitted, waiting for a period of time to ensure that all packets have been processed.
CLOSE_WAIT	TCP Transmission Control Protocol State	Indicates that the TCP connection has been closed, but the local application has not closed the connection.
LAST_ACK	TCP Transmission Control Protocol State	Close request has been sent and waiting for the other end's close request.
CLOSING	TCP Transmission Control Protocol State	Indicates that the TCP connection is in the process of closing.