Got woken up by my monitoring server in the middle of the night to my lights flashing red and my phone telling me there was an issue with the server I use to host a few websites.
server load was really high and ram was maxed out triggering the alarm
sshed into the server and ran htop to verify the load and to see if there was anything standing out
seemed like it was coming from the webserver side so I checked the logs
noticed a shitton of traffic in the logs coming through trying to hit admin and database urls, so it seemed like my webserver was under attack
started trying to narrow it down why I was seeing a spike in traffic
was not sure what i was looking for in the logs so i changed directions and checked the frontend
i went through a couple of the sites i was hosting and noticed this come up instead of the site
ran this image past a friend who was a bit more versed on the security side of things and the let me know this was a reverse shell
FML i know what that means
i took a snapshot of the server at that moment beccause I knew i was gonna nuke the server
i took a clone of the VM and killed the network and started it back up to start digging
knowing i was reversed shelled, i looked for the typical stuff these fucks do, and BAM SPAM
noticed a lot of traffic on the vm knocking on the email ports 25 110
checked the processes and sure enough i saw postfix (an email server) installed
looked into postfix and found out it was trying to send out emails, those emails were hitting my firewall, the firewall would bounce them back causing the memory issue that triggered my alarm on the monitoring server
fuck email, that is the first thing i disabled and locked down to prevent this type of shit
here is one of the many emails it was trying to send out unsccessfully.
it was sending out spam emails for a Russian Ladies of the Night thingamajig
ended up tracking down the site to these fucks
site was: j 0 b g i r 1 2 4 DOT r u (spelled wrong but general idea)
fuck these people
lookin for the entry point
started with the reverse shell found how it was running / installed
reverse shell was being started by a cron job using an older version that somehow managed to get installed on the machine
tracked the orgin of the job to a wordpress install i was hosting for my cousin
the sites custom theme that the dev had stopped maintain that had some code in it that allowed them access to install that old cron version with a job that would install the reverse shell
postmortem
WordPress sucks when you do not maintain it. I should know better
custom coded themes suck, stick to the ones that have a long support cycle
keep shit up to date, especially running WordPress
monitoring and alerts saved my ass. this could have gone unnoticed for a while before i had seen this or thought to check
current hosting method is not the best idea. 1 VM with multiple WordPress installs behind Apache and Nginx
1 website took down all my other websites
gonna look into containerizing my stack with docker or some other container manager
should be more secure each website is contained to its own instance separate from the host machine usually
good thing i had backups too, made it easy to restore the sites to the new temp server
made sure to go through each site and sanitize it as best as i could. Nuked everything but the DB wp-config.php and the wp-content folders
