My first YouTube Scary WordPress Story is now live on Youtube, I encourage you to watch it, but here is a slightly modified version for the blog!
Sometimes the most innocent actions have unintended consequences. This is a story, a story about Joe, a story about how something small can cascade.
Joe’s website is a magazine website he actually started in print publication many many years ago and he was one of the very first in his niche.
Joe has had throughout the time of the magazine many employees and at height the publication had 28 people.
Now it’s just Joe and some freelance writers.
Joe went fully digital a few years ago and uses WordPress and WooCommerce with subscriptions.
Joe’s site is well trafficked, and it’s his primary source of income, but joe doesn’t believe he has a budget for security or even development so he relies on stock plugins. At the heart of his site is a popular page builder, it gives him the flexibility to do almost everything he wants.
He’s a pretty normal guy one day he was on Twitter as it was called back then just browsing when he saw a funny GIF but there was a link to a website we could sign up to get more funny gifts I don’t know why he chose to sign up that day but he did people do funny things and sign up to any old rubbish.
Over the next few weeks, Joe got several emails from the site, one caught his eye, a build your own meme generator!
That sounds so great! It’s like this really cool website that allows you to remix memes. One of the features was a browser extension, to allow you to quickly insert memes on to your website.
Joe installed the extension, and logged into his site to try it, he was left disappointed the extension didn’t seem to work.
But something had happened.
As his browser made the post request on the wp-login.php page, the extension made its own request. The payload:
- URL,
- Username
- Password
The browser extension was malware, malware designed specifically aimed at site owners of WordPress, Joomla and Drupal.
All the extension would ever do is, harvest credentials and send them to a remote server.
Weeks passed, Joe had forgotten he had the extension installed, even though it was not working he like many others just left it and so it carried on harvesting across the couple of sites he logged into.
The original meme site was long gone, the extension no longer available to install.
One day, on a forum in the deepest darkest parts of the web, a new post appeared, offering a dump of 50k legitimate WordPress Admin credential logins.
$1 per account
One dollar, “exclusive” 24 hour access.
The seller would give the credentials to the buyer 24 hour ahead, before the whole lot on general sale as a bundle.
As part of the forum post, was a link to a public facing CSV dump.
The dump included the domain name, the number of users on the site and SEO metrics.
Hacking is a business and like any business has supply chains.
The group that created the extension, and harvested the credentials will have sold the list to a middle man seller. This seller is the person selling on the forum.
We can’t be sure if the credentials were specifically bought that day, or were in the non-exclusive cred list but it’s a good guess that they were.
Joe life was carrying on, he went to login into the site, and a strange thing happened, his password wouldn’t work. It just kept saying, incorrect password, frustrated he reset, the email arrived and he created a new password and carried on with his day.
It wasn’t for another 2 weeks, things started to go awry
Joe was updating WordPress core and it just wouldn’t update, some sort of permissions error,
Joe would tell you he was an untechnical person, an untechnical person with knowledge of FTP, file permissions and a deep knowledge of how he uses WordPress, he even occasionally downloads the backups from his backup plugin to his local “dev” environment.
When someone says they are “technical” or “nontechnical” it is often based on an assumed baseline that might in no way reflect your own.
For many Joe has a good technical knowledge of how to use WordPress.
Today though, he just couldn’t get it to work, so he contacted his web host. An email a little while later they had forced an update on the site and it was fine now.
Joe logged in and indeed the site was fine.
Let us leave Joe, and briefly visit the web host, the technician who had got Joe ticket was incredibly busy that day, things had been going wrong, the local coffee shop was closed, someone was out sick.
When the technician got in they found several of their colleagues were also out sick.
The technician logged on to the server, and to Joes files and used wp-cli to update, like Joe they got a file permission error.
So he tried `sudo wp core update –allow-root`
The site updated, brilliant, the technician went on with his day, to the next client and their email problems.
On Monday morning he goes to login. His emails have been quiet, indeed if Joe had been paying closer attention, his emails had been quiet since 2am on Saturday morning.
When he went to his website, he wasn’t presented with the login screen, instead he was presented with nothing. His site wouldn’t load, his emails wouldn’t load, his website appeared gone.
Panicking he went to log into his hosts control panel, it wasn’t working, it was gone.
He tried using his phone, different IPs, asked friends, nothing and no one could get to them.
He went through his call log and found the host’s phone number. It just rang and rang.
Nothing.
Let’s go back to the late hours of Friday night,
The late shift at the hosting company was having a quiet evening. Really the only ticket item for them was one of the shared hosts had triggered some alerts, about backups failing , which was odd they were not due to run for a couple of hours.
The admin logged on to the server, and found that the ssh connection between the host and the backup server wasn’t working for some reason, they tested with their own credentials everything appeared fine. Some fiddling and they realised a background service wasn’t enabled, so they restarted.
Around 1AM on Saturday morning, things started to go wrong, the backup server was running its daily backups.
Really the backup server is a cluster of servers and tasks run every night, it backs up everything, hosts, mail servers, even the office PCs.
Suddenly everything was going offline, it started with servers, connections being lost, then the office desktop machines, soon the entire host was offline.
They had been a victim of a ransomware attack.
Email gone, websites gone, their own control systems gone, their PBX telephone system gone, their DNS gone.
In their place, a ransom note for a very large amount of bitcoin
They worked throughout the weekend, to try and restore things, they had no way to reach out to customers. Even their Twitter account had been compromised.
Do they pay?
Would you?
A week passed, the host had made the technical press, web hosting victim of ransomware, all client data potentially lost. It was nearly 10 days later, when Joe was reached out too, on twitter by an account offering to restore his site.
For a fee
The ransom group, who had failed to reach an agreement with the host, went to the individual sites affected and was offering to give them their site data.
For Joe the offer was tempting, but with his host being the domain registrar, and where he had his DNS, the data was almost less important. It wouldn’t restore his site, it wouldn’t get his email back.
3 weeks later, the host had partially restored DNS facilities, Joe got his email back and DNS. He used an old backup from a few weeks before.
he finally tried to login in to his site, urgh his password wouldn’t work, oh that’s right he needed to use the old password as this was an old backup. But once in, everything looked fine.
The End
So lets with the power of hindsight go through and see what went wrong.
Joe installed a browser extension, perhaps without really understanding how much power such things have. An extension with the right privileges can steal ANY data that passes through your browser. Much like a WordPress plugin can do ANY thing on your site. Knowing and trusting the source is imperative.
Passwords rarely just stop working, in this case our bad actor had deliberately changed it to prevent other bad actors access to the site knowing the credentials would be known shortly to many others knowing that he had only 24 hours before general access. This should be a huge red flag. By this point the site had been compromised. Our bad actor had a backdoor and had placed malware, they had also changed the file permissions to lock the version of WordPress.
This could have all been prevented with Two-Factor Authentication
a 2FA plugin and I recommend two-factor as the plugin to do this, would have prevented this chain.
When the bad actor logged in, they would have needed the second factor, there are other solutions including SSO and magic links, but for most sites the simple solution is 2FA.
The malware, we will never know the payload, but it was designed to provide a backdoor, and a trigger if it was run in a privileged state i.e root.
Which leads to the poor technician, NEVER run WP-CLI as root, that giant warning it’s there for a reason. When you run it as root, then it can do anything a root user can.
From there our bad actor got root access to the web host, this is where it went from something automated to a manual attack. The actor investigated the network, found a vulnerability in the way they handled backups and exploited it to pivot once more to a full control and then infected every system with malware.
For Joe, he was screwed because all his eggs were in one basket, had he had his domain registered away from the host, a DNS managed by another service, and email through something like Gmail for business or whatever it’s called today, then he would have been significantly less screwed. By consciously spreading the risk to multiple service providers, you potentially increase costs and complication, but if one part is affected by ransomware or technical error, you can work around it.
Two simple mistakes, by two separate people resulted in a catastrophic chain.
We can always put technical solutions for each problem. Install two-factor plugin, prevent WP-CLI from ever being able to run as root.
But security isn’t always about technical solutions, it’s about mindsets and humans. Everyone is responsible for security, we all need to develop those tingly danger senses, we need to be given the space to develop and not to be put in positions to cut corners.
But please, seriously set two factor authentication for you and your clients, and yes that includes the MD who has an administrator account, Why?
This tale was fiction but grounded on real events, there was a website meme generator, and browser extension that grabbed WordPress credentials. More than once someone has accidentally run WP-CLI as root, warning signs, normally have a history after all, and there has been more than one host, who have become a victim of ransomware.
I think we should let Joe just get on with his day, it’s not been a great few weeks. I guess we can just hope that he changes his password, before one of the many buyers of that bulk buy list tries to login.