Question: What's the worst attack you've defended against?

Grant MacLean answered on 1 Jul 2024:

---

@axes1wry, I see no-one has answered and I think it’s because none of us were specifically in Cyber Security, so I reached out to a friend for a story. Here’s what I received:

The Incident:

 

In a previous role, we received alerts from our users very early (4-5am) in the morning about emails they were receiving that looked strange. They had come from one of our joint venture companies where we provided 50% of the staff but they had all their own IT systems at the location. This meant we knew the senders by name; but what was strange was those users were actually on annual leave. The emails contained attachments reporting to be project updates, but the text in the email didn’t read like it had been written by the person. Lots of spelling errors and general mistakes.

 

Whilst the team started investigating and making calls to the joint venture, we started getting major alarms from our operations centre. Some users in their early morning haze had started clicking on the attachments and opening them which caused the anti-virus software to detect unusual activity with attempts to install backdoor remote access software also being detected. Oh dear… We placed the machines into isolation and got on the phone to the local IT team.

 

The frantic calls that followed then had us reviewing every joint venture user to see if they were on site or leave; and the sender of these unusual emails was very much not on site. We disabled the account and started investigating with the JV IT team the email server it was sent from.

 

We were able to detect that a hacker (location unknown) had discovered that the email server had not patched a number of critical vulnerabilities (CVE-2021-31207 and CVE-2021-34523 if you’re interested) allowing them to get remote access on the server. The server was located in the middle of the desert, so the internet connection wasn’t great meaning it was missing patches, and the anti-virus software hadn’t been updated recently. We could see from the logs that the attacker had accessed a number of user mailboxes (approx 80); they didn’t download any company data but they did regularly read the emails.

 

The defence:

 

We had to take down the email servers whilst we then checked every other server for infection. The email servers needed to be restored from a 6-month-old backup, patched and then have their security settings tightened. This took at least 2 weeks whilst we then had to review every other system in the environment. This was made harder as the systems were not set to the English language.

 

The attacker wasn’t able to move off the email server and seems in their anger sent out emails to anyone that the joint venture had emailed. This meant we had to contact hundreds of organisations advising them of the infected emails and asking them to remain vigilant.

 

The users were mostly unaffected and were able to work without email whilst we cleaned up. The business quickly gave the IT security team a higher budget to buy new anti-virus software and better monitoring. 

 

Given this was a major project in the desert with millions of dollars of equipment only losing email for two weeks was a near miss…

 

The aftermath:

 

The email server remains to this day in the desert, but with better monitoring and security tools. We did need to reach out to regulators to advise of the data leak but no fines were issued. Wins all round.