At this years DSEI conference, a figure was quoted by a number of senior UK military leaders of 6.8million cyber attacks hitting Ministry of Defence networks over the last year - but what does that number actually mean? I’ll be honest, my personal opinion is that it’s completely made-up.
The MOD are not alone in producing big numbers associated to the cyber threat - a quick google for the phrase “million cyber attacks” reveals a number of similar claims:
The Taiwanese Government were hit with 5million attacks DAILY
The 2023 Nigerian Presidential Election experienced 12.9million cyber attacks
The 2018 FIFA World Cup hosted by Russia had 25million attacks
The Port of Los Angeles claims 40million attacks every month
Switching up the search to a “billion cyber attacks” and there are even more, with one example cited of the Northern Ireland Assembly experiencing 1billion attacks in 2022! By comparison, the MOD seems to have got away lightly - maybe we should cut their budget for cyber defence as they clearly don’t need it as much as others…
The reality is, given how massively subjective these numbers are, they simply aren’t comparable - and arguably as a result, fairly pointless. So lets pick apart some of the common issues with these sorts of claims.
Language
It’s probably worth just thinking about some of the very specific terminology that the cyber security domain has developed over the years, that’s designed to help ensure that we can be really clear with what we mean.
A “Cyber Attack” is initiated by an attacker. For a defender, the first thing we see is “an Event”, this triggers “an Alert”, which is then “Investigated”, resulting in either a “True Positive” or “False Positive”, with those “True Positives” raised as an “Incident”. Simple, right?
Ok, “An Event” is something that was observed, a symptom of the attack, and in a cyber security context this could be a whole range of things - a phishing email, a DDOS attack, unauthorised software installation, even ransomware.
“An Alert” is the output of some form of detection system, whether rule-based (and there’s a match against a list of previously observed “bad things”), or Machine Learning based (and the activity looks unusual or suspicious compared to what it knows about either good or bad).
“An investigation” is a set of manual steps (often supported by automated tooling to an extent) where a SOC analyst will review the alert output of their tooling, combined with some open source research, other contextual information from their environment, and maybe some technical deep-diving to determine if the alert was valid given the event that generated it. This results in either a “True Positive”, meaning the alert correctly identified a malicious event, or a “False Positive”, in other words the event behind the alert wasn’t malicious.
Finally, “an incident” is then a broader set of steps taken by the security team to suitably resolve the issue and get the business returning back to normal. This process can be fairly time consuming (depending on the size of the network), and it’s not uncommon to see some organisations take weeks if not months to recover from an incident, especially a destructive incident such as ransomware.
For me, “Cyber Attack” refers to the full end-to-end encapsulation of actions by an attacker. Considering the Cyber Kill Chain as a reference point, this includes everything from the initial reconnaissance activities all the way through to the final actions they take to achieve mission success. But worth remembering this is from the attackers viewpoint, and defenders may not have the full view of the totality of everything an attacker did.
So from that defender’s position, when an organisation quotes a number of “Cyber Attacks”, what are they specifically referring to? The number of fully triaged, investigated and resolved Incidents? Highly unlikely - that’s a lot of effort to connect events together, triage, assess and manually count. The reality is that its often a mixture of the “Events” and “Alerts” observed by the system - and there are a whole range of issues with that.
1 event ≠ 1 alert ≠ 1 incident ≠ 1 “Cyber Attack”
In many cases, the actions taken by a would-be attacker will involve multiple lines of activity, across different parts of the estate. From conducting a scan of the external footprint of the network to identify vulnerable hosts, to sending phishing emails to a good chunk of the work force, to installing malware on compromised hosts, to moving laterally, to exfiltrating data <deep breath> and more besides. Each individual stage of that attack could generate not just one, but multiple alerts. So when we’re counting, are we treating every phishing email as a single atomic attack, or are we combining those sent as part of a common campaign as a single “Cyber Attack”? The scanning of a network, is that a single contained incident, or are we treating each individual network ping and TCP connection request as their own individual cyber attack? Tbh, when I see really big numbers associated to cyber attacks, it’s typically down to this dissection of a network scan to its individual constituent parts, each treated as an “attack” in it’s own right which is typically the culprit.
Think of the following example - an attacker gains access to a network and wants to know what devices are present. They run an nmap scan across a subset of the network using the following command (in reality, this is quite a basic/noisy approach, but not unusual, so bear with me):
$ nmap 192.168.1.0-255
This will attempt to send a SYN packet to the 1000 most commonly used ports on the 256 IP addresses in the range above. That could potentially generate 1/4 million network packets, or 1/2million if you include both TCP and UDP ports. So, do you count the 1/2million packets, or the single line command entered by the attacker? If you don’t have visibility of the command entered, how would you even know?
Getting to the root cause
As alluded to above, there is a big difference around cause (a single line command) and effect (>1/2million SYN packets). It’s important to recognise that the alerts a SOC team receive are only the symptom of an overall incident, from which they need to determine the root cause. This is no simple task - and depending on the visibility the team has, they may never be able to fully identify what happened. Worse still, how do you define intent? How do you determine whether the symptom wasn’t just an unintended byproduct of a completely benign activity? Or even just unwanted activity such as email spam?
Lets say an alert gets triggered for an unauthorised USB device plugged into a corporate laptop - is this the action of an insider threat trying to pilfer sensitive corporate documents; early warnings of the next Stuxnet attack; or just Barry from accounts who got a new novelty mouse and keyboard set for Christmas that he wants to use at work. Depending on how you’re counting it - could all of these be defined as a “Cyber Attack”?
You can only count what you can see
It’s somewhat obvious, but your ability to count something, is completely dependent on your ability to actually see it. Otherwise it’s like running an accurate census on the Loch Ness Monster, Unicorns and Big Foot1. When you hear references to “Zero Days”, “Sophisticated Attackers” and “unprecedented", how on earth can you be expected to see, observe and thus even count this activity with any form of accuracy? The MOD’s 6.8million attacks can therefore only really relate to the stuff they already know about and can see, which for the most part, will be just the “unsophisticated” and “precedented” attacks. Arguably this is just a measure of the background noise of the internet, the non-targeted dross that everyone gets hit with every day. Surely what would be more interesting is threat activity that is just targeted at the MOD, the bit that makes them special or needs them to take special action beyond just basic cyber hygiene steps. Unfortunately, that’s really hard to measure.
If the number goes up next year, is that good or bad?
Cyber security as a business function presents a challenge for finance and audit teams. A lot of the successes are somewhat intangible, more an insurance policy that you only really test when things go wrong, not when they go right. Much like a goalkeeper who are assessed on the goals they let in, rather than the ones they stop. So if you spend $1million on a technical countermeasure - how do you determine the Return on Investment? As a result, anything that can generate a number becomes invaluable - an issue that often results in the misuse of Phishing Simulations.
As recently explained in a great FT article, sometimes there are reasons beyond the threat changing that makes the number change. Maybe our detection got better and as a result we can now see more of the attacks that were always there, but we just couldn’t see them before. Maybe the methodology for counting changes, for example we decide to include or exclude certain attack types from our totals. Maybe the attack surface changes (for example, you roll out a new network or publish a new website), and whilst the number of attacks per endpoint may remain stable, the total attack count increases.
Or maybe your detection actually got worse. I remember the story of one SOC analyst who had spotted what they thought was an incredibly unique string of characters that almost every phishing page they looked at contained. With this potential silver bullet, it was deployed… and promptly blew up (metaphorically speaking) the SIEM that was being used - can you work out why the following Hex string might be not be the best signature?2
68 74 74 70 73 3a 2f 2f
Or maybe it’s a sign that preventative actions are actually failing and more attacks are getting through the first layer of defences. Not necessarily a great measure of the success of your cyber security strategy!
So, if MOD’s numbers go up next year, that might be actually a good thing. Or maybe a bad thing. Or maybe it might mean nothing at all.
And, according to my son, Vikings. Don’t ask.
For those struggling, its the hexadecimal representation of the string “https://”… which is fairly common on most websites these days (good and bad!).