Presentation made using reveal.js. Type ? for navigation help, or just skip through using space key.
Oslo, November 20th 2015.
On modeling CSIRTs
213 slides long (sry) · Made using Reveal.js
Because I believe we are being
outperformed by our adversaries
Not The Answer™ but rather
the status of my own research
and learning experience
(Which is far from done...)
I hope to spark some interest
in CSIRT modeling, I hope for
feedback, comments and new ideas
and I hope you find it useful.
A quick intro to the presentation.
This presentation has
grown beyond 200 slides.
So a presentation of the
presentation is in order.
First there's the introduction,
where I present the backdrop
of my work, as I perceive it.
(If you click the link, hit back in your browser to get back here.)
Next we inspect two critical loops
I believe every mature CSIRT
should be an important driver for.
(Sort of to set the CSIRT stage before discussing models.)
Then there's a short break
for comic relief as we look at
5 fun facts about your defense.
(And yes, they're funny because mostly they're true...)
We then have a very quick
look on our adversaries.
(But only briefly. This presentation is about us, not them.)
An equally brief section on the
analytical bedrock of IR analysis for
countering our adversaries follows.
(We're not as special as we think...)
Finally it's time for one of the
major models of the presentation,
the CSIRT Analysis Pyramid.
During the presentation I use
the word tactical a lot. So we
divert shortly for a definition.
Next up is a deep dive into a
key concept of all the models,
namely tactical analysis.
Before we delve into the CSIRT OODA
loop, we take a look at the original.
And finally it's time for
the CSIRT OODA loop.
(Let's get inside that adversary OODA loop.)
And that's it. I hope
you will enjoy the ride.
(Coming up next is the introduction.)
But it's not all fun and games...
Throughout human history we have always had conflicting interests, competition, conflicts, unrest and war. We have also always had crime. So we have more or less always had soldiers, spies, rebels and criminals; armies, intelligence organizations, rebel movements and criminal syndicates. None of these things are new.
Yet we seem to have been caught completely off guard when we — after having invented computers; decided to connect every single of them to the same network; and then digitizing every piece of information and every service and stuck it all onto said interconnected computers — discovered that all those people adopted new technology really well and very quickly.
The result? We have opened the gates and brought back raiding and pillaging at a level not seen for a long, long time, at least in large parts of the world.
We have enabled spies and criminals to operate at large scale with little risk of being caught and brought to justice or face any kind of consequences.
Their cost / benefit ratio is ridiculous.
We have a leaky boat design, but instead of fixing the leaks, we are just making faster and bigger boats with more bells and whistles to carry more people more often.
And more often than not security people are only brought in to bail water out of the boat when things get too bad, instead of being asked to help fixing the design.
(I've sort of addressed this issue before, talking directly to developers: The Internet is on fire: Don't just stand there — grab a bucket)
And boy, is there a lot of water to bail. We hear about new critical vulnerabilities and severe security breaches almost every week. Big companies, everywhere.
We are experiencing breach fatigue.
We have actually had to conclude that we are unable to design, deploy and operate IT infrastructure securely.
Your prevention never holds up.
And worse, we don't seem to learn. We
keep putting way too much faith in prevention.
How sustainable does this sound to you?
We need a shift in thinking
from Secure IT to Defendable IT.
(Or Rugged IT or Sustainable IT or whatever. But better!)
This presentation is dedicated to
CSIRT models and methods I hope
will help pushing towards defendable IT.
But first, a quick look at how
a typical CSIRT engagement
at a mature CSIRT looks.
driven by high functioning CSIRTs
Let's have a look at a typical IR engagement.
And there you are, happily spinning
in your SOC hamster wheel...
(Doesn't sound familiar at all, does it...?)
Now let's have a look
at the improvement loop.
So 7 improvement vectors ...
... vs 1 improvement vector.
Your overall security performance probably
improves more with 8 than just 1, right?
To outperform your adversaries and avoid damage and loss, you need to iterate both loops faster than your adversaries are advancing their ability to breach you.
Your job is to outperform your adversaries.
Through process and capability maturity:
N/A · Ad hoc · Defined · Managed · Optimized
But how — I hear you cry...
The CSIRT Analysis Pyramid
The CSIRT OODA Loop
We'll get back to that in a bit.
(But now — some fun facts)
about your defense — that aren't
OK, I'm just guessing, but bear with me...
1. Your network isn't properly segmented.
(Seriously, they never are...)
2. Your servers aren't properly patched.
(Not all of them, not all of the time. Just, nope...)
3. You don't have most logs easily available.
(Actually, we have all the logs we need — said no DFIR pro, ever...)
4. Your security monitoring is weak.
(More often than not it's actually a joke...)
5. You don't really learn from incidents.
(Most of us aren't anyway, and should listen to Russel Thomas.)
Bonus: You don't know your adversaries.
(Which incidents brought onto you by which adversaries?
What are their motives and their capabilities?)
We have brought a broom to a gun fight.
We want to engage, but are mostly
equipped to clean up after the fact.
According to our ISMS we are compliant...
they are pondering their strategies
They ... they're compliant.
We're just ... out of options.
They are of course not out of options at all.
Want to know more? Go search for:
security crime espionage threat report ext:pdf
I have also written two (hopefully)
relevant essays on the topic:
I'm not delving deeper into
the subject of adversaries now.
But rest assured. They are there.
Plotting against you...
Stopping your adversaries in their tracks.
Hypothesize · Collect · Detect · Identify · Act
It's quite similar to the intelligence process:
Requirement ⇨ Plan · Collect · Analyze · Assess ⇨ Exploit
Next: Let's apply it to incident response!
Goal: Getting from incidents via
threat intel to risk management.
Objective: Driving the improvement loop
so you consistently achieve improvements
along as many of the 8 vectors as possible.
But first, a few words on
the hierarchy of knowledge.
All analysis starts by collecting data
from the real world. No data, no analysis.
Information is what you have after having processed, enriched and recorded your data.
To get from information to knowledge you must correlate, analyse and interpret the meaning hidden within the information.
Once you have built knowledge on
something you can gain insight
into relevant consequences.
And once you have gained that insight
you have the opportunity to share it.
And once shared, it can be used
to make informed decisions.
And that is the goal of this model.
Supporting informed decisions.
Going from technical incident data
to useful incident information
requires technical analysis.
But there is a lot of information
available in other sources as well
and you should exploit that also.
Transforming available incident information
into knowledge about what is going on
I prefer to refer to as tactical analysis.
Through post incident analysis focusing on business and mission consequences you can gain insight into affected risks.
Tactical analysis is the bridge between
(incident driven) technical analysis and the
risk management process in your company.
Tactical analysis is a crucial driver behind
many of the 8 mentioned improvement vectors.
The CSIRT Analysis Pyramid builds on the
following capability model for the CSIRT:
(I will expand on that another time)
You keep saying
A tactical action is one that
you do as part of a plan for
achieving what you want.
The art or skill of
employing available means
to accomplish an end.
The science and art of disposing
and maneuvering forces in combat.
Collect and analyze all available
clues to establish the course of
action, motives behind that action
and identify the people involved.
(As used by e.g. Norwegian police)
And now a term you're familiar with:
Tactics · Techniques · Procedures
(As used in e.g. counter-terrorism and infosec)
Do yourself a favour, read this: Ryan Stillions: On TTPs
Now, how do you analyze these things?
Enter tactical analysis.
As awesome as it sounds
Tactical analysis is something you
undertake to uncover your adversary's
tactical actions and mission objectives.
That's my definition anyway.
Also check out
When Threat Intel Met DFIR
It is absolutely brilliant.
Why is what being done by whom?
This is supported by technical analysis, which answers the questions when, where and how was it done.
Where technical analysis is finding the puzzle pieces, tactical analysis is piecing the puzzle together to glean the story being told by the available evidence.
Where technical analysis is looking for the relevant, dots, tactical analysis is connecting the dots and becoming aware of the shapes hidden within the mess.
Collect and analyze the evidence
to learn how and why your
adversaries operate against you.
You need to uncover this if you want to
be able to plan and execute a proper
I am working on a practical guide with
research questions for doing tactical
analysis during incident response.
(Unfortunately they're not ready for presentation yet.)
Some additional points
What is this adversary doing in my
infrastructure right now, and why?
What is this adversary usually doing
within my infrastructure, and why?
Based on all my prior knowledge, how
big a threat is this adversary to us?
And to follow that trail further up
the CSIRT Analysis Pyramid:
Based on everything I know about both
my company and this adversary, what kind
of security risk is it posing to us?
And how does this security risk influence
the total mission risk of our company?
And what can and should we do about it?
Now we're well outside and above the CSIRT,
but hopefully you get why it is important
to develop and communicate this insight.
Tactical analysis is the bridge between
incident driven technical analysis (and
threat research) and the risk management processes at your company.
And there are probably more cookies to be had by CSIRTS that manage to show that they are relevant to the risk management processes at their company than not.
A story about dogfighting
In a dogfight you act quickly
and correctly. Or you die.
Wikipedia: The OODA loop.
Notice the similarities
to the intelligence cycle?
It's actually similar to all decision making.
Next: Let's apply it to incident response.
WARNING: Brace for impact...
(Complicated figure ahead)
From intrusion to response
This will all be simplified,
but please bear with me.
What do we want?
Access to information!
When do we want it?
Preferably straight away!
What do we want?
Control of the capability!
When do we want it?
Preferably straight away!
Attackers want two things. Accass to information and control over capabilities.
Access example: Strategy documents.
Control example: Computer to send spam.
From the attacker's side there are
three main foothold perspectives:
ENV example: Running malware "as user".
SYS example: Rootkit.
From the defender's side there are
five main footprint perspectives:
APP example: Browser cache.
ENV example: Autoruns.
SYS example: Authentication logs.
Most of the time incident response starts with the discovery of suspicious or known bad behaviour and activities on one or more of your IT systems.
As you discover more (possibly) related evidence within the available forensic footprint, it is important that it is properly recorded, registered and organized.
Now starts the challenge of identifying which of the discovered evidence is relevant to understanding what the attacker is doing within your systems.
The technical analysis, evidence triage and information management part of your incident response engagement is closely related to the observe stage in the OODA loop.
Next comes the orientation phase, where you analyze your findings, look for correlations and connections and figure out what is going on.
This stage is driven by tactical analysis.
You are literally
connecting the dots.
Once you have figured out how your evidence connects, you need to interpret it's significance and the story it tells.
Forensic evidence is the show part of show and tell, but you also need to be able to tell the story. What is really happening?
You must establish and test hypotheses. Rigorously. Or the evidence will only tell you what you want it to...
CHINESE APT ALL TEH THINGS !!1!
Once you have a good understanding of what's going on, it's time to plan a response.
Key point. Plan, then act.
And when you do act, act decisively.
(E.g. reinstall all affected computers simultaneously.)
But don't just react. Think ahead.
Plan for your adversary's response.
Getting inside your adversary's OODA loop means performing these steps faster and better than the tactical actions of your adversary during your engagement.
To drive the analysis and response process
during an incident you need to staff up at
least three distinct and separate roles.
1. Incident lead
Coordinator, process driver and responsible for the outcomes.
2. Tactical analyst
The big picture person, setting the analytical course.
3. Information manager
Managing the inevitable information overload.
These three distinctly different roles should comprise the core team of an incident response engagement.
They obviously need support from technical analysts, management, operations and so forth. You need more than three people involved in an incident engagement.
But an incident engagement not run by a core team will most likely struggle with focus, process and progress.
Achieving that is the core team's function. Keeping the right focus, adhere to established processes and make sure you make good progress and get results.
Do this right, and you may
outperform your adversaries.
Do this right, and you can
clench your fist in triumph.