Helping remote MSSP/MDR enrich tickets

I’ve worked in the SOC space for almost 10 years and I find a very strong divide of the end customers:
– We allow the MSSP/MDR access to our network
– We don’t allow them any access, all information requests come through us.

Both are understandable, but it’s more work for the 2nd MSSP when they need to complete a tier 1 investigation.  How can automation help?

Imagine a SOAR platform inside the customer, that accepts emails [1] from the MSSP/MDR which contains an IOC/attribute. This playbooks then calls out and enriches accordingly….

  • Query AD for system owner
  • Query Asset Management for context
  • Query SIEM for end point alerts
  • Query Firewall for logs
  • ….and anything else.

The last step in the playbook is to email this report back to the MSSP/MDR, but only after the penultimate check where an end-client analyst reviews the contents first.

 

Workflow

This way, with almost no overhead by the end customer, the MSSP/MDR, who still has no access to the network, any usernames, passwords API keys, is able to enrich their own investigation and in-turn increasing value.

Sometimes the simplest playbooks are the best!

Andy

[1] – Email, or maybe a HTTPs front end, validating source IP, with 2FA, etc

Analyst’s first reaction to seeing SOAR in action

But just a few of the realities are:

There simply aren’t enough professionals in our career.  The vast majority of SOC teams ALWAYS have open head count, and that isn’t going to change in the coming months/years.

Applications increase, traffic increases, network complexity increases, and so the alerts also increase (on top of that your security team is currently POCing even more tools.   More tools, each trying to validate their investment by raising their profile with alert volume, oh the irony).

And SLA agreements don’t care, they expect a ticket to be opened, responded to, and closed.   I’ve talked with analysts that had less than 3 minutes per ticket, I can’t imagine the quality of work, or even job satisfaction here.

 

If these weren’t the reality in IT security I might agree with Raj.  But for us a more appropriate line is:

“this means that when I arrive in the morning I can actually do my job (the cool stuff), and not have to instead simply copy/paste IOC or rinse/repeat the same task 200 times before lunch?”

 

I honestly believe SOAR is more like having a Personal Assistant for all the mundane fluff.

 

Andy

(Thanks James for the inspiration)

Automation at it’s Simplest – Enrichment only

We met a SOC team recently that had a problem with scaling IOC enrichment, but didn’t have the time/resources/appetite to automate anything else (yet).  Can SOAR provide value with such a small scope?

Yes, here is SOAR focusing on 1 job, and doing it well.

  • Technology ‘x’ submits a log or potential IOC
  • SOAR ingests and creates a ticket type “Enrichment
  • If the IOC has been logged before, re-run the existing task and don’t open a new one
  • If not, run against the usual Intel providers (VirusTotal, Whois, Anomali, Censys, SafeBrowing, Shodan, etc) along with file detonation (FireEye, Cuckoo, ThreatGrid, etc)
  • Each output is saved to a summary report
  • If any score comes back as malicious, a new ticket is opened in their existing ticketing system (Zendesk, Jira, SalesForce, etc) with a ticket type “Investigation” for an Analyst to investigate.
  • This first ticket is closed

With this design no analyst has Enrichment tasks, but only when something is bad does an analysts get involved with a 2nd ticket Investigation,

The benefits

  • All enrichment happens 100% automatically – Save analyst fatigue with 100 x copy paste
  • No error in copy-pasting – aka process deviation
  • Enrichment starts with in seconds – boosts SLA metrics across the team

This approach saves several hours a day which can be used to up skill, or maybe even build the next time saving playbook.

Future Work

This time saving deployment was day 1 design.  When proven the next steps were:

  • If anything malicious found, check Splunk/Elastic if this IOC was found internal in last 4 weeks
  • Search packet capture solutions, can we pull a recording of +/- 10 minutes of this IOC being detected
  • Extract logs and attach to the Investigation ticket

Exciting times!

Andy

 

How big is your Toolbox?

A nice easy place to start in SOAR is with an inbound alert (SIEM, email, manual creation/API) that updates a blacklist and maybe informs a user.  It’s something we all do, often, and this can be a nice money and time saver.  But as we look bigger and wider what other technologies can we bring in as we mature alongside SOAR?

So here is my attempt to list all the categories of Security Tools that we as professionals can have at our disposal.  I’m confident this isn’t exhaustive but it opens the eyes to the possibilities, I wonder how many you have and whether they are reaching their potential (I’ll talk more in the future about how we can use them together).

  • Firewalls
  • Endpoint, EDR, MDM
  • Proxy, Reverse proxy, WAF, CASB
  • Messaging (Email, SMS, etc)
  • NIDS / HIDS / IPS
  • DLP and Data Discovery
  • Full Packet Capture and Netflow
  • Asset Management
  • Malware Detonation service
  • Vuln Scanning and Management
  • Deception and Network Access Control
  • SIEM
  • Case Management and Ticketing
  • User management and Authentication
  • Threat Intelligence
  • UEBA
  • …more?

https://www.ncsc.gov.uk/guidance/20-critical-controls

https://www.sans.edu/cyber-research/security-laboratory/article/security-controls

https://en.wikipedia.org/wiki/Security_controls

The Checklist Manifesto

One of the first professional books I read was “The Checklist Manifesto” [1], a short book with powerful takeaways about how working to a checklist can bring value and safety. The book starts with the topic of Healthcare but broadens to include examples from all career paths.

Without giving away the best bits I’ll borrow a couple of the quotes from the book.

Without trying to compare the amazing work of emergency medical professionals to SOC analysts, there is some definite overlap here:

“… then you must add the difficulties of orchestrating them in the right sequence, with nothing dropped, leaving some room for improvisation, but not too much.”
“… at any point, we are as apt to harm, as we are to heal”
“…you have to get the knowledge right and then you have to make sure that the 178 daily tasks that follow are done correctly despite some monitor’s alarm going off”
“But however supremely specialized and trained we may have become, steps are still missed. Mistakes are still made”

Regarding implementing a simple checklist covering extremely basic and obvious steps:

“The results were so dramatic they weren’t sure whether to believe them (the checklist) prevented 43 infections, 8 deaths and saved two million dollars in costs”

And in general:

“Experts are up against two main difficulties […] fallibility of human memory and attention, especially when it comes to mundane matters […] (and) people can lull themselves into skipping steps even when they remember them”

10 years ago when I started working in a SOC environment, when clients had a small handful of tools, tickets were steady so I had real energy and focus to invest every time. These days it’s common to have a backlog so long that we rush and skip critical steps.

You can probably see where I’m going here.  I’ve previously mentioned “Standardise” as a value of SOAR, but how would that look at different maturity stages?

  1. Not-yet-matured SOC: No formalised processes, or processes that are held in our memory
  2. Maturing SOC: Formalised processes are written down in a book somewhere (and only opened when the auditors visit) but actions are still done manually and from memory
  3. Mature SOC: Formalised processes that start automatically, in seconds, without exception, where every single step documented for audit, and have a standardised output with.

Just a thought.

Andy

[1]  ISBN: 0805091742

#Soar.. huh… what is it gooood for?#

#Absolutely… quite a lot.#

Whilst I’m not new to SOC/API/Coding/Integrations, I am slightly new to the concept of SOAR.

So in my first few weeks and months what what potential VALUE have I identified? (I will talk about specific detailed use cases in future posts, today I’m just looking at high level concepts).

In no specific order:

  1. Reduce alert volume – automation closing trivial tasks
  2. Reduce alert overhead – automated manual labour of more complex tasks
  3. Quicker to act – With fewer alerts in the first place, and with the remaining alerts having the donkey work done analysts can get to the meat of an incident quicker.
  4. Standardise your workflow – same work process fires every time, no deviation by junior/tired analysts
  5. Standardise approvals – An incident can mandate direct formal involvement and sign off from HR, business owners etc (non repudiation)
  6. Standardise your playbook – use a playbook that maps to a framework (e.g. NIST)
  7. Coach new tier 1 analysts – An interactive log of all tickets, and a playbook helping hand for them to get up to speed.
  8. Automation of boring/simple/repetitive – reduce eye and brain strain by removing the tasks that you have to do every day
  9. Revitalise legacy/simple tools – Things you can’t afford to replace (mission critical, expenseive..), and hook them into your modern security stack using SOAR
  10. Aid Incident Response – Have your IR team review your playbooks so that the right information is collected for post breach analysis.
  11. Reduce risk – Some remote selling services have moved to DTFM Payments to remove risk and scope of internal abuse. I see SOAR having a similar benifit to SOCs.

I have no doubt I will revise and restructure this list as time goes on, and add specific real world use cases.

Andy