Best Practise : How to POC/POV SOAR

“Best Practise” : commercial or professional procedures that are accepted or prescribed as being correct or most effective.

Over the last 2-3 years I’ve carried out a LOT of SOAR workshops and POC. Here are my findings:

Maturity of environment/processes/team

There are no prerequisites to using SOAR… however… SOAR was created to fix the pain. If the team does not suffer pain (yet), the value of SOAR might be hard to convey.

  • Has the prospect performed any process enough times to truly understand it? i.e. Are processes defined and understood?
  • Has the prospect identified which processes cause the most issues, and which issues? analyst burn-out, process deviation, slowness, etc
  • Has the prospect tried to create homebrew automation? And how long did they last before they realised it doesn’t scale (and unless you have the resources of Netflix, your homebrew won’t scale)

POC Success Criteria

Your SOAR project likely needs some executive sign off, and the exec team drivers are typically similar to the technical team but positioned different.

Usually the technical team want to see:

  • Speeding up time to visibility
  • Reduce clicks to resolution
  • Automate challenging processes
  • Time savings / quicker results to business
  • A reduction in stress
  • Reduced errors/deviation
  • Process enforcement
  • Alert consolidation
  • Deliver more with same team size
  • Deputise(/delegate) processes to other teams
  • More examples here

Whereas I often see executive teams have the same concerns but framed differently:

  • Strategically over the next 5 years we need to offer more services (either internal, or to customers) therefore we need to prove that SOAR can automate new business flows allowing expanded/improved service delivery…. aka automate processes
  • To comply to regulation xyz we need certain process to be enforced, full and perfect audit records…. aka process deviation and logging
  • The cost of the platform is compared directly against the cost of increasing team size (which is hard due supply/demand of skilled IT analysts/workers). Therefore for every $xxx of platform cost, you need to automate yyy Hours… aka time savings
  • etc
Interestingly, in 3 years I never hear companies say "we need to use Automation to decrease this team size to save money".  
It's always the opposite, we need to grow but we're simply unable to, it's about doing more with what you have, not doing the same with less. 

With these high level goals identified we know if POC will focus on Automation vs Case Management vs ChatOps vs TIP/TIM vs reporting and visibility vs ….

Preparation and Delays

By far, the biggest delay in any POC is internal preparation. For each technology that SOAR integrates with, we need a password/API key/private cert, we need network access. It’s not always the security/SOC/IT team who owns every technology, this often leads to big process delays. The same issue with firewall rules, network access, etc. Be prepared to power through.

The good news is I can fall back to other options if integrating with your technology isn’t possible. Here are some examples:

  • If your internal ServiceNow team are unwilling/incapable to create a dummy account for testing, create a Developers instance at SNOW direct. Their dev instances only lasts for 10 days but is great to use and abuse to prove the functionality
  • If you can’t integrate to technology <ABC> directly, consider ingesting logs via existing technology paths, e.g. SIEM
  • If integrating with Active Directory is tough, ask your SOAR vendor to provide a mock integration that generates/consumes fake data as a way to show the workflow processing and completing

Think Blueberries not Watermelons

To begin with I strongly recommend small usecases that saves you 10-20 minutes of work, multiple times a day. This quick win is easier to design and deploy, and you’ll notice the impact more. Small and often, think Blueberries.

As SOAR matures, start to consider usecases that are huge but infrequent, there is still value here but it isn’t always as quick to realise. Infrequent and huge, think watermelons.

Think C-P3O not Skynet

You might be ready for automation, but elements of the business might not. Also I find many companies don’t understand their process as much as they thought they did.
Therefore when designing a usecase, I don’t plan for SOAR to do 100% of the work and make 100% of the decisions. This would be Skynet, and Arnold Schwarzenegger will teleport from the future to stop you.

Start smaller with SOAR performing the laborious work, but not doing any decision making, use manual checks at every important step. Remember SOAR should take the arduous work away from you, allowing you to merely supervise, authorise and guide workflows.

Think C-P30

Think Avengers, not Thanos

Each time you acquire a new technology you have to decide which vendor is best. I’ve often seen companies say “product A had 92% detection, product B had 93% detection, therefore B is better we shall buy their warez”

How often do you rate a technology in it’s ability to work as a team?

In my experience, the best setups emerge when we get multiple technologies working together as a team getting the best out of each other, like the Avengers.

A technology that is 1% stronger than each of it’s competitors, but can only be operated by a human analyst will fail when compared to team work, just like Thanos did.

I hope this is helpful?

Andy