Security Automation Shopping?

In this edition, I want to tackle a question I often get asked: What are the requirements when buying a SOAR or Hyperautomation solution? Essentially, this is your buyer’s guide.

Now, you’re probably wondering: What about all those guides vendors host on their websites? I’ll admit, there’s often some solid insight in there. But keep in mind, these guides are part of their marketing materials, so they’re not exactly unbiased. Their checklists are often designed to conveniently match the platform they’re trying to sell you.

Here, my goal is different. I’m giving you a genuinely neutral, no-fluff guide. The questions I’ve laid out aren’t tailored to sell you on any one platform, but instead, they’re aimed at helping you find the best solution that fits your unique needs—whether that’s SOAR, hyperautomation, or something else entirely.

Instead of the typical Excel spreadsheet approach (which, as I’ve said before, isn’t bad in some cases—I’ll explain when), I’m giving you a set of critical questions to ask yourself and the vendor when evaluating security automation platforms.

Bookmarks

• Where to Start: Who Are the Stakeholders?

• Do You Have Existing Automation Capabilities?

• What Is Your Current Org Structure?

• Do You Need the Platform Purely for Automation, or …

• Key Questions to Ask the Vendor

• Scoring spreadsheet

• Conclusion

Where to Start: Who Are the Stakeholders?

Why this question is important: Are you buying the platform to serve just SecOps and IR, or will it support your entire security stack? (Check out my blog on the difference between SOAR and Hyperautomation) Knowing who the stakeholders are will help you determine whether you need a SOAR solution or a more comprehensive Hyperautomation platform.

Do You Have Existing Automation Capabilities?

Nowadays, many security platforms offer some level of automation. Most SIEM vendors come with SOAR capabilities, and the same goes for EDR, CSPM, IAM, etc. Make a list of all the platforms you’re already using that offer automation and see to what extent you can automate using them.

Now, I know I said earlier that an Excel spreadsheet might not be the most exciting way to tackle this… but hey, I told you it makes sense sometimes! Just imagine that sweet moment when your spreadsheet finally does something useful—like showing you exactly where you’re already automated. That’s the kind of magic only Excel can deliver. 😄

It’s helpful to have a list of your current SOPs and map them to the platforms that can automate those processes. If you find you’re already covered, you might not need a dedicated automation platform. If not, proceed to the next question.

What Is Your Current Org Structure?

Do you have an engineering team that’s proficient in scripting languages? Or is your team mostly made up of analysts? This is crucial because it will help narrow down whether you need a low-code platform or one that’s entirely no-code.

Do You Need the Platform Purely for Automation, or Do You Need Case Management Too?

This is one of the key questions because not all security automation platforms offer case management capabilities. Some are strictly focused on automation, while others include built-in case management.

Key Questions to Ask the Vendor

• Code Change Tracking Ensure the platform has clear auditing of changes—what was changed, when, and by whom. This should be visible in the UI, as well in the logs.

• Versioning: Every change to an automation should be saved as a new version, with rollback capabilities. This is especially important when handling sensitive automations with elevated access rights.

• Approval Process: Just like a pull request workflow, no one should be pushing automations directly into production without review. Automations are powerful and typically run with elevated rights, so a mistake could trigger a domino effect. For example, you might accidentally block a legitimate domain across your entire company due to a faulty IOC. This is why approval processes are crucial.

• Testing: Automations should never be pushed into production without testing. Make sure the platform includes QA features for pre-production testing.

• Auto-Remediation: Many platforms offer auto-remediation capabilities, but in reality, implementing this isn’t always simple. Remediation usually requires change management integration. If the platform integrates with your change management tools (e.g., ServiceNow, Jira, or via Git for infrastructure as code like Terraform), you’re in good shape.

• Integrations: Don’t just look at the vendor’s checklist to see if they support the tools you care about. Make sure the platform offers the right integration depth. Some platforms only support a handful of API actions that may not be relevant to your use case. Also, can you develop custom integrations, or are you dependent on the vendor? Be cautious with promises of “quick” integration turnaround times—they often slow down once the vendor grows and gets overwhelmed with requests.

• Pricing Models: Pricing varies. Some vendors use the traditional per-user model (common with case management-heavy solutions), while others charge based on the number of integrations, data processed, or workflow runs. Be careful—what starts cheap can get expensive as you build more automations. Keep in mind that new models often aim to make it harder to switch platforms later by embedding automation deeply into your processes.

• Multi-Threading: This one’s about scale. Can the platform run multiple automations at once? What happens if an automation gets stuck waiting for a response—can other automations run in parallel? What happens if there’s a spike, like during a phishing or DDoS attack? You’ll want to know where the bottlenecks are.

• Data Processing: Especially important with SaaS vendors, understand where and how your data is processed. If you’re automating outside of traditional SecOps use cases, you need to ensure the platform handles your data correctly and complies with regulations. Also, does the vendor train their AI models using your data? Are you comfortable with that?

Scoring spreadsheet

Now, I know I’ve been teasing the traditional Excel spreadsheet approach, but let’s be real—it has its place. Putting together a list of requirements in a spreadsheet can be a good way to evaluate multiple vendors side by side. For example, when you need to compare which platforms have specific certifications (maybe ISO, SOC 2, or GDPR compliance), pricing models, or which integrations are covered, a well-organized spreadsheet can help keep things clear and structured.

But beyond these checklist items, I’d encourage you to shift focus to actual use cases. Instead of just ticking off boxes, ask yourself and the vendor: Can my users do X, Y, and Z with this platform? Can they automate a specific workflow, integrate with a certain tool, or scale the automation as your business grows?

At the end of the day, both approaches have their merits. So, whether you’re listing out those must-have features or deep-diving into real-world scenarios, use whichever method helps you get the best insights—whether that’s the trusty Excel sheet or a more hands-on, use-case-driven evaluation.

Conclusion

This guide should help you navigate the critical questions to ask when considering a security automation platform. Of course, there’s always more to explore, but these questions cover the essentials that apply to most teams in the cybersecurity space. What questions are you asking your security automation vendors? Let me know—I’d love to hear your thoughts.

If you want to get on a call and have a discussion about security automation, you can book paid consultancy here:

Become an Ultimate Supporter of our blog and gain exclusive access to cutting-edge content, while playing a pivotal role in sustaining our community.

By joining the Ultimate Supporter tier, you decide how much you wish to contribute, directly aiding in the maintenance and growth of our website. Your support helps us cover essential costs, ensuring we can continue to deliver top-notch insights and tools for engineers and cybersecurity leaders.

As an added benefit, each Ultimate Supporter will receive a link to the editable versions of the visuals used in our blog posts. This exclusive access allows you to customize and utilize these resources for your own projects and presentations.