As a UK member of the Threat Operations team, I have the immense pleasure of working with colleagues in the USA and Australia. When we are not busy trying to create pithy memes for the latest incident, we happen to do some cool defensive security work.
There's a careful choreography across Huntress that supports and enables the team to shut threats down before they have the opportunity to ruin anyone’s day. (I guess we ruin threat actors’ day?) But that sounds like a them problem.
Keep reading to take a look behind the curtain to see how the Huntress Threat Operations team functions. And you may as well take a peek with me of all people, as I only turned up here in January 2022!
Passing the Torch
When I start my day, I’m picking up where my fantastic Australian colleagues have left off.
Something we do ridiculously well is pass information across our 24-hour teams. This will be a common thread you find in this article, but we take significant care to ensure a consistent service is delivered and our colleagues are able to pick up and run with any leftover investigations.
There’s a brilliant rhythm we’ve developed to hand over incomplete cases between regional teams. Even if someone only has time or capacity to contribute one or two points for a case before they hand it over, it makes a huge difference.
And we must be doing an alright job to earn the praise of Threat Operations Director, Max Rogers:
Syncing Up Internally
Syncing up with our team is one of the highlights of my day.
Sometimes it’ll be Zoom calls, just catching up as a squad. We’ll use this time to share how some tooling is working or not working and brainstorming possible improvements for how we do things.
But a global team means that communication will be asynchronous, and some members of the team don’t get to contribute at that moment in time. But no matter where someone is in the world, no one is left out from feeling the love.
Investigating
Once something sus pops up, we use every tool and telemetry source we’ve got available to get context, findings and solutions. The more we’re able to discover, the better we can help the partner in question with a more detailed analysis.
Our alerts vary depending on severity. For the real spicy stuff, we have our Slack webhook to give us a jolt!
And for some other alerts, we can play with the aesthetically stunning process tree that has been newly unveiled for the team for our rollout of Process Insights.
And we’re constantly tuning detections and alerts to reduce false positives and ensure that the volume of data the team sees has been surgically filtered to only include likely true positives!
When we’ve got some down time, we’ll escape the shackles of set detections and start to explore our telemetry for new and novel attacks that don’t yet have detections.
One of the tools to do this includes a gorgeous dashboard.
And sometimes we can use dashboards that are less aesthetically exciting but actually bring you closer to the raw form of data, which is really useful when hunting novel and new threats.
Other times, we’re digging into the detail of various third-party logs and data sources on a machine to try and contextualize an initial alert we receive.
Reporting
When we investigate an alert, we’re obsessively collecting screenshots and evidence, and sharing our hypotheses with the team.
I could probably write a 10-page report for each case! But when we’re done investigating, we keep the partners’ perspective in mind—and no offense to us, of course, but a partner probably doesn’t want a detailed 10-page report. 🤓
Here, a careful sifting occurs where the data we have collected is funneled down into a concise evidence-led timeline of the attack, with advice on how to respond to this attack.
We take care in how we word these reports. They’re all evidence-based. We’re careful not to blame users, but we do use language that shows the user account has been leveraged and compromised unknowingly.
Documentation
As a 24-hour operation here at Huntress, we have to keep especially good documentation and over-communicate. This is critical to ensuring that my US and Australian colleagues can have access to material even when we Brits have signed off for the day.
We document seriously cool stuff, ranging from how to leverage forensic artifacts to clear the fog of an investigation, to how to use particular tools for malware analysis.
When we’re done with novel investigations, we do really short write-ups summarizing what we found and how we found it. What would an investigation be without knowledge-sharing at the end?
Support
The dialog between partners and our team is facilitated by our brilliant support team. For especially strange incidents and occurrences, partners and support will drill down and look into a case. And when they reach the limit of evidence, sometimes the ThreatOps team will be recruited and deployed!
These are some of my favorite cases to take, as they're always strange and spicy.
Engineering and R&D
It is humbling to work with people so clever, creative and modest. We on the ThreatOps team have regular conversations with engineering and R&D—sometimes about feature requests, roadmap development or even when an incident is occurring and it’s all-hands-on-deck.
The wealth of knowledge and precision of execution that stem from engineering and R&D drive the underlying mechanics of ThreatOps. If ThreatOps is the engine for investigations, then engineering and R&D are the expert pitstop crew at an F1 race.
Contributing to the Community
The genesis of tweet threads like this come from short conversations where we’re sharing our findings and wondering the quickest, best way to communicate this information out to the community.
And on other occasions when we want something more substantial (and with better memes), we work with the content creation team. We hammer out draft after draft, making sure that what we’re serving up for the community is not just piping fresh but is fun, correct and valuable.
Making the content team snort is just an added bonus. 😁
Serving up Security
There are probably a billion things, people and processes I’ve not mentioned, given we’re just talking about the ThreatOps team. There’s a small army of talented people across departments who aren’t mentioned here—but rest assured they’re darn important, and I'm humbled to make a small contribution to what we do here.
Was this interesting? Did you enjoy a peek at the factory floor for one of Huntress’ teams? Maybe you enjoyed it so much that you want to work here?
Well, take a look at our available openings.* I’m sure you’ll find a role you like the look of, and we’ll have you on board serving up piping hot security work soon enough.
Think you'd be more interested in the work our threat researchers do? Stay tuned—that blog is coming soon!
* Don’t tell them I sent you - that’s a sure-fire way to get your application thrown into the fire.
