By nature, threat hunting requires a series of traits: natural curiosity, a love of learning and the desire to break (and then fix) things - to name a few.
Not only do these traits contribute to the well-being and cybersecurity posture of an individual company - they also improve the security industry as a whole. It’s an unspoken ethical commitment that whenever someone joins the cybersecurity field, they must be concerned about the greater good - looking past profit margins and bottom lines to lift others up and share their knowledge.
To uphold our end of that commitment, we’re detailing a recent vulnerability we stumbled across at a conference we recently attended. The vulnerability, which has since been remedied by the vendor, impacted survey software built specifically for MSPs.
Read on to learn what we found - and how the vendor was able to quickly remedy the vulnerability.
What Did We Find?
During the week of September 20, the Huntress team attended the “BigBig” conference alongside a handful of other industry players in the managed service provider channel. Vendors showcased their products and services, delivered presentations and participated in panels for the greater MSP community members and security practitioners.
One of our Huntress team members was invited to speak at the event and was introduced to a customer satisfaction and employee recognition platform called Crewhu to help audience members easily offer feedback on a given presentation. This system used a simple QR code at the end of each presentation so attendees could easily scan it and be presented with a form to offer feedback.
Somehow, our Huntress team member was given the wrong QR code. To try and understand how these QR codes were generated for each presenter, the team member scanned a QR code at the end of a different individual’s presentation. While examining the survey page, they noticed there was additional information included in the HTTP response - information that should not be included in a survey.
The response page included JSON data with settings and configurations for the survey itself as well as sensitive information that no end user should be able to see.
The survey included the following information specific to the company it was collecting feedback for:
- PSA or integration server domain name and URL
- PSA or integration server username in clear text
- PSA or integration server password in clear text
- PSA or integration server public and private key in clear text
Crewhu's Response to the Vulnerability
We believe that it takes a community of like-minded MSPs and MSP vendors to keep the channel secure, which is why we thought it was important to highlight how Crewhu responded to the vulnerability.
Thankfully, the Crewhu team was attending this same conference. We notified their team, and they responded quickly, obscuring and redacting the sensitive information with asterisks within two hours of being notified.
We met with members of the Crewhu team, which included Stephen Spiegel, CEO; Oscar Eduardo, CTO and Ryan Denning, VP of Sales, and we mentioned the importance of transparency for the betterment of the community. We mentioned the industry standard of 90 days to communicate to the channel and breathed a sigh of relief when Stephen and his team said they wanted to act as quickly as possible.
Stephen immediately got to work on a communication plan with his team and then started an outreach campaign a week later to roll the APIs.
Stephen and his team felt the responsibility to not only solve and communicate the vulnerability but also to go a step further to protect their partners and clients from future attacks. To do this, they partnered with Atlantic Data Forensics.
Atlantic Data Forensics (ADF) performs application vulnerability assessments, which go deeper than usual checkbox risk assessments. Security researchers and practitioners actually go through environments with a fine-tooth comb to help protect against attacks as well as identify and remedy any vulnerabilities that may exist.
Ultimately, the redaction addressed our most pressing concern—the disclosure of sensitive data.
However, we do want to note some of our initial questions that we had immediately after we discovered this vulnerability, such as how long the information had been publicly available, if any bad actors had collected credentials and if the redaction would tip off a bad actor or enable them to leverage access to do more damage. We want to mention these questions for two main reasons: 1) to knowledge share with the cybersecurity community by detailing our thought process as these events unfolded in real-time, and 2) to reiterate the importance of a timely response from a vendor.
After we flagged this vulnerability, the Crewhu team responded immediately and transparently, and they were eager to collaborate with us to remedy the vulnerability and get the word out to keep folks safe. This response made all the difference during this scenario. It’s critical that all players work in concert during an event like this.
Neither Huntress nor Crewhu is aware of any customers adversely affected.
Why Is This Important?
Thus far, we have seen no indication of any breaches or security threats coming from this information disclosure. With that said, Huntress has not explored further to determine if there are other information disclosures and leaves that to the security experts at Atlantic Data Forensics that Crewhu has coordinated with.
Out of an abundance of caution, we recommend - as does Crewhu - that you update your API keys used to communicate with Crewhu.
Just last week, we issued a threat advisory after discovering a critical vulnerability in a popular time and billing system called BillQuick—and unfortunately in this particular instance, the damage had been done before the vulnerability could be patched.
While we're grateful the Crewhu team was able to remedy this particular vulnerability before hackers had a chance to abuse it, the BillQuick scenario represents just one of several large incidents over the past year or so that has resulted in an MSP or IT service provider being compromised.
With that said, we offer a gentle reminder that no organization is immune to vulnerabilities and software flaws. Each and every business, company, provider or solution—large or small—can have their own “oops” moment.
We often say “a rising tide raises all ships,” but what we don’t often clarify is that we mean you. You can be the rising tide, or the change and betterment of the whole community.
If you discover a vulnerability or any semblance of a security risk, follow responsible disclosure procedures and notify the vendor as quickly as you can—and then share info with the community to help increase knowledge and awareness. We cannot do this alone.
What You Should Do
If you are a security practitioner, you should be on the lookout for bugs in products and solutions and notify the vendor accordingly. If you are an MSP, it might be worth doing a quick recycle on your passwords/private keys for your PSA and integration server for all PSA integrations on a periodic basis. If you are a vendor, please audit your code and review what is user-facing.
Too often we stumble across the low-hanging fruit—whether it is sensitive information visible with a “right-click, view source,” or just looking at the HTTP responses, or entering a simple apostrophe and watching a database break. All of these things are very easily tested, and they need to be tested internally—before the world finds these flaws for you.
Parting Words
Another gentle reminder: there is no shame in this. Everyone has these “oops” moments or accidental incidents. How a vendor reacts and responds is what matters, and we applaud Crewhu for their swift remediation, their transparency and their understanding and openness to share this publicly.
Finding a security issue like this opens everyone’s eyes and reminds them that there might be more stones left unturned. Huntress is continuing to work with the Crewhu team and we will share with you any further findings as we improve the security of the channel together.
