What is secret scanning

7min
|
HCP
Vault
Video

HCP Vault Radar automates the detection and identification of unmanaged secrets in your code so that teams can take appropriate actions to remediate issues.

Leaked or exposed secrets can lead to unauthorized access to systems, and other critical data.

Secrets scanning is a process that allows you to find and identify secrets and other sensitive data hidden in source code or other locations such as documentation. With the correct tools, secret scanning functionality helps deliver secure code without compromising speed or innovation.

Having the ability to scan for secrets and other sensitive data will help protect your customers, limit the potential for breaches due to leaked credentials, as well as the company's reputation as one that prioritizes security.

HCP Vault Radar provides a Software-as-a-Service (SaaS) solution for scanning source code for secrets and sensitive data. Radar scans for the following types of information:

Secrets
Personally identifiable information (PII)
Non-inclusive language (NIL)

Once the scanning completes, the detected risks are displayed by categories and risks.

In this series of tutorials, you will learn about HCP Vault Radar through the lens of HashiCups as their engineering teams attempts to remove sensitive data in their source code.

Scenario introduction

HashiCups produces and sells its coffee cups at both retail locations and through its online store. They support both a web application and mobile application. The team at HashiCups is concerned about leaking secrets such as usernames and passwords, and API keys in their source code.

The CTO and CISO have presented the following business and technical requirements to the engineering teams:

All source code must be free of sensitive data
Any time sensitive data is detected, teams must be notified
Scans for leaked secrets must occur at multiple stages of the software development life cycle
Any potential solution can not store HashiCups owned source code

The team has several groups who will collaborate on the review of, and implementation of the selected solution(s).

Click on each tab to learn more about the teams and their responsibilities.

Alice leads the engineering architect team. The architect team is tasked with:

Understand system, resource, and connectivity requirements for all users and applications.
Identify supported services within the solution that other users, and systems will use to authenticate.
Compare and contrast features and functions available in any proposed solution.
Design implementation process, including support for high availability, disaster recovery, observability, and support runbooks.
Create as-built documentation to hand off to other teams.

Oliver manages the operations team. The operations team ensures:

All systems are implemented according to the architect team's design.
Performs system and application deployments, updates, and upgrades.
Ensures source code, systems, and applications have appropriate security scanning in place.
Tunes systems and applications to ensure adequate performance.
Provides support to other teams when accessing applications and system.
Manages access to applications and systems.

Danielle is the lead engineer and manager for the development. The development team:

Develops HashiCups internal and customer facing applications.
Works with the architecture team to identify new platform services the team can integrate into its applications such as single-sign-on.
Provides support for incidents that may require temporary access to systems.
Implement vendor SDKs to support integrating third party applications.

HashiCups has brought in HashiCorp to see how they can help achieve the goals set by the CISO and CTO.

HCP Vault Radar concepts

Before diving in to how HCP Vault Radar works, there are several key concepts that the teams at HashiCups would like to understand.

Data sources

Danielle, who leads the development team, has asked how and when their source code will be scanned.

Through the HCP Portal you can connect HCP Vault Radar to GitHub, GitLab, Bitbucket, and Azure DevOps.

Both cloud-based and on-premises data sources are supported. Public data sources connect to the HCP cloud scanner. On-premises data sources can be scanned using the Radar agent, or be publicly exposed to the cloud scanner.

The HCP Vault Radar CLI supports additional data sources such as local system files and directories, Docker, Amazon S3, and Terraform Enterprise.

Danielle has asked how they can achieve one of the goals set by the CISO and CTO of being able to scan for sensitive data throughout the SDLC.

HCP Vault Radar can also scan for sensitive data throughout the SDLC, such as using precommit hooks during local development, when branches are pushed to its source code repository, or when pull requests are opened.

Types of sensitive data

Steve from the SRE team would like to understand what types of sensitive data can be scanned. Is it just passwords and keys?

HCP Vault Radar can natively scan for multiple formats of sensitive data, including:

Secrets such as usernames, passwords, and keys.
Personally identifiable information (PII) such as social security, or credit card numbers.
Non-inclusive language(NIL) such as race or gender attributes.

Beyond the supported patterns that HCP Vault Radar can scan for, HashiCups can also create their own custom regular expressions (regex) to scan for sensitive data that may be specific to HashiCups such as product model numbers or financial information.

Integrations

Oliver points out that scanning for sensitive data is only one of the requirements. They would like to know how the operations and SecOps teams can be notified and triage alerts.

HCP Vault Radar supports the alert and triage requirements set by the HashiCups CISO and CTO.

HashiCups can configure alerts for sensitive data found by HCP Vault Radar using native integrations for PagerDuty, Slack, and Splunk.

You can configure multiple alert integrations to match your existing processes. For example, you can enable the Microsoft Teams or Slack integration for real time notifications, and also enable the PagerDuty integration to follow your defined escalations until the alert is resolved.

HashiCups can also use the ticketing integrations to open a ticket in Jira or ServiceNow, allowing the incident to be tracked through to the incident's conclusion.

How HCP Vault Radar works

The team thus far is excited about the possibilities of HCP Vault Radar, however Alice from the architecture team would like more detail on what happens with HashiCups source code when secrets are detected.

The first step to set up HCP Vault Radar is to connect a supported source code management (SCM) system. Once set up, the HCP Vault Radar scanning engine reviews the selected repositories, including available branches for sensitive data.

No source code or sensitive data is sent back to HCP Vault Radar. Instead, a two-phase hash or peppering is performed so HCP Vault Radar can identify if the sensitive data exists in multiple locations. This hash is then tokenized and returns a universally unique identifier (UUID) that is stored in the HashiCorp Cloud Platform.

The generated UUID, the commit hash, and the line number where the sensitive data was found are available in the HCP Portal.

HashiCorp demos the set up for HashiCups.

Next steps

In the next tutorial, the engineering teams at HashiCups will work together to implement a proof-of-concept deployment of HCP Vault Radar.

The POC will demonstrate:

Scan the origination's GitHub repositories to detect leaked and unmanaged secrets
Integrate with PagerDuty to receive security incidents from Vault Radar
Set up a ticket automation using Jira to triage and track incidents

Collection Overview

HCP Vault Radar

Scan a repository for secrets