Guide: How to Start with Secrets Detection

When it comes to secret detection, remember, it's not a secret if everyone knows it-especially the hackers. Accidentally pushing a password or connection string into remote source code repository can happen even to the most skilled developers. This guide describes how to use two of them quickly and easily.

29 Oct 2024 Matěj Smyčka DevOps

Secret Detection utilities can be automated with CI/CD pipelines, GitHub actions, and other automation tools. For more information on this topic, read our guide on automation of secrets detection.

This guide explores the basics of secret detection with Gitleaks and TrufleHog.

All the tools mentioned below are open source and free.

Gitleaks

"Gitleaks is a SAST tool for detecting and preventing hardcoded secrets like passwords, api keys, and tokens in git repos. Gitleaks is an easy-to-use, all-in-one solution for detecting secrets, both past and present, in your code."

By default, this tool checks all commits in the repository, which is very helpful.

How to:

Install Docker
Run the following command on the current directory to fetch Gitleaks from the remote registry.

docker run -v $(pwd):/tmp/out \
zricethezav/gitleaks:latest detect \
--source="/tmp/out" -v --no-color  > result.txt

Output will be in result.txt file.

TruffleHog

Similiar to Gitleaks, TruffleHog is a tool for finding secrets in source code based on entropy and regular expressions.

By default, it checks only the current branch. One of its notable features is, -only-verified, which returns only true positive findings. However, it will check secrets against real service. In the context of university services,it may not be as helpful since we have many custom domains that are not defined in TruffleHog. Nevertheless, it is useful for checking secrets from public services like AWS, Azure, and others.

How to:

Install Docker
Run the following command on the current directory (sed command escapes ANSI colors):

docker run --rm -it -v "$(pwd):/out" \
trufflesecurity/trufflehog:latest git file:///out |
sed -r 's/\x1b_[^\x1b]*\x1b[\]//g; s/\x1B\[[^m]*m//g' > result.txt

You may run into the error fatal: detected dubious ownership ... It is only a warning, and you can ignore it. It is explained here.

Try it yourself!

There is an infinite tradeoff between precision and variance. Suppose you need more variance and don't mind more manual reviewing. In that case, you can try RegFinder, which is like grep but more suited for secret detection (faster in bigger repos, more clear output). Alternatively, you can use grep directly. Most valuable are the regexes contained in the repo, not the tool you will use.

Clone this repository
Run grep -n -r your_app/ -Ef regex_dir/general.txt

Run ./regfinder.elf -d your_app/ -f regex_dir/general.txt

It is straightforward to extend existing regex patterns. This tool is not feasible for automated pipelines. However, it comes in handy if you need to find a non-standard secret or in other assessments, such as security reviews, where more manual work is expected.

Contact

If you want to know more or need help with deployment, feel free to contact us at csirt@muni.cz.