azuWe have released Secretlint v6, which finds API tokens, passwords, and other sensitive information contained in files.
Secretlint, when executed with a glob or file, such as secretlint "**/*", finds confidential information contained in the file and prints it to standard output.
On the other hand, unlike ESLint and textlint, Secretlint cannot be automatically modified with --fix.
This is because finding and deleting API tokens, etc. will simply break them.
Therefore, any sensitive information found must be reported and manually fixed by the user.
For example, modify it so that it is not hard-coded into the source code, but rather received in an environment variable, etc.
In most cases, you can't automatically fix any confidential information you find, but I've noticed that it's OK to automatically remove or mask any confidential information that has been left in .bash_history or .zsh_history.
To automatically modify API tokens left in history files, Secretlint v6 adds a --format=mask-result formatter .
Usage: masking confidential information in .zsh_history.
The secretlint package itself does not contain rules, but the @secretlint/quick-start package ships with recommended rules, so here is how to do it using @secretlint/quick-start.
- Requirements: Node.js 16+
You can check if .zsh_history contains sensitive information with the following command.
(You can also check .bash_history by simply changing the file name)
npx @secretlint/quick-start ~/.zsh_history
~/.zsh_history
9178:25 error [GITHUB_TOKEN] found GitHub Token(GitHub personal access tokens): ghp_wWPw5k4aXcaT4fNP0UcnZwJUVFk6LO0pTEST @secretlint/secretlint-rule-preset-recommend > @secretlint/secretlint-rule-github
✖ 1 problem (1 error, 0 warnings)
If included, an error will be reported.
Before masking this sensitive information, back up .zsh_history just in case.
cp ~/.zsh_history ~/.zsh_history.bak
You can use --format=mask-result to see the results of masking sensitive information in ~/.zsh_history on standard output.
npx @secretlint/quick-start ~/.zsh_history --format=mask-result
...
: 1672748457:0;GITHUB_TOKEN="****************************************" gh issue list
...
If all is well, use the --output option to overwrite ~/.zsh_history with the masked results.
(Do not overwrite with <command> > ~/.zsh_history)
npx @secretlint/quick-start ~/.zsh_history --format=mask-result --output ~/.zsh_history
Finally, delete the ~/.zsh_history.bak that you backed up.
rm ~/.zsh_history.bak
Motivation
Last month, there was a supply chain attack against PyTorch's PyTorch-nightly using Dependency Confusion.
pip installs PyPI first (strictly speaking, index-url is preferred over extra-index-url) if the in-house repository (registry) and PyPI have the same package name.
Therefore, if you upload malicious code to PyPI with the same name as a package that PyTorch-nightly depends on but was not yet in PyPI, the malware will be automatically installed when you install PyTorch-nightly.
This problem is called Dependency Confusion.
The same problem exists with npm, etc.
When a private registry such as JFrog Artifactory is used in conjunction with a public registry, it is easy to have the same name but different contents in different registries. The following article summarizes how to deal with this Dependency Confusion, and talks about combining measures such as not mixing registries, using namespaces for package names, and using lock files.
The PyTorch-nightly malware seems to have included a process that uploads and steals files in $HOME/* in addition to environment variables, system information, /etc/hosts, /etc/passwd and .ssh/.
The .ssh/ directory can be managed with an SSH key, such as 1Password, so that it cannot be stolen with only the permission to read files.
Credentials are often stored as raw text in .config/ or ~/.aws.
These can be found in 1Password Shell Plugins, op run, zenv, envchain, etc. to avoid storing raw tokens in files.
Other things that can easily be credentialed in the HOME directory are shell history files such as .zsh_history and .bash_history.
I was not comfortable deleting all the history files, and on the other hand, it is troublesome to check them visually, so I thought it would be a good idea to use secretlint to check them automatically.
Although Secretlint does not have an automatic correction mechanism like --fix, I thought it would be possible to implement a pseudo automatic correction by devising an implementation of a formatter that displays error results I implemented a --format=mask-result formatter.
I actually found some issues of them when I tried it, so I masked them out and removed them from the history.
