When developers disclose information

In Norway we say "Det brenner på dass", can be translated to "It's burning on the toilet". This is our way of saying that "shit has already hit the fan".
Private information, that is private keys, passwords and tokens, or other type of sensitive information can end up getting spread throughout your infrastructure, or worse, on the internet. This is a case of Det brenner på dass. A developer committed a password or token to the repository, or forgot to remove a logging sentence. Your application is now leaking critical information.
What do you do when disaster strikes? The only prevention is to have routines and an infrastructure that lets you detect these forms of breaches before anyone else.

Ok, so a password was committed to Git, what do you do now? Remove the line of code that had the password, push to the repository, grab a cup of coffee and pretend it did not happen?
This is not the way to do it. Git will store the password, and if not correctly removed, an attacker could find it by sifting through the Git-logs. There are two options, git filter-branch, that lets you rewrite Git revision, all the way down to the possibility of modifying each tree and each commit. The problem with this is that rewriting Git-history for a large project is time consuming, and the pitfalls are many.
The second option is BFG Repo-Cleaner. Those of you that has played Doom knows what BFG stands for, and it is exactly what this tool is, but it's simpler and faster to use for removing passwords, credentials or other private data from Git.

Removing sensitive information in the logs is often as simple as to purge the logs, and remove the code that is logging the information. But you will also need to check that information has not been backed up, or logged by another system monitoring the logs. Once this information spreads, it is almost impossible to remove it. Your best option is to revoke the password/token/private key, and this leads us to the next topic.

You, or someone in your organization is bound to mess up, it will happen, and there is nothing you can do that will prevent it. What you can prevent is the consequences of it. What you should do, is to run automated scans that look for passwords, tokens and private keys in your infrastructure. And have implemented good routines for revocation and rotation of this information.

We are aware that some of this information is hard to look for. Find unique markers that make it easy to detect, or design your access information in a way that lets you search for it. We want to mention Docker Swarm's tokens, that are designed with a prefix with the intent of it being easy to search for.

Closely related to our post on containers and orchestrators, this talk that explains the tokens, and security in Docker Swarm is recommended.

Photo credit:
Blogtrepreneur Computer Data Hacker by Blogtrepreneur, under CC 2.0
Docker Inc. Secure Substrate: Least Privilege Container Deployment - Diogo Monica