GitHub’s Model for Secret Detection: Finding More Secrets with Less Noise
GitHub improves secret detection, reducing false positives and execution time.
Intelligence analysis by Qwen 2.5 (3B)
GitHub enhances its secret detection model, improving accuracy and efficiency, especially for post-push scanning.
GitHub made its secret detector better so it can find real secrets faster and with fewer mistakes, helping developers and security teams catch bad passwords and other secrets without getting too many false alarms.
Analysis
Background
GitHub's secret detection model has been improved to better distinguish real secrets from lookalikes, reducing false positives and execution time.
How the Model Works
The new model uses a two-stage pipeline: candidate generation and context-aware classification. Regex patterns identify potential secrets, and a fine-tuned ModernBERT classifier evaluates each candidate and its context to determine if it's a real secret.
Impact
The new model has led to a 11.5% increase in user acceptance, 86% faster post-push secret detection execution time, and over 98% cost savings compared to the previous LLM-based pipeline.
Comparison with Previous Pipeline
The new model is more efficient and cost-effective, with a median post-push scan execution time reduced by 86% compared to the previous pipeline.
Challenges and Solutions
Detecting real secrets is challenging due to the overlap between generic secrets and other strings. The new model addresses this by using context signals, such as file path, secret type, variable names, and nearby code or text, to distinguish between real secrets and lookalikes.
Key points
- GitHub improved its secret detection model
- Reduces false positives and execution time
- More than 98% cost savings compared to previous pipeline
The new model will allow GitHub to scale secret detection with the fast development speeds, reducing the risk of secrets slipping into repositories and improving the security of software.
If the new model fails to accurately distinguish between real secrets and lookalikes, it could lead to false positives and missed real secrets, compromising the security of software.
