Evaluating Feature Steering
Anthropic evaluates feature steering for model outputs, finding mixed results on bias reduction and capability impact.
Intelligence analysis by Qwen 2.5 (3B)
Anthropic researchers assess feature steering for model outputs, revealing mixed results on bias reduction and capability impact.
Anthropic researchers looked at how changing certain features in their AI model affected its output. They found that sometimes changing these features helped reduce bias, but it could also make the model less useful overall.
Analysis
Feature Steering Sweet Spot
We identified a feature steering 'sweet spot' where feature steering does not significantly impact model capabilities. This 'sweet spot' is shared across all 29 features tested.
Targeted Effects
Feature steering can influence model evaluations in targeted domains. For example, increasing the value of a feature that fires on discussions of gender bias increases the gender identity bias score.
Off-Target Effects
We found some evidence that we can't always predict a feature's effects just by looking at the contexts in which it fires. Features we think might be related to gender bias may also significantly affect age bias, a general trend we refer to as off-target effects.
Neutrality Feature
We identified a neutrality feature that significantly decreases social biases on nine social dimensions without necessarily impacting capabilities tested too much.
Key points
- Identified a feature steering 'sweet spot' where feature steering does not significantly impact model capabilities.
- Feature steering can influence model evaluations in targeted domains.
- We found some evidence of off-target effects, where features related to one bias could affect others.
Feature steering can help reduce bias in AI models, and we can learn more about how it works by sharing our findings.
Feature steering might come at the cost of reducing a model's overall usefulness, so more research is needed to understand its full impact.
