One Strategy Does Not Fit All: Rethinking Data Labeling for Just-in-Time Vulnerability Prediction
Story 16, from arXiv
A study shows that different data labeling strategies for vulnerability prediction models significantly impact their performance on classification, prioritization, and localization tasks.
Key points
- Multiple labeling strategies for identifying vulnerability-fixing and introducing commits lead to different datasets.
- Manual VFC labeling with vulnerability-specific VIC labeling is best for classification and prioritization.
- Precision-first VFC labeling with LLM-assisted VIC labeling is best for localization.
- No single labeling strategy is universally optimal; task orientation is key.
- The study evaluates six JIT-VP models across OpenSSL and Linux kernel datasets.
Why it matters
Software engineers using AI for security analysis should be aware that the quality of their models depends heavily on how data is labeled. Choosing the right labeling strategy can significantly improve the effectiveness of vulnerability prediction tools.
Counterpoint
The study focuses on specific datasets and models, so the findings may not generalize to all software ecosystems or AI tools.
Sources
- One Strategy Does Not Fit All: Rethinking Data Labeling for Just-in-Time Vulnerability Prediction (arXiv)
This summary was written by a language model and may contain errors. Read the original for the full story.