Human Acceptance of Machine Generated Code Security Analysis

The current trend in software development calls for shorter release cycles, and several tools support developers in this process. Unfortunately, the very process that generates quick updates and increases business opportunities is also a source of security vulnerabilities. In fact, developers often struggle to deliver secure products due to a lack of resources, difficulty in prioritizing security, or knowledge gaps. To address the trade-off between business opportunities and security risks, qualitative studies with developers have shown that developers would appreciate automated tools to quickly and correctly patch security vulnerabilities.

In this thesis, we first address the challenge of providing an optimized design of experiments to bridge the gap in performing experimental studies in software engineering with a relatively small number of human subjects. We then considered program slicing as a technique to simplify the work of developers in identifying vulnerabilities, finding that slicing increases the likelihood of detecting a vulnerability but does not have a significant impact compared to reviewing the full source code.

We subsequently studied technologies based on mining software repositories (MSR) and machine learning (ML) for the identification of security-relevant commits, finding that simple, interpretable rule-based tools can perform on par with state-of-the-art ML techniques while providing clarity and confidence to developers. We also explored automated program repair (APR) tools as an alternative technique to patch vulnerabilities, observing that reviewers are skilled at discarding incorrect patches but less confident in recognizing correct ones, and that their trust is shaped by the perceived nature of the tool.

Finally, we studied human perception of AI-generated recommendations in Threat Analysis, finding that participants disagreed slightly more with AI-generated recommendations, yet judged human and AI sources as equally biased. In summary, we find that automation can support human judgment, but cannot completely replace it. Future automated techniques should take into account the role of the human as part of the loop in making the final decision.