"I'm trying to learn…and I'm shooting myself in the foot": Beginners' Struggles When Solving Binary Exploitation Exercises

Vulnerability discovery is an essential security skill that is often daunting for beginners. Although there are various supportive organizations and ample online resources to learn from, beginners often struggle, become frustrated, and quit. We conducted semi-structured observational interviews with 37 vulnerability discovery beginners attempting to exploit 51 vulnerable programs. We capture the questions beginners have when trying to identify and exploit vulnerabilities, how they search for answers, and the challenges they face applying their searches' results. We performed a rigorous qualitative coding of our dataset of 3950 events characterizing participants' actions to identify several behaviors and obstacles faced, along with quantitative measures to determine their most frequent issues.

We found beginners struggle to understand how to exploit vulnerabilities, craft their solutions, and even complete common technical tasks. They were often unable to find relevant information online to overcome these struggles, as they lacked the relevant vocabulary to craft effective keyword searches. When they did find relevant web pages, they struggled to properly transfer information from the web to their challenges due to misunderstandings and missing context. Based on our results, we offer suggestions for vulnerability discovery educators and resource creators to produce higher-quality materials to help facilitate beginner learning.