Atoms of confusion do really cause confusion? A controlled experiment using eye tracking

Abstract

Code comprehension is crucial in software maintenance activities, though it can be hindered by misunderstandings and confusion patterns, namely, atoms of confusion. They are small pieces of code using specific programming language constructs, such as Conditional Opera-tors and Comma Operators. A previous study showed that these atoms of confusion impact developers’ performance, i.e., time and accuracy, and increase code misunderstandings. However, empirical knowledge of the impact of such atoms on code comprehension is still scarce, especially when it comes to analyzing that impact on developers’ visual attention. The present study evaluates whether developers misunderstand the code in the presence of atoms of confusion with an eye tracker. For this purpose, we measure time, accuracy, and analyze the distribution of visual attention. We conduct a controlled experiment with 30 students and software practitioners. We ask the subjects to specify the output of three tasks with atoms and three without atoms randomly assigned using a Latin Square design. We use an eye-tracking camera to detect the visual attention of the participants while solving the tasks. From an aggregated perspective, we observed an increase by 43.02% in time and 36.8% in gaze transitions in code snippets with atoms. Also, we observed na increase of 163.06% in the number of regressions when the atom is present. For accuracy, no statistically significant difference was observed. We also confirm that the regions that receive most of the eye attention were the regions with atoms. Our findings reinforce that atoms hinder developers’ performance and comprehension. So, developers should avoid writing code with them.