icpc

How Does Feature Dependency Affect Configurable System Comprehensibility?

icse

Author : Djan Santos and Cláudio Sant'Anna

Abstract :


Background: Conditional compilation is often used to implement variability in configurable systems. This technique relies on #ifdefs to delimit feature code. Previous studies have shown that #ifdefs may hinder code comprehensibility. However, they did not explicitly took feature dependencies into account. Feature dependency occurs when different features refer to the same program element, such as a variable. Comprehensibility may be even more affected in the presence of feature dependency, as the developer must reason about different scenarios affecting the same variable. Aim: Our goal is to understand how feature dependency affects the comprehensibility of configurable system source code. Method: We carried out an experiment in which 30 developers debugged programs with different types of feature dependency. We record the time each of them spent to find a bug. Also, we used an eye-tracking device to record developers' gaze movements while they debugged programs. Results: It took longer for developers to debug programs with global and interprocedural dependency. In addition, debugging programs with these same types of dependencies required higher visual effort. Conclusion: Our study showed that #ifdefs affect comprehensibility in different degrees according with the type of feature dependencies. Therefore, when possible, developers should prefer to use intraprocedural dependency and take more care when dealing with code with global and interprocedural dependency.
 
Procedure:

1 - training on variability, features and configurations;
2 - warm-up task;
3 - presentation about the goal of the experiment and the risk of participation;
4 - sign the consent forms;
5 - calibrate eye-tracking device;
6 - perform the task in order based in Latin square group;

 
Source codes of our experiment:

Latin Square 6 x6: Row = Groups, Column = Variability Bugs
Characteristic = acronyms.

Variability Bugs

Null pointer dereference
Assertion error
Logic error
Nested feature
Undefined variable
Uninitialized variable
Group 1 GI GW IAI IAW IEI IEW
Group 2 GW IAI IAW IEI IEW GI
Group 3 IAI IAW IEI IEW GI GW
Group 4 IAW IEI IEW GI GW IAI
Group 5 IEI IEW GI GW IAI IAW
Group 6 IEW GI GW IAI IAW IEI
Tasks: find the bug in source codes with different Caracteristics: with #IFDEF without #IFDEF
GI - Global dependency GW - Global dependency equivalent
IAI - Intraprocedural dependency IAW - Intraprocedural ependency equivalent
IEI - Interprocedural dependency IEW - Interprocedural dependency equivalent

Quantitative collected data

Script R

Attention map and gaze transitions diagram

Time to find bugs
Time.R Attention map - Global Dependency
Correctly found bugs
Correctly.R Attention map - Intraprocedural Dependency
Number of fixations
Fixations.R Attention map - Interprocedural Dependency
 

Gaze transitions:

Gaze transitions - Null pointer dereference Gaze transitions - Nested feature
  Gaze transitions - Uninitialized variable  
 

Variability Bugs

AOIs

Attention maps

Null pointer dereference AOIs - Global dependency Attention map - Global Dependency
AOIs - Intraprocedural dependency Attention map - Intraprocedural Dependency
AOIs - Interprocedural dependency Attention map - Interprocedural Dependency
Assertion error AOIs - Global dependency Attention map - Global Dependency
AOIs - Intraprocedural dependency Attention map - Intraprocedural Dependency
AOIs - Interprocedural dependency Attention map - Interprocedural Dependency
Logic error AOIs - Global dependency Attention map - Global Dependency
AOIs - Intraprocedural dependency Attention map - Intraprocedural Dependency
AOIs - Interprocedural dependency Attention map - Interprocedural Dependency
Nested feature AOIs - Global dependency Attention map - Global Dependency
AOIs - Intraprocedural dependency Attention map - Intraprocedural Dependency
AOIs - Interprocedural dependency Attention map - Interprocedural Dependency
Undefined variable AOIs - Global dependency Attention map - Global Dependency
AOIs - Intraprocedural dependency Attention map - Intraprocedural Dependency
AOIs - Interprocedural dependency Attention map - Interprocedural Dependency
Uninitialized variable AOIs - Global dependency Attention map - Global Dependency
AOIs - Intraprocedural dependency Attention map - Intraprocedural Dependency
AOIs - Interprocedural dependency Attention map - Interprocedural Dependency