André · Zenodo (CERN European Organization for Nuclear Research) · 2021
This is the dataset for the paper "What Code Is Deliberately Excluded from Test Coverage and Why?", IEEE/ACM International Conference on Mining Software Repositories (MSR), 2021. This is the dataset for the paper "Excluding Code from Test Coverage: Practices, Motivations, and Impact", Empirical Software Engineering, 2021.
Cited by 0Open access
Melissa Finster (20513440), Maxim Moinat (20513443), Elham Taghizadeh (20513446) · Figshare · 2025
<p>The Table comprises all OMOP CDM domains including local concepts and ICD-10-GM to standard mapping. Example data 1 contains some fictional ICD-10-GM codes. A PZN mapping to map codes from domain Drug to standard concepts is lacking.</p>
Cited by 0Open access
Filip Gurbáľ, Jaroslav Porubän · IPSI Transactions on Internet Research · 2023
When testers design test cases based only on specification, they can introduce significant redundancies into them and also create gaps of untested software. More insight into the impact of test cases on implementation can guide testers to design more effective test cases to cover implementation and prevent redundancies to save time. In this paper we describe challenges of UI tester and how we can help them using coverage analysis of UI source code. We prepared infrastructure to automatically collect code coverage data from UI testing for further analysis.
Cited by 1
Ferenc Horváth · 2023
A kódlefedettség mérés fontos szerepet játszik az ipari gyakorlatban és a tudományos kutatásokban is. Számos terület függ a kódlefedettségtől, például a teszt esetek generálása, a tesztek priorizálása és a hibalokalizáció is. Ezek közül az értekezés két fő témára fókuszál, és a tézispontok ennek megfelelően két részre oszlanak. Az értekezés első részének 3. fejezete a Java kódlefedettség mérésére szolgáló eszközök (forráskód- és bájtkód alapú) által adott lefedettségi eredmények összehasonlításáról számol be. A kísérleteket 5 népszerű eszközzel, 8 nyílt forráskódú programon és metódus szinten végeztük. Megállapítottuk, hogy a lefedettségibeli különbségek 1,5% és 40% közöttiek és mindkét irányban fellépnek. A különbségeket részletesen is vizsgáltunk az egyes teszt esetek és programelemek szemszögéből. A kódlefedettség számos alkalmazásánál (például a hibakeresésnél) az ilyen szintű finom különbségek komoly zavart okozhatnak. Az egyes tesztesetek között akár 14%-os, a metódusok között pedig több mint 20%-os különbségeket mértünk. Szisztematikusan megvizsgáltuk a különbségek okait, és azt találtuk, hogy a különbségek egy része eszköz-specifikus, míg a többi a mérési megközelítésnek tulajdonítható. Az okok listája iránymutatásként szolgálhat a lefedettségi eszközök felhasználói számára, hogy miként kerüljék vagy hárítsák el a problémákat, ha bájtkód-alapú megközelítést alkalmaznak. Azt is megmértük, hogy a különbségek milyen hatással vannak a kódlefedettség-alapú teszt priorizálásra és azt találtuk, hogy a priorizált listák jelentősen eltértek egymástól. Az alacsony korreláció kockázatot jelent, mert megjósolhatatlan egy adott lefedettségi pontatlanság potenciális felerősödése egy adott alkalmazásban. Az automatikus hibakeresés eszköztámogatása korlátozott, mivel gyakran a legkorszerűbb algoritmusok sem nyújtanak hatékony segítséget a felhasználónak. Általában csupán a gyanús kódelemek rangsorolt listáját kínálják és a hiba jellemzően nem a rangsor elején található. A spektrumalapú hibalokalizáció (a kódlefedettséget és a tesztek eredményeit használja a rangsor kiszámításához) esetén a fejlesztőnek jellemzően több elemet is meg kell vizsgálnia, mielőtt megtalálja a valóban hibásat. Az értekezés második részének 4. fejezete egy olyan megközelítést javasol, amelyben a fejlesztő interakcióba léphet az algoritmussal azáltal, hogy visszajelzést ad a rangsorolt lista elemeiről. Ezzel kihasználjuk a felhasználó tudását a rangsorolt lista aktuális elemének környezetéről, így akár nagyobb kódegységek is átpozícionálhatóak. A megközelítést először szimulált felhasználókkal értékeltük ki, ahol kétféle “bizonytalanságot” modelleztünk: a tudást és magabiztosságot. Az eredmények még erős felhasználói bizonytalanság esetén is figyelemre méltó javulást mutattak a hibalokalizáció hatékonyságában. Ezután a megközelítés hatékonyságát valódi felhasználókkal is vizsgáltuk és az eredmények alapján ígéretesnek bizonyult a módszer. Spektrumalapú hibalokalizáció során a programelemeket egy gyanússági érték alapján rangsorolják, amely bár segítheti a programozót a hiba megtalálásában, de nem ad semmilyen információt a gyanús kódelemekről. Bár történtek kísérletek arra, hogy a folyamatba vezérlési- vagy adatfolyam alapú információkat építsenek be, ezek a skálázhatósági problémák miatt nem jártak sikerrel. Az értekezés második részének 5. fejezete a spektrumalapú hibalokalizáció kiegészítését javasolja függvényhívási láncokkal (azaz a hívási verem pillanatfelvételeivel), amelyeken először elvégezzük a hibalokalizációt, majd az eredményeket visszavezetjük a függvényekre. A valós programokkal végzett kísérletek azt mutatják, hogy a javasolt módszer hatékonysága jelentősen jobb a hagyományos függvényszintű megközelítéshez képest és a számítási többletköltség is kezelhető mértékű.
Cited by 0Open access
Sterk, Alexander (author) · Research Repository (Delft University of Technology) · 2023
Software development has increasingly become an activity that is (partially) done online on open-source platforms such as GitHub, and with it, so have the tools developers typically use. One such category of tools is that of code coverage tools. These tools track and report coverage data generated during CI tests. As the adoption of these tools has grown, so does the amount of available coverage data. In this thesis we explore a large database of coverage data from Codecov, a popular coverage tool. What sets our work apart from existing research is that it spans a large number of projects which vary in size, language, and domain. Furthermore, we conduct a survey, which was disseminated among a wide variety of open-source developers, instead of at a single company or in an enterprise setting. Our research consists of three parts. Firstly, we assess whether there is a relationship between the time to merge a PR and its coverage levels. We find that such a relationship does exist in certain projects. Secondly, we look at the impact of PR comments mentioning coverage on the odds of said coverage improving. Using the odds ratio test, we conclude that there are greater odds of coverage improving when it is mentioned than when it is not. Thirdly, we conduct a survey to ask developers their reasons for ignoring a failing status check related to code coverage. Some reasons they give are the complexity of testing, the triviality of the proposed changes, or the pull request being too important to wait for proper testing. Furthermore, respondents who identify as code contributors find themselves twice more likely to find fixing coverage a waste of their time than those who identify as code maintainers, while code maintainers are more concerned with not scaring away new contributors with strict coverage guidelines.
Cited by 0Open access
Jordan Samhi, Andreas Zeller · 2024
Dynamic analysis has emerged as a pivotal technique for testing Android apps, enabling the detection of bugs, malicious code, and vulnerabilities. A key metric in evaluating the efficacy of tools employed by both research and practitioner communities for this purpose is code coverage. Obtaining code coverage typically requires planting probes within apps to gather coverage data during runtime. Due to the general unavailability of source code to analysts, there is a necessity for instrumenting apps to insert these probes in black-box environments. However, the tools available for such instrumentation are limited in their reliability and require intrusive changes interfering with apps’ functionalities. This paper introduces AndroLog, a novel tool developed on top of the Soot framework, designed to provide fine-grained coverage information at multiple levels, including class, methods, statements, and Android components. In contrast to existing tools, AndroLog leaves the responsibility to test apps to analysts, and its motto is simplicity. As demonstrated in this paper, AndroLog can instrument up to 98% of recent Android apps compared to existing tools with 79% and 48% respectively for COSMO and ACVTool. AndroLog also stands out for its potential for future enhancements to increase granularity on demand. We make AndroLog available to the community and provide a video demonstration of AndroLog.
Cited by 3Open access
Samhi, Jordan, Zeller, Andreas · Figshare · 2024
Dynamic analysis has emerged as a pivotal technique for testing Android apps, enabling the detection of bugs, malicious code, and vulnerabilities. A key metric in evaluating the efficacy of tools employed by both research and practitioner communities for this purpose is code coverage. Obtaining code coverage typically requires planting probes within apps to gather coverage data during runtime. Due to the general unavailability of source code to analysts, there is a necessity for instrumenting apps to insert these probes in black-box environments. However, the tools available for such instrumentation are limited in their reliability and require intrusive changes interfering with apps' functionalities. This paper introduces AndroLog a novel tool developed on top of the Soot framework, designed to provide fine-grained coverage information at multiple levels, including class, methods, statements, and Android components. In contrast to existing tools, AndroLog leaves the responsibility to test apps to analysts, and its motto is simplicity. As demonstrated in this paper, AndroLog can instrument up to 98% of recent Android apps compared to existing tools with 79% and 48% respectively for COSMO and ACVTool. AndroLog also stands out for its potential for future enhancements to increase granularity on demand. We make AndroLog available to the community and provide a video demonstration of AndroLog (see section 8).
Cited by 0Open access
Romdhana, Andrea, Ceccato, Mariano, Georgiu, Gabriel Claudiu et al. · Zenodo (CERN European Organization for Nuclear Research) · 2021
The degree of code coverage reached by a test suite is an important indicator of the thoroughness of testing. Most coverage tools for Android apps work at the bytecode level and provide no information to developers about which source code lines have not yet been exercised by any test case. In this paper, we present COSMO, the first fully automated Android app instrumenter publicly available that operates at the source code level in a completely transparent way, making it fully compatible with existing system level testing technologies and Android test generators. The experiments that we have conducted on a large benchmark of Android apps show that COSMO can successfully instrument most apps without altering their execution traces, introducing a small, acceptable runtime overhead.
Cited by 0Open access
Komal Agarwal, Manish Goyal, Praveen Ranjan Srivastava · International Journal of Bio-Inspired Computation · 2012
Software testing is one of the most challenging and arduous phase of software development life cycle (SDLC) which helps in determining the software quality. Code coverage is a widely used testing paradigm, which describes the degree to which the code has been tested. Aim of the current paper is to propose an optimised code coverage algorithm with the help of an emerging technique, i.e., intelligent water drop (IWD). This approach uses dynamic parameters for finding all the optimal paths using basic properties of natural water drops. It proposes how test cases can be considered as an IWD moving on the edges of the control dependency graph for finding the optimal paths. The algorithm guarantees complete code coverage by generating automated test sequences.
Cited by 24
Xiaojun Ma · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · 2000
Cited by 3
Caroline Jay, Jorge Arturo Wong-Mozqueda, Robert Haines · Research Explorer (The University of Manchester) · 2015
A good test suite is vital for minimising errors, and ensuring that software is easy to maintain. Another factor viewed as being important for the success and longevity of software is code quality. We report on work examining whether there is a correlation between code quality and test coverage, using seven different metrics: lines of code, McCabe's cyclomatic complexity, number of local methods, depth of inheritance tree, coupling between objects, improvement of lack of cohesion in methods and lack of documentation. An analysis of three large, open source Java projects showed that all of the response variables had a modest but significant relationship with line coverage, and a stronger relationship with branch coverage: as coverage rose, so did software quality. We propose that writing tests may help people to adopt a 'software quality' mindset, by encouraging them to think about how code will be used as it is written. Testing may improve software sustainability not only by helping to ensure code does not regress, but also by supporting developers in adopting good software engineering practices.
Cited by 2Open access
Mechelle Gittens, Keri Romanufa, David Godwin et al. · 2006
The increasing size and complexity of software systems has led to increased challenges in evaluating code coverage. In parallel, development projects for large complex software are increasingly limited in schedule and resources. Common code coverage analysis provides a complete measurement of testing breadth, but does not give guidance for coverage in a project setting. We propose a prioritized coverage approach that is indigenous to the development organization and provides capabilities for evaluating code coverage and setting priorities for testing. This paper presents an approach to code coverage analysis for large software that considers modern project constraints.
Cited by 14
Shihe Zhang, Chaohao Liao, Renhua Liu · 2023
Fuzzing is a common method in software test engineering, which tries to trigger potential vulnerabilities in the target program by continuously providing modified input to the test target. The traditional fuzzy test tools are white box test, gray box test, black box test. The mutation strategies used in these traditional testing tools have high randomness. As a result, the traditional test methods have high randomness in the selection of strategic actions, so effective strategic actions maybe be ignored. Aiming at improving the coverage rate of fuzzy test code based on variation, we implement RLHavoc to improve the efficiency of American fuzzy lop(AFL) based on Deep Q Network(DQN) algorithm by using DQN to prioritize the generation of test cases in AFL. RLHavoc combines AFL’s Havoc phase mutation strategy with DQN algorithm. DQN algorithm takes advantage of AFL’s Havoc strategy to conduct grey box testing for different binaries and use comprehensive historical data to guide the action selection of variation strategies in the fuzzy test vulnerability detection process, taking seeds as state and new the number of unique execution paths as reward, which enhances the action space orientation of the secondary fuzz target binaries, thus improving the code coverage. Compared to the traditional AFL fuzz effect, our experimental results show that the code coverage rate is improved by about 3%.
Cited by 0
Sergio Walter Ferrero · 2004
Cited by 0Open access
Žiaková, Nikoleta · Brno University of Technology Digital Library (Brno University of Technology) · 2011
Tato práce se zabývá analýzou pokrytí kódu testovací sadou aplikačního serveru JBoss. Pro sběr informací o pokrytí kódu byly použity nástroje Emma a Cobertura. Bakalářská práce je rozdělená na dvě hlavní části. První z nich seznámí čtenáře s použitými technologiemi a připraví teoretický základ pro druhou část. Ta má praktický charakter, obsahuje postup pro získání informací o pokrytí kódu vybranými nástroji a jejich následnou analýzu. Výsledky poskytují pohled na vývoj pokrytí kódu v průběhu vývoje aplikačního serveru a také porovnání použitých nástrojů.
Cited by 0
Samhi, Jordan, Zeller, Andreas · CISPA Helmholtz Center · 2024
Dynamic analysis has emerged as a pivotal technique for testing Android apps, enabling the detection of bugs, malicious code, and vulnerabilities. A key metric in evaluating the efficacy of tools employed by both research and practitioner communities for this purpose is code coverage. Obtaining code coverage typically requires planting probes within apps to gather coverage data during runtime. Due to the general unavailability of source code to analysts, there is a necessity for instrumenting apps to insert these probes in black-box environments. However, the tools available for such instrumentation are limited in their reliability and require intrusive changes interfering with apps' functionalities. This paper introduces AndroLog a novel tool developed on top of the Soot framework, designed to provide fine-grained coverage information at multiple levels, including class, methods, statements, and Android components. In contrast to existing tools, AndroLog leaves the responsibility to test apps to analysts, and its motto is simplicity. As demonstrated in this paper, AndroLog can instrument up to 98% of recent Android apps compared to existing tools with 79% and 48% respectively for COSMO and ACVTool. AndroLog also stands out for its potential for future enhancements to increase granularity on demand. We make AndroLog available to the community and provide a video demonstration of AndroLog (see section 8).
Cited by 0Open access
Journal of International Crisis and Risk Communication Research · 2004
A superwide-angle coverage code-multiplexed optical scanner is presented that has the potential to provide 4pi-sr coverage. As a proof-of-concept experiment, an angular scan range of 288degrees for six randomly distributed beams is demonstrated. The proposed scanner achieves its superwide coverage by exploiting a combination of phase-encoded transmission and reflection holography within an in-line hologram recording-retrieval geometry. The basic scanner unit consists of one phase-only digital mode spatial light modulator for code entry (i.e., beam scan control) and a holographic material from which we obtained what we believe is the first-of-a-kind extremely wide coverage, low component count, high speed (e.g., microsecond domain), and large aperture (e.g., > 1-cm diameter) scanner. (C) 2004 Optical Society of America.
Cited by 0Open access
Perez, Alexandre Campos · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · 2012
Tese de Mestrado Integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 2012
Cited by 0Open access
Sangharatna Godboley, Monika Rani Golla, P. Radha Krishna · 2023
White-box testing typically involves structural code coverage criteria that measure the extent to which a program's source code has been tested. To efficiently generate test cases while achieving high code coverage, it is important to consider the Short-Circuit evaluation, a feature present in high-level programming languages. In this paper, we propose a new coverage technique called Short-Circuit Multiple Condition Coverage (SC-MCC) and compare it with the state-of-the-art Modified Condition/Decision Coverage (MC/DC). We analyze the test suite size and the time required for the final test suite generation by varying the number of clauses in the conditions. Our experimental results demonstrate that SC-MCC provides sufficient confidence and proves to be more efficient than MC/DC for higher-level languages.
Cited by 0
Krystof Sykora · Cvut DSpace (Czech Technical University) · 2018
Cílem této práce je vytvoření metody pro generování testovacích scénářů a vyhodnocení jejich effektivity. Tato metoda je vytvořená na základě analýzy konkrétního programu, která je založená na určení dopadů, jaký mají jednotlivé parametry programu na množstvý provedeného kódu. K dosažení tohoto cíle byla vytvořena aplikace, která analyzuje pokrití kódu testovaného programu na základě různých permutací jeho argumentů, generuje testovací scénáře a určuje jijich effektivitu ve hledání vložených chyb. Tento dokument dokumentje vytvořený software, použité metody a výsledky experimentů. Programy touto cestou testované jsou Flex, Grep a Gzip.
Cited by 0Open access