Asgari, Ali, de Koning, Milan, Derakhshanfar, Pouria et al. · ArXiv.org · 2025
Large language models and deep learning models designed for code intelligence have revolutionized the software engineering field due to their ability to perform various code-related tasks. These models can process source code and software artifacts with high accuracy in tasks such as code completion, defect detection, and code summarization; therefore, they can potentially become an integral part of modern software engineering practices. Despite these capabilities, robustness remains a critical quality attribute for deep-code models as they may produce different results under varied and adversarial conditions (e.g., variable renaming). Metamorphic testing has become a widely used approach to evaluate models' robustness by applying semantic-preserving transformations to input programs and analyzing the stability of model outputs. While prior research has explored testing deep learning models, this systematic literature review focuses specifically on metamorphic testing for deep code models. By studying 45 primary papers, we analyze the transformations, techniques, and evaluation methods used to assess robustness. Our review summarizes the current landscape, identifying frequently evaluated models, programming tasks, datasets, target languages, and evaluation metrics, and highlights key challenges and future directions for advancing the field.
Cited by 0Open access
Sardar K. Jabrw, Qusay I. Sarhan · ARO-The Scientific Journal of Koya University · 2025
The rapid development of large language models (LLMs) has transformed code generation, offering powerful tools for automating software development tasks. However, evaluating generated code’s quality, security, and effectiveness remains a significant challenge. The present systematic survey comprehensively analyses studies published between 2021 and 2024, focusing on utilizing LLMs in the code generation process. The survey explored ten research questions, such as the most commonly used programming languages, the metrics employed to evaluate the quality of code, and scenarios in which LLMs are applied by developers during the software development process, outlining the scope in which prompt engineering influences code generation and security concerns with the types of benchmarks, models evaluated, and code analysis tools used in studies. The findings indicate that the most frequently used evaluation metrics in code generation are Pass@k and Bilingual Evaluation Understudy. It also shows that Python, Java, and C++ are the most widely used languages. Furthermore, identifying security vulnerabilities and establishing robust evaluation metrics remain challenges. This survey underlines present practices, detects gaps, and suggests future research to enhance the reliability and security of code generated by LLMs in real-world applications.
Cited by 1Open access
Kai Trepte, Sebastian Schwalbe · ChemRxiv · 2019
Accurate numerical calculations of porosities and related properties are of importance when analyzing metal-organic frameworks (MOFs). <br>We present porE, an open-source, general-purpose implementation to compute such properties and discuss <br>all results regarding their sensitivity to numerical parameters.<br>Our code combines the numerical efficiency of Fortran with the user-friendliness of Python.<br>Three different approaches to calculate porosities are implemented in porE, and<br>their advantages and drawbacks are discussed. In contrast to commonly used implementations, <br>our approaches are entirely deterministic and do not require any stochastic averaging.<br>In addition to the calculation of porosities, porE can calculate pore size distributions and offers the possibility to analyze pore windows. <br>The underlying approaches are outlined, and pore windows are discussed concerning their impact on the analyzed porosities. <br>Comparisons with reference values aim for a clear differentiation <br>between void and accessible porosities, which we provide for a small benchmark set consisting of 8 MOFs.<br>In addition, our approaches are used for a bigger benchmark set containing 370 MOFs, <br>where we determine linear relationships within our approaches as well as to reference values. <br>We show how these relationships can be used to derive corrections to a give porosity approach, <br>minimizing its mean error. As a highlight we show how complex workflows can be <br>designed with a few lines of Python code using porE.<br>
Cited by 0Open access
Lingyun Li, Shujuan Hou, Peiyao Yang · 2014
A novel distributed turbo code (DTC), called dual-systematic bits DTC (DS-DTC), is proposed for a two-hop relay system. The source broadcasts a punctured turbo code to both relay and destination. After iterative decoding the received codes, the relay forwards the systematic bits to the destination in case of successful cyclic redundancy check (CRC). And a modified turbo decoder based on double versions of systematic bits is designed at destination in order to make full use of the information coming from source and relay. The superiority of the proposed DS-DTC to conventional DTC and punctured DTC is revealed through simulations.
Cited by 0
Yan‐Haw Chen, Trieu‐Kien Truong, Chien‐Hsiang Huang et al. · Information Sciences · 2009
Cited by 21
Rachel Roberts, Kelsey Leonard Grabeel · Journal of the Medical Library Association JMLA · 2025
The reasons people requested health information from the library aligned with hospitalizations and ED visits in most of the zip codes. Providing further outreach to the hospital's targeted zip codes will benefit both the hospital and the library by increasing usage of the health information service.
Cited by 0Open access
Nawaf Alomari, Moussa Redah, Adnan Ashraf et al. · Information and Software Technology · 2025
Cited by 0
Kai Trepte, Sebastian Schwalbe · ChemRxiv · 2020
Accurate numerical calculations of porosities and related properties are of importance when analyzing metal-organic frameworks (MOFs). We present porE, an open-source, general-purpose implementation to compute such properties and discuss all results regarding their sensitivity to numerical parameters.Our code combines the numerical efficiency of Fortran with the user-friendliness of Python.Three different approaches to calculate porosities are implemented in porE, andtheir advantages and drawbacks are discussed. In contrast to commonly used implementations, our approaches are entirely deterministic and do not require any stochastic averaging.In addition to the calculation of porosities, porE can calculate pore size distributions and offers the possibility to analyze pore windows. The underlying approaches are outlined, and pore windows are discussed concerning their impact on the analyzed porosities. Comparisons with reference values aim for a clear differentiation between void and accessible porosities, which we provide for a small benchmark set consisting of 8 MOFs.In addition, our approaches are used for a bigger benchmark set containing 370 MOFs, where we determine linear relationships within our approaches as well as to reference values. We show how these relationships can be used to derive corrections to a give porosity approach, minimizing its mean error. As a highlight we show how complex workflows can be designed with a few lines of Python code using porE.
Cited by 1Open access
Nikola Luburić, Simona Prokić, Katarina-Glorija Grujić et al. · 2021
<div>Code smells are structures in code that indicate the presence of maintainability issues. A significant problem with code smells is their ambiguity. They are challenging to define, and software engineers have a different understanding of what a code smell is and which code suffers from code smells.</div><div>A solution to this problem could be an AI digital assistant that understands code smells and can detect (and perhaps resolve) them. However, it is challenging to develop such an assistant as there are few usable datasets of code smells on which to train and evaluate it. Furthermore, the existing datasets suffer from issues that mostly arise from an unsystematic approach used for their construction.</div><div>Through this work, we address this issue by developing a procedure for the systematic manual annotation of code smells. We use this procedure to build a dataset of code smells. During this process, we refine the procedure and identify recommendations and pitfalls for its use. The primary contribution is the proposed annotation model and procedure and the annotators’ experience report. The dataset and supporting tool are secondary contributions of our study. Notably, our dataset includes open-source projects written in the C# programming language, while almost all manually annotated datasets contain projects written in Java.</div>
Cited by 2Open access
Sonnekalb, Tim · elib (German Aerospace Center) · 2022
Cited by 0
Zixian Zhang, Takfarinas Saber · IEEE Access · 2025
Source code similarity measurement, which involves assessing the degree of difference between code segments, plays a crucial role in various aspects of the software development cycle. These include but are not limited to code quality assurance, code review processes, code plagiarism detection, security, and vulnerability analysis. Despite the increasing application of ML technique in this domain, a comprehensive synthesis of existing methodologies remains lacking. This paper presents a systematic review of Machine Learning techniques applied to code similarity measurement, aiming to illuminate current methodologies and contribute valuable insights to the research community. Following a rigorous systematic review protocol, we identified and analyzed 84 primary studies on a broad spectrum of dimensions covering application type, devised Machine Learning algorithms, used code representations, datasets, and performance metrics, as well as performance evaluations. A deep investigation reveals that 15 applications for code similarity measurement have utilized 51 different machine learning algorithms. Additionally, the most prevalent code representation is found to be the abstract syntax tree (AST). Furthermore, the most frequently employed dataset across various code similarity research applications is BigCloneBench. Through this comprehensive analysis, the paper not only synthesizes existing research but also identifies prevailing limitations and challenges, shedding light on potential avenues for future work.
Cited by 5Open access
Axel Panamá Velásquez, Christian Pérez-Salazar, Lizbeth A. Hernández-González et al. · 2024
With the growing demand for software, it is necessary to look for new approaches that allow faster de-liveries, without compromising the quality of the final product. Among these new approaches to development is low-code, which emerged as a proposal for applications development, minimizing the need for manual coding. Its application is associated with the use of low-code development platforms (LCDPs), a term most commonly used when discussing low-code in general. This work presents the results obtained from a Systematic Literature Review (SLR), which sought to understand the current state and future trends of the low-code approach to be considered by software engineers as an alternative to high-level programming languages. Applying knowledge of Software Engineering would take care of the quality of the process and the product. This would allow the decrease of time and costs in the construction of software. Fol-lowing Kitchenham's methodology, the characteristics, benefits, challenges and trends of low-code were identified. Although low-code has gained significant ground, there is a notable gap in comprehensive SLRs focused exclusively on this approach, as it is often discussed in conjunction with the no-code approach. After examining various databases such as IEEE Xplore, ACM Digital Library, ScienceDirect, and SpringerLink, fifteen rele-vant primary studies (published between 2018 and 2023) were selected. Among the findings, it was discovered that the characteristics of LCDPs include to use graphical interfaces, support for external services, collaborative development and reuse. The benefits encompass rapid development, cost reduction, ease of use and strong integration. However, challenges such as lack of customization and scalability are faced. Future trends include adoption by small and medium sized enterprises (SMEs), integration with DevOps, and digitalization efforts. Careful evaluation is recommended for effective implementation of low-code approach.
Cited by 2
Gene M. Alarcon, Tyler J. Ryan · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · 2018
The popularity of code reuse and the prospect of computer generated code raises questions of how programmers trust in computer code. Psychological understanding of computer code perceptions and comprehension has yet to be explored in regards to the decision making processes involved with software development and reuse practices. A review of current literature on trust, automation, software reuse, and the intersection of the three is presented. The authors propose a theoretical model of this decision making process, building off of a heuristic-systematic model of persuasion. Future research directions and possible applications are discussed.
Cited by 20Open access
Monika Jedynak · International Journal of Contemporary Management · 2018
Cooperation with suppliers has an important bearing on the organisation's ability to achieve its objectives. The supplier selection processes include various criteria which increasingly have also other than purely economic interest. Supplier code of conduct is one of tools for shaping relations between organisations and their suppliers which, applied mostly on international markets, expand dimensions of requirements suppliers should meet. The article analyses degree and scope of presence of supplier code of conduct in reviewed scientific publications.
Cited by 7Open access
X. Chen, Astrophysics Laboratory, Keble Road et al. · 1996
Neural network technique and Chi square fit method have been used to determine the major radioactivity background in SNO and got good results. Studies have been done to check the robustness of these techniques against the systematic errors of the SNOMAN code. Some results are presented here. 1
Cited by 0
Khalida Shaheen, Muhammad Shumail Naveed, Anwar Ali Sanjrani et al. · VFAST Transactions on Software Engineering · 2025
Programming is an essential skill, but it can be difficult for beginners, especially when it comes to logical concepts like recursion. Despite the development of many computational and pedagogical methods to simplify programming, recursion remains a challenging topic to understand, implement, and debug. Artificial intelligence has led to the development of large language models (LLMs), such as ChatGPT, Gemini, and DeepSeek that can generate programming source code. Various studies have analyzed the quality of code produced by LLM. However, the complexity of the recursive code generated by these models has not been studied. This study compared and analyzed recursive Python programs generated by Gemini (2.5 Pro), DeepSeek (V3.1) and ChatGPT (GPT-5) in an attempt to fill this gap. For the study, 250 programs generated by each model were examined using Halsted and cyclomatic complexity metrics. The results showed that ChatGPT produced less complex code, indicating easier recursion, while DeepSeek produced more complex programs due to higher Halstead and cyclomatic complexity scores. Gemini programs have a medium level of difficulty. The Kruskal-Wallis test was used to further analyze the data, and it revealed significant differences between the recursive code generated by ChatGPT, DeepSeek, and Gemini. Overall, the study found that each LLM has a distinct pattern: ChatGPT emphasizes simplicity, Gemini takes a balanced approach, and DeepSeek's generated code promotes clarity but suffers from complexity. More comprehensive analysis will be conducted in the future by expanding the dataset and including larger language models.
Cited by 0Open access
A. Kalogeropoulos, Johan Alwall · arXiv (Cornell University) · 2018
Undisputedly, derivation of theoretical systematic uncertainties is an inseparable ingredient of any robust analysis dealing with experimental data. However, it is not uncommon, even for those analyses that use state of the art methods and tools to suffer from insufficient statistics when it comes to the simulated datasets used to estimate systematic uncertainties. This practically limits the power, and sometimes the robustness of the analysis. In this paper, we present SysCalc, a code which is able to derive weights for various important theoretical systematic uncertainties, including those related to the choice of the Parton Distribution Function sets and the various scale choices. SysCalc utilizes the central sample generated events to estimate the related systematic uncertainties, thus, omitting the need for generating dedicated systematics datasets, and with only a minimal added cost in terms of computing resources. In this paper we discuss the working principles of the code accompanied by various validation plots. We also discuss the structure of the code followed by a practical guide for how to use the tool.
Cited by 49Open access
Deepika Badampudi, Ricardo Britto, Michael Unterkalmsteiner · arXiv (Cornell University) · 2019
Reviewing source code is a common practice in a modern and collaborative coding environment. In the past few years, the research on modern code reviews has gained interest among practitioners and researchers. The objective of our investigation is to observe the evolution of research related to modern code reviews, identify research gaps and serve as a basis for future research. We use a systematic mapping approach to identify and classify 177 research papers. As preliminary result of our investigation, we present in this paper a classification scheme of the main contributions of modern code review research between 2005 and 2018.
Cited by 20Open access
Mrs. Deepthi Repalle, Aswini, Sindhuja et al. · American Journal of Management and IOT Medical Computing · 2025
The systematic review on "Machine Learning Approaches to Code Similarity Measurement" provides a comprehensive analysis of how machine learning (ML) techniques have been applied to assess the degree of resemblance between code segments. Code similarity measurement is crucial in various software engineering tasks such as code quality assurance, plagiarism detection, vulnerability analysis, and malware detection. This review surveyed 84 primary studies, identifying 51 different ML algorithms used across 15 application areas. The abstract syntax tree (AST) representation of code emerged as the most prevalent, while the BigCloneBench dataset was the most frequently used benchmark. The review not only catalogs existing methodologies but also highlights the integration of deep learning and hybrid ML models that combine syntactic and semantic code features to improve similarity detection accuracy and scalability. It emphasizes the trend toward more sophisticated models, such as neural networks and Siamese networks, which better capture intricate code patterns and cross-language similarities. Additionally, the review addresses challenges faced by the field, notably the scarcity of diverse and high-quality public datasets and the limitations of current ML models in handling large-scale and multilanguage code bases. It underscores the importance of developing scalable tools capable of processing complex and vast code repositories efficiently. The findings also touch on the emergence of new applications, including code recommendation systems and automated code review, which benefit from improved code similarity assessments. Future research directions proposed include enhancing the interpretability of ML models, better handling of semantic context, and expanding the scope of code similarity studies beyond conventional clone detection to include novel tasks in software maintenance and security. Overall, this systematic review serves as a detailed roadmap for researchers and practitioners aiming to apply or advance ML techniques in code similarity measurement with a focus on both theory and practical impact.
Cited by 0Open access
Јелена Сливка, Nikola Luburić, Simona Prokić et al. · 2023
<p><strong>This is a preprint of an article published in the</strong> <em><strong>Science of Computer Programming</strong></em><strong>. The final peer-reviewed publication is available online at:</strong><br> <a href="https://doi.org/10.1016/j.scico.2023.102999" target="_blank"><u>https://doi.org/10.1016/j.scico.2023.102999</u></a> </p> <p><br></p> <p>Code smells are structures in code that indicate the presence of maintainability issues. A significant problem with code smells is their ambiguity. They are challenging to define, and software engineers have a different understanding of what a code smell is and which code suffers from code smells.</p> <p>A solution to this problem could be an AI digital assistant that understands code smells and can detect (and perhaps resolve) them. However, it is challenging to develop such an assistant as there are few usable datasets of code smells on which to train and evaluate it. Furthermore, the existing datasets suffer from issues that mostly arise from an unsystematic approach used for their construction.</p> <p>Through this work, we address this issue by developing a procedure for the systematic manual annotation of code smells. We use this procedure to build a dataset of code smells. During this process, we refine the procedure and identify recommendations and pitfalls for its use. The primary contribution is the proposed annotation model and procedure and the annotators’ experience report. The dataset and supporting tool are secondary contributions of our study. Notably, our dataset includes open-source projects written in the C# programming language, while almost all manually annotated datasets contain projects written in Java.</p>
Cited by 1Open access