Benjamin S. Meyers, Nuthan Munaiah, Andrew Meneely et al. · Zenodo (CERN European Organization for Nuclear Research) · 2019
This dataset will be released as part of the following publication. Benjamin S. Meyers, Nuthan Munaiah, Andrew Meneely, and Emily Prud'hommeaux. <strong>Pragmatic Characteristics of Security Conversation: An Exploratory Linguistic Analysis. </strong><em>Forthcoming.</em><strong> </strong>Proceedings of the 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE 2019). Montréal, QC, Canada. <strong>Files:</strong> <pre><code>security_bug_conversations.csv</code></pre> The full dataset containing over 2.1 million comments posted by developers discussing bugs in the Chromium project. The dataset also includes the values we calculated for the five pragmatic features (described in Section 3 of the paper cited above). <strong>CSV Fields:</strong> <strong>Organizational:</strong> <em>Bug ID:</em> Unique identifier of a bug discussion in the Chromium project. The URL https://bugs.chromium.org/p/chromium/issues/detail?id=<Bug ID> may be used to access the bug online <em>Comment ID:</em> Unique identifier of a comment in a bug discussion <strong>Classification:</strong> <em>Is Security:</em> Binary indicator of whether or not a comment is part of a bug that is about security <strong>Natural Language:</strong> <em>Comment Text:</em> The raw natural language text of the bug comment <strong>Linguistic Metrics:</strong> <em>Min. Formality:</em> Minimum of the formality of sentences in the bug comment <em>Max. Formality:</em> Maximum of the formality of sentences in the bug comment <em>Max. Informativeness:</em> Maximum of the informativeness of sentences in the bug comment <em>Max. Implicature:</em> Maximum of the implicature of sentences in the bug comment <em>Min. Politeness:</em> Minimum of the politeness of sentences in the bug comment <em>Max. Politeness:</em> Maximum of the politeness of sentences in the bug comment Number of Tokens Number of Sentences <em>Has Doxastic Uncertainty:</em> Binary indicator of presence of a sentence with doxastic uncertainty in the bug comment <em>Has Epistemic Uncertainty:</em> Binary indicator of presence of a sentence with epistemic uncertainty in the bug comment <em>Has Conditional Uncertainty:</em> Binary indicator of presence of a sentence with conditional uncertainty in the bug comment <em>Has Investigational Uncertainty:</em> Binary indicator of presence of a sentence with investigational uncertainty in the bug comment <em>Has Uncertainty:</em> Binary indicator of presence of a sentence with any uncertainty in the bug comment
Cited by 1Open access
Benjamin S. Meyers, Nuthan Munaiah, Andrew Meneely et al. · Zenodo (CERN European Organization for Nuclear Research) · 2019
This dataset will be released as part of the following publication. Benjamin S. Meyers, Nuthan Munaiah, Andrew Meneely, and Emily Prud'hommeaux. <strong>Pragmatic Characteristics of Security Conversation: An Exploratory Linguistic Analysis. </strong><em>Forthcoming.</em><strong> </strong>Proceedings of the 12th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE 2019). Montréal, QC, Canada. <strong>Files:</strong> <pre><code>security_bug_conversations.csv</code></pre> The full dataset containing over 2.1 million comments posted by developers discussing bugs in the Chromium project. The dataset also includes the values we calculated for the five pragmatic features (described in Section 3 of the paper cited above). <strong>CSV Fields:</strong> <strong>Organizational:</strong> <em>Bug ID:</em> Unique identifier of a bug discussion in the Chromium project. The URL https://bugs.chromium.org/p/chromium/issues/detail?id=<Bug ID> may be used to access the bug online <em>Comment ID:</em> Unique identifier of a comment in a bug discussion <strong>Classification:</strong> <em>Is Security:</em> Binary indicator of whether or not a comment is part of a bug that is about security <strong>Natural Language:</strong> <em>Comment Text:</em> The raw natural language text of the bug comment <strong>Linguistic Metrics:</strong> <em>Min. Formality:</em> Minimum of the formality of sentences in the bug comment <em>Max. Formality:</em> Maximum of the formality of sentences in the bug comment <em>Max. Informativeness:</em> Maximum of the informativeness of sentences in the bug comment <em>Max. Implicature:</em> Maximum of the implicature of sentences in the bug comment <em>Min. Politeness:</em> Minimum of the politeness of sentences in the bug comment <em>Max. Politeness:</em> Maximum of the politeness of sentences in the bug comment Number of Tokens Number of Sentences <em>Has Doxastic Uncertainty:</em> Binary indicator of presence of a sentence with doxastic uncertainty in the bug comment <em>Has Epistemic Uncertainty:</em> Binary indicator of presence of a sentence with epistemic uncertainty in the bug comment <em>Has Conditional Uncertainty:</em> Binary indicator of presence of a sentence with conditional uncertainty in the bug comment <em>Has Investigational Uncertainty:</em> Binary indicator of presence of a sentence with investigational uncertainty in the bug comment <em>Has Uncertainty:</em> Binary indicator of presence of a sentence with any uncertainty in the bug comment
Cited by 1Open access
Kiva (The University of Tennessee, Knoxville)
Cited by 0
Sultan S. Alqahtani · International Journal of Information Security · 2023
Cited by 3
Wei Zheng, Jingyuan Cheng, Xiaoxue Wu et al. · Knowledge-Based Systems · 2022
Cited by 48
Weiqin Ying, Xiaobing Sun, Lili Bo et al. · Journal of Software Evolution and Process · 2021
Abstract Security bugs can catastrophically impact our increasingly digital lives. Designing effective tools for detecting and fixing software security bugs requires a deep understanding of security bug characteristics. In this paper, we conducted a comprehensive study on security bugs and proposed the classification criteria for security bug category, that is, root cause, consequence, and location. In addition, we selected 1076 bug reports from five projects (i.e., Apache Tomcat, Apache HTTP Server, Mozilla Firefox, Linux Kernel, and Eclipse) in the NVD for investigation. Finally, we investigated the correlation between the classification results and obtained some findings: (1) memory operation is the most common security bug; (2) the primary root causes of security bugs are CON (Configuration Error), INP (Input Validation Error), and MEM (Memory Error); (3) the severity of more than 40% of security bugs is high; (4) security bugs caused by INP mainly occur on web; and (5) security bugs caused by LOG (Logic Resource Error) usually lead to DoS (Denial of Service). We discussed these findings through data analysis, which can also help developers better understand the characteristics of security bugs.
Cited by 26
Muhammad Laiq · arXiv (Cornell University) · 2026
Security bug reports require prompt identification to minimize the window of vulnerability in software systems. Traditional machine learning (ML) techniques for classifying bug reports to identify security bug reports rely heavily on large amounts of labeled data. However, datasets for security bug reports are often scarce in practice, leading to poor model performance and limited applicability in real-world settings. In this study, we propose a few-shot learning-based technique to effectively identify security bug reports using limited labeled data. We employ SetFit, a state-of-the-art few-shot learning framework that combines sentence transformers with contrastive learning and parameter-efficient fine-tuning. The model is trained on a small labeled dataset of bug reports and is evaluated on its ability to classify these reports as either security-related or non-security-related. Our approach achieves an AUC of 0.865, at best, outperforming traditional ML techniques (baselines) for all of the evaluated datasets. This highlights the potential of SetFit to effectively identify security bug reports. SetFit-based few-shot learning offers a promising alternative to traditional ML techniques to identify security bug reports. The approach enables efficient model development with minimal annotation effort, making it highly suitable for scenarios where labeled data is scarce.
Cited by 0Open access
Chris Bucholtz · Network Security · 1995
Cited by 0
Lou · 2013
To view a PDF of this First Cut, click here. Author: Lou Latham Topic: Mobile Issue: What are the trends impacting mobile computing? Summary: A security research firm has found a flaw in the Android operating system that makes nearly a billion phones...
Cited by 0
Muhammad Laiq · ArXiv.org · 2026
Security bug reports require prompt identification to minimize the window of vulnerability in software systems. Traditional machine learning (ML) techniques for classifying bug reports to identify security bug reports rely heavily on large amounts of labeled data. However, datasets for security bug reports are often scarce in practice, leading to poor model performance and limited applicability in real-world settings. In this study, we propose a few-shot learning-based technique to effectively identify security bug reports using limited labeled data. We employ SetFit, a state-of-the-art few-shot learning framework that combines sentence transformers with contrastive learning and parameter-efficient fine-tuning. The model is trained on a small labeled dataset of bug reports and is evaluated on its ability to classify these reports as either security-related or non-security-related. Our approach achieves an AUC of 0.865, at best, outperforming traditional ML techniques (baselines) for all of the evaluated datasets. This highlights the potential of SetFit to effectively identify security bug reports. SetFit-based few-shot learning offers a promising alternative to traditional ML techniques to identify security bug reports. The approach enables efficient model development with minimal annotation effort, making it highly suitable for scenarios where labeled data is scarce.
Cited by 0Open access
Yunwu Xu, Yan Li · IT Professional · 2024
The investigation develops a method for improving the quality of security bug report (SBR) prediction during the software development and application processes. The research includes three stages. The first stage is preparing the source data. The second stage is constructing an original SBR prediction method using a machine learning algorithm [random forest (RF)]. The third stage is evaluating our method with well-established methods like filtering and ranking for security bug report prediction (FARSEC) and Keywords Matrix. It was shown that the values of such indicators as accuracy, precision, recall, and F-score when using the RF algorithm are, on average, 0.2–1% higher than when using the FARSEC and Keywords Matrix methods. The more initial number of reports the database contains, the higher the value of accuracy, precision, recall, and F-score that can be obtained. A new method can be used to predict SBRs during the software development and application processes.
Cited by 0
Tie Du, Long Zheng, Shaopeng Chen et al. · 2016
Program security bugs pose a great threat to users' privacy and security. A great deal of effort, e.g., runtime defense, dynamic detection, and static detection, has been conducted to attempt to be aware of the existence of security bugs. Most of the prior work focuses on detecting the security bugs. They report a mixed set of security bugs, regardless of whether the elements in the set are useful to the developers for the debugging. In this paper, we are instead devoted to automatically classifying the security bugs for the purpose of the productivity to the developers. Our insight is that the existing common security bugs can be featured by a simple rule that can be further simplified into a mathematical assertion problem. Based on this insight, we propose a Compile Time Error Segregator (CTES), which can automatically classify the security bugs into three categories, including deterministic bugs, internal indeterministic bugs, and external indeterministic bugs. The core idea of achieving the above includes three steps: 1) building a rule library according to the feature of each type of security bugs (e.g., buffer overflow, null-pointer dereference, and divide-by-zero), 2) obtaining the requisite information appearing in the rule, 3) verifying if the rule is established. If so, a deterministic bug is found, otherwise, a novel inverse taint analysis is further performed to distinguish the remaining two categories. We implement CTES on top of LLVM (3.5.0), running in parallel with normal compile procedure. Our experimental results on micro-benchmark and 14 real programs demonstrate the efficiency of CTES, and also show that CTES is able to precisely make the reported security bugs well-classified into three-categories.
Cited by 0
Zhou, Jiali ISOM, Hui, Kai Lung · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2019
Bug bounty programs are gaining popularity, but practitioners have not agreed on their effectiveness. We use a stylized model to analyze the economic trade-offs in bug bounty programs. Our analysis provides six main insights: （1) Unless the company needs to pay an excessive bounty reward to entice a strategic hacker to participate, it is always beneficial for the company to launch a bug bounty program. (2) The company enjoys two benefits from a bug bounty program: attack diversion and protection delegation. (3) How the company designs the bounty and selects its in-house protection strategy depend on the balance of two incentives: bounty-payment squeezing and protection free-riding. (4) Law enforcement affects private protection only when the level of enforcement is relatively small. In this case, strengthening law enforcement increases the company’s payoff but could variously increase or decrease the strategic hacker’s payoff. Excessive enforcement may make a system less secure. (5) When the exogenous threat is significant, the company prefers a more capable participant. But when the exogenous threat is small, the company prefers a complementary participant. (6) Bug bounty programs need not provide better protection.
Cited by 3
Farnaz Soltaniani · Zenodo (CERN European Organization for Nuclear Research) · 2026
Cited by 0Open access
Fayola Peters, Thein Than Tun, Yijun Yu et al. · IEEE Transactions on Software Engineering · 2017
Security bug reports can describe security critical vulnerabilities in software products. Bug tracking systems may contain thousands of bug reports, where relatively few of them are security related. Therefore finding unlabelled security bugs among them can be challenging. To help security engineers identify these reports quickly and accurately, text-based prediction models have been proposed. These can often mislabel security bug reports due to a number of reasons such as class imbalance, where the ratio of non-security to security bug reports is very high. More critically, we have observed that the presence of security related keywords in both security and non-security bug reports can lead to the mislabelling of security bug reports. This paper proposes FARSEC, a framework for filtering and ranking bug reports for reducing the presence of security related keywords. Before building prediction models, our framework identifies and removes non-security bug reports with security related keywords. We demonstrate that FARSEC improves the performance of text-based prediction models for security bug reports in 90 percent of cases. Specifically, we evaluate it with 45,940 bug reports from Chromium and four Apache projects. With our framework, we mitigate the class imbalance issue and reduce the number of mislabelled security bug reports by 38 percent.
Cited by 85
Farnaz Soltaniani · Zenodo (CERN European Organization for Nuclear Research) · 2026
Cited by 0Open access
Wajdi Aljedaani, Yasir Javed, Mamdouh Alenezi · 2020
Security bug reports (SBR) depict potential security vulnerabilities in software systems. Bug tracking systems (BTS) usually contain huge numbers of bug reports including security-related ones. Malicious attackers could exploit these SBRs. Henceforth, it is very critical to pinpoint SBRs swiftly and correctly. In this work, we studied the security bug reports of the Chromium project. We looked into three main aspects of these bug reports, namely: frequencies of reporting them, how quickly they get fixed and is LDA effective in grouping these reports to known vulnerabilities types. We report our findings in these aspects.
Cited by 12
Farnaz Soltaniani · Zenodo (CERN European Organization for Nuclear Research) · 2026
Cited by 0Open access
Deqing Zou, Zhijun Deng, Zhen Li et al. · Lecture notes in computer science · 2018
Cited by 12
Farnaz Soltaniani, Shoaib Razzaq, Mohammad Ghafari · ArXiv.org · 2026
Early detection of security bug reports (SBRs) is critical for timely vulnerability mitigation. We present an evaluation of prompt-based engineering and fine-tuning approaches for predicting SBRs using Large Language Models (LLMs). Our findings reveal a distinct trade-off between the two approaches. Prompted proprietary models demonstrate the highest sensitivity to SBRs, achieving a G-measure of 77% and a recall of 74% on average across all the datasets, albeit at the cost of a higher false-positive rate, resulting in an average precision of only 22%. Fine-tuned models, by contrast, exhibit the opposite behavior, attaining a lower overall G-measure of 51% but substantially higher precision of 75% at the cost of reduced recall of 36%. Though a one-time investment in building fine-tuned models is necessary, the inference on the largest dataset is up to 50 times faster than that of proprietary models. These findings suggest that further investigations to harness the power of LLMs for SBR prediction are necessary.
Cited by 0Open access