preprint · arXiv (Cornell University) · 2023

Uncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs

Mia Mohammad Imran, Preetha Chatterjee, Kostadin Damevski

Cited by 1Open access

Abstract

Understanding and identifying the causes behind developers' emotions (e.g., Frustration caused by `delays in merging pull requests') can be crucial towards finding solutions to problems and fostering collaboration in open-source communities. Effectively identifying such information in the high volume of communications across the different project channels, such as chats, emails, and issue comments, requires automated recognition of emotions and their causes. To enable this automation, large-scale software engineering-specific datasets that can be used to train accurate machine learning models are required. However, such datasets are expensive to create with the variety and informal nature of software projects' communication channels. In this paper, we explore zero-shot LLMs that are pre-trained on massive datasets but without being fine-tuned specifically for the task of detecting emotion causes in software engineering: ChatGPT, GPT-4, and flan-alpaca. Our evaluation indicates that these recently available models can identify emotion categories when given detailed emotions, although they perform worse than the top-rated models. For emotion cause identification, our results indicate that zero-shot LLMs are effective at recognizing the correct emotion cause with a BLEU-2 score of 0.598. To highlight the potential use of these techniques, we conduct a case study of the causes of Frustration in the last year of development of a popular open-source project, revealing several interesting insights.

Cite this paper

APA 7

Imran, M. M., Chatterjee, P., & Damevski, K. (2023). Uncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2312.09731

MLA 9

Imran, Mia Mohammad, et al. “Uncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs.” arXiv (Cornell University), 2023, https://doi.org/10.48550/arxiv.2312.09731.

Chicago

Imran, Mia Mohammad, Preetha Chatterjee, and Kostadin Damevski. 2023. “Uncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2312.09731.

Harvard

Imran, M.M., Chatterjee, P. and Damevski, K. (2023) ‘Uncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.2312.09731.

BibTeX

@misc{imran2023uncovering,
  title = {{Uncovering the Causes of Emotions in Software Developer Communication Using Zero-shot LLMs}},
  author = {Imran, Mia Mohammad and Chatterjee, Preetha and Damevski, Kostadin},
  journal = {arXiv (Cornell University)},
  year = {2023},
  doi = {10.48550/arxiv.2312.09731}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.