dataset · Figshare · 2015
Extraction of function words.
Maryam Ebrahimpour (379421), Tālis J. Putniņš (379422), Matthew J. Berryman (379423), Andrew Allison (379424), Brian W.-H. Ng (379425), Derek Abbott (105888)
Cited by 0Open access
Abstract
<p>In this paper, our investigation uses up to 100 function words for the English corpus, the <i>Federalist Papers</i>, and the <i>Koine</i> Greek texts. The frequencies of function words in each text are used as classification features. The texts are initially pre-processed as follows: (i) all letters are changed to lower case, (ii) all accents are removed, (iii) all ASCII characters not in the set a-z (ASCII codes 97–122) and space (ASCII code 32) are removed without insertion of a space. This is with the exception of a hyphen (ASCII code 45) that is substituted with a space (ASCII code 32), (iv) all headings are removed from the texts, so that they only contain free flowing paragraphs, (v) any extra items added by modern editors, such as editorial notes are removed. Whilst hagiographers wrote the Greek texts in capital letters without accents or punctuation, modern editors insert these items for ease of interpretation. Thus, in order to recover the original Greek text, we apply steps (i) to (v). We do the same to English texts so that they act as a punctuation-free benchmark test. As our software only handles the reduced 27-character ASCII set a-z and space, the Greek text is transliterated using the <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0054998#pone-0054998-t008" target="_blank">Table 8</a>. After this pre-processing, all the texts within each corpus are concatenated and word frequencies are counted. Words are ranked in descending order of frequency of occurrence. This is shown in the table below. These key words are called <i>function words</i>.</p>
Cite this paper
(379421), M. E., (379422), T. J. P., (379423), M. J. B., (379424), A. A., (379425), B. W.-H. N., & (105888), D. A. (2015). Extraction of function words. Figshare. https://doi.org/10.1371/journal.pone.0054998.t004
(379421), Maryam Ebrahimpour, et al. “Extraction of function words.” Figshare, 2015, https://doi.org/10.1371/journal.pone.0054998.t004.
(379421), Maryam Ebrahimpour, Tālis J. Putniņš (379422), Matthew J. Berryman (379423), Andrew Allison (379424), Brian W.-H. Ng (379425), and Derek Abbott (105888). 2015. “Extraction of function words.” Figshare. https://doi.org/10.1371/journal.pone.0054998.t004.
(379421), M.E. et al. (2015) ‘Extraction of function words’, Figshare. Available at: https://doi.org/10.1371/journal.pone.0054998.t004.
@misc{3794212015extraction,
title = {{Extraction of function words.}},
author = {(379421), Maryam Ebrahimpour and (379422), Tālis J. Putniņš and (379423), Matthew J. Berryman and (379424), Andrew Allison and (379425), Brian W.-H. Ng and (105888), Derek Abbott},
journal = {Figshare},
year = {2015},
doi = {10.1371/journal.pone.0054998.t004}
}Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.