Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “language processing models”

Natural Language Processing and Large Language Models

Peter Wulff, Marcus Kubsch, Christina Krist · Springer texts in education · 2025

Abstract In this chapter we introduce the basics of natural language processing techniques that are important to systematically analyze language data. In particular, we will utilize simple large language models and showcase examples of how to apply them in science education research contexts. We will also point to recently advanced large language models that are capable of solving problems without further training, which opens up novel potentials (and challenges) for science education research.

Cited by 1Open access

Efficient Natural Language Processing for Language Models

Xu, Canwen · eScholarship (California Digital Library) · 2024

Despite achieving state-of-the-art performance on many NLP tasks, the high energy cost and long inference delay prevent Transformer-based language models (LMs) from seeing broader adoption including for edge and mobile computing. Our efficient NLP research aims to comprehensively consider computation, time and carbon emission for the entire life-cycle of NLP, including data preparation, model training and inference.We demonstrate ways to promote computational efficiency in natural language processing, thus reducing hardware and software bottlenecks of training and inference, which is crucial in applying such models in production. Efficient NLP further facilitates democratization of language technology and allows language models to be accessible to more people.

Cited by 0Open access

Neural Language Models in Natural Language Processing

Zihao Chen · 2023

People cannot live without natural language in their daily life, and natural language is also an essential part of the heritage of human civilization. With the rapid development of information technology and the explosive growth of various data, natural language processing (NLP) technology based on deep learning and other technologies in the field of artificial intelligence has emerged as the times require. With the rapid development of deep learning models in recent years, breakthroughs have been made in the field of natural language processing. Based on recent research, this paper briefly introduces the development process of deep learning, the concepts of deep learning and NLP, the methods of deep learning used to solve the core problems of NLP, the application of the neural network model in natural language modeling. This paper summarizes the current development achievements and forecasts its future development.

Cited by 4

Exploring Large Language Models for Natural Language Processing

Ayodeji Olalekan Salau, Esuku Dickson Emmanuel, Ahmed Al‐Emran et al. · 2024

Large language models have become increasingly popular in natural language processing due to their ability to generate high-quality text and perform various language-related tasks. However, there are growing concerns regarding the ethical implications of these models, particularly concerning potential misuse and bias. This paper delves into the advancements and limitations of large language models for Natural Language Processing (NLP), exploring their ability to improve language understanding and generation. The paper also examines the challenges these models face in terms of computational power and model interpretability. Finally, the paper provides recommendations for future research to address these concerns, ensuring the responsible development of large language models and their integration into the future of NLP and Artificial Intelligence (AI).

Cited by 3

Language Processing—language models, text analysis General Terms

Kazuhiro Seki · 2012

This paper proposes a novel application of a statistical language model to opinionated document retrieval targeting weblogs (blogs). In particular, we explore the use of the trigger model—originally developed for incorporating distant word dependencies—in order to model the characteristics of personal opinions that cannot be properly modeled by standard n-grams. Our primary assumption is that there are two constituents to form a subjective opinion. One is the subject of the opinion or the object that the opinion is about, and the other is a subjective expression; the former is regarded as a triggering word and the latter as a triggered word. We automatically identify those subjective trigger patterns to build a language model from a corpus of product customer reviews. Experimental results on the TREC Blog Track test collections show that, when used for reranking initial search results, our proposed model significantly improves opinionated document retrieval by over 20 % in MAP. In addition, we report on an experiment on dynamic adaptation of the model to a given query, which is found effective for most of difficult queries categorized under politics and organizations.

Cited by 0

Accelerating materials language processing with large language models

Jaewoong Choi, Byungju Lee · Communications Materials · 2024

Abstract Materials language processing (MLP) can facilitate materials science research by automating the extraction of structured data from research papers. Despite the existence of deep learning models for MLP tasks, there are ongoing practical issues associated with complex model architectures, extensive fine-tuning, and substantial human-labelled datasets. Here, we introduce the use of large language models, such as generative pretrained transformer (GPT), to replace the complex architectures of prior MLP models with strategic designs of prompt engineering. We find that in-context learning of GPT models with few or zero-shots can provide high performance text classification, named entity recognition and extractive question answering with limited datasets, demonstrated for various classes of materials. These generative models can also help identify incorrect annotated data. Our GPT-based approach can assist material scientists in solving knowledge-intensive MLP tasks, even if they lack relevant expertise, by offering MLP guidelines applicable to any materials science domain. In addition, the outcomes of GPT models are expected to reduce the workload of researchers, such as manual labelling, by producing an initial labelling set and verifying human-annotations.

Cited by 56Open access

Interactive Neurocognitive Models of Language Processing

Svitlana Shestakova, Liubov Oliinyk, Nataliia Rebryk et al. · Revista Romaneasca pentru Educatie Multidimensionala · 2022

The article shows that lexico-semantic innovations in the Ukrainian language system of proper names which are characterized by a specific neurophysiological mechanism of formation require in-depth study. The relevance of the article lies in the fact that lexico-semantic word formation used to be mainly considered in the diachronic aspect, which led to little attention to ergonyms and pragmonyms as relevant classes of proper names. The article aims to analyze lexico-semantic innovations in the context of ergonyms and pragmonyms and determine their role in word formation in the system of naming-related tools in the modern Ukrainian language. Research methods include observations, analogies, analysis, synthesis, linguistic description, component analysis, structural-semantic analysis, statistical methods. The article proves that lexico-semantic word formation is the most active and productive way of replenishing the fund of ergonyms and pragmonyms in the Ukrainian language under the conditions of a transition economy. The main types of lexico-semantic word formation are derivational metaphor and derivational metonymy. They enable the formation of both simple and compound lexico-semantic innovations mostly represented by binary word combinations. Including all layers of vocabulary in the scope of such a type of word formation can bring unlimited prospects of its functioning in the future. The international value of the research implies clarifying the role of lexico-semantic word formation in the naming system of modern Ukrainian, describing important fragments of the lexical structure of the Ukrainian language, systematizing modern lexico-semantic naming models, and determining the ways of activating lexico-semantic substitution. For the first time in Ukrainian linguistics, the research analyzes peripheral classes of proper names in various ways, classifies ergonyms by thematic groups and motivational features underlying the naming unit and reveals the specifics of different semantic groups of Ukrainian pragmonyms.

Cited by 0Open access

Plaut Connectionist Models of Language Processing

Douglas L. T. Rohde, David C. Plaut · 1993

Traditional approaches to language processing have been based on explicit, discrete representations which are difficult to learn from a reasonable linguistic environment— hence, it has come to be accepted that much of our linguistic representations and knowledge is innate. With its focus on learning based upon graded, malleable, distributed representations, connectionsit modeling has reopened the question of what could be learned from the environment in the absence of detailed innate knowledge. This paper provides an overview of connectionist models of language processing, at both the lexical and sentence levels. 1

Cited by 0

Incremental models for natural language processing

Dary, Franck · theses.fr (ABES) · 2022

Cette thèse s'inscrit dans le domaine du traitement automatique de la langue, elle concerne plus particulièrement la prédiction de la structure syntaxico morphologique des phrases.Il s'agit donc, à partir d'un texte, de découper ce dernier en phrases puis en mots et d'associer aux mots une partie de discours et des traits morphologiques puis de relier les mots pour expliciter la structure syntaxique.La thèse propose un modèle de prédiction qui permet de réaliser ces tâches simultanément et de manière incrémentale : le texte est lu caractère par caractère et l'intégralité des prédictions linguistiques sont mises à jour par l'information apportée par chaque nouveau caractère.La raison pour laquelle nous avons exploré cette architecture est la volonté de s'inspirer de la lecture humaine qui impose ces deux contraintes.D'un point de vue expérimental, nous calculons la corrélation entre des variables oculométriques mesurées sur des sujets humains et des mesures de complexité propre à notre modèle.De plus, nous proposons un mécanisme de retour arrière, inspiré des saccades régressives observées chez l'humain. Pour cela, nous utilisons l'apprentissage par renforcement, qui permet au modèle d'effectuer des retours arrières lorsqu'il se trouve dans une impasse.

Cited by 0Open access

Review on natural language processing models

LI Yi-kun · Applied and Computational Engineering · 2024

Accessing information has grown simpler as a result of the internet's expanding use and the arrival of the big data era. Compared to traditional approaches, employing NLP for information condensation and amalgamation proves to be a highly effective method. This article focuses primarily on the sentiment analysis aspect of NLP, offering a comprehensive exploration of two deep learning models: BERT and CNN. It delves into the intricacies of their principles, analyzes their respective strengths and weaknesses, and proposes potential avenues for enhancement. By delving into these models, Researchers and practitioners can obtain a better understanding of sentiment analysis and its applications in diverse fields.

Cited by 0Open access

Diffusion models for natural language processing

Hoang, Minh Nhat · DR-NTU (Nanyang Technological University) · 2024

The autoregressive method has long been fundamental in natural language processing (NLP), where models predict the next token based on previously generated context. While effective, this approach suffers from inherent limitations, including slow generation speed and sequential dependency, hindering its scalability and efficiency in large-scale applications. Non-autoregressive methods offer a promising solution by enabling parallel decoding and mitigating the sequential bottleneck. In recent years, diffusion models have proven its superior performance in continuous domain such as image and video, and have also recently emerged as a non-autoregressive alternative in NLP. This paper explores the application of diffusion models across various NLP tasks, ranging from language modeling to sequence generation. Through extensive experimentation, we demonstrate that diffusion models not only rival but surpass predominated autoregressive architecture such as the fine-tuned GPT-2 Large models. Moreover, our analysis reveals that diffusion models exhibit a remarkable ability to generate diverse outputs under identical settings, indicating their potential to revolutionize NLP tasks. In summary, this paper presents a comprehensive investigation into the application of diffusion models in NLP, shedding light on their efficacy and potential to address longstanding challenges associated with autoregressive methods. These findings underscore the significance of diffusion models as a viable alternative for enhancing efficiency and scalability in natural language processing tasks.

Cited by 0