Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 5,417 papers for “data incompleteness”

Explaining Data Incompleteness in Knowledge Aggregation

Honglei Zeng, Richard Fikes · 2005

Knowledge aggregation is the problem of taking information from multiple heterogeneous sources and aggregating it into a unified knowledge base. One of the main challenges in that work has been dealing with data incompleteness because data sources seldom contain complete answers to a user’s query. Current approaches leverage users’ preferences over data sources when trying to aggregate incomplete data. Nevertheless, these approaches are not adequate to satisfy users’ needs to trust aggregated data before they can use them with confidence in the presence of incomplete information. We believe such trust may be earned by providing users with the explanations for incomplete data. In this paper, we build a decision tree-based classification system to acquire context knowledge about the sources and present techniques for applying the knowledge to explain incomplete data. Our experiments suggest the decision trees we built being 87% accurate in predicting unseen data. Further, context knowledge provides good characterizations of sources that we show to be valuable and often critical to users.

Cited by 3

Modern Data Management: addressing schemaflexibility and data incompleteness

Mohamed-Amine Baazizi · SPIRE - Sciences Po Institutional REpository · 2025

The thesis covers several contributions addressing challenges in the context of modern data management.The first challenge is related to the problem of schema inference for massive JSON collections. The second challenge is motivated bythe adoption of JSON Schema as the standard language for describing JSON collections and is related to the problem of generatingwitnesses and instances from schemas. The last challenge, which is more traditional, concerns the problem of data incompleteness inthe context of analytical databases.

Cited by 0

Analyzing Data Incompleteness for MRI Data for Quality Enhancement

SOUJANYA SHANBHAG, S. Sivanaga Raju, Varadraj P. Gurupur et al. · IEEE Access · 2024

Magnetic resonance imaging (MRI) is a powerful medical imaging technique widely used for diagnosing various conditions because it provides detailed images of internal structures within the body. However, like any imaging modality, MRI images can be susceptible to artifacts that may arise from various sources, including hardware imperfections, patient motion, and image acquisition techniques. Detecting and mitigating these artifacts are crucial steps in ensuring MRI scans’ reliability and clinical utility. In this paper, we present algorithms specifically designed to address the challenges of undersampling and motion artifacts in MR images. Our approach involves leveraging advanced image processing techniques, including line detection algorithms for undersampling detection and blur parameter estimation for motion artifact analysis. By accurately identifying and quantifying these artifacts, our algorithms aim to improve MRI data’s overall quality and completeness, ultimately enhancing diagnostic accuracy and patient care.

Cited by 0Open access

Embracing Data Incompleteness for Better Earthquake Forecasting

Leila Mizrahi, Shyam Nandan, Stefan Wiemer · Journal of Geophysical Research Solid Earth · 2021

Abstract We propose two methods to calibrate the parameters of the epidemic‐type aftershock sequence (ETAS) model based on expectation maximization (EM) while accounting for temporal variation of catalog completeness. The first method allows for model calibration on long‐term earthquake catalogs with temporal variation of the completeness magnitude, m c . This calibration technique is beneficial for long‐term probabilistic seismic hazard assessment (PSHA), which is often based on a mixture of instrumental and historical catalogs. The second method generalizes the concept of m c , considering rate‐ and magnitude‐dependent detection probability, and allows for self‐consistent estimation of ETAS parameters and high‐frequency detection incompleteness. With this approach, we aim to address the potential biases in parameter calibration due to short‐term aftershock incompleteness, embracing incompleteness instead of avoiding it. Using synthetic tests, we show that both methods can accurately invert the parameters of simulated catalogs. We then use them to estimate ETAS parameters for California using the earthquake catalog since 1932. To explore how model calibration, inclusion of small events, and accounting for short‐term incompleteness affect earthquakes' predictability, we systematically compare variants of ETAS models based on the second approach in pseudo‐prospective forecasting experiments for California. Our proposed model significantly outperforms the ETAS null model, with decreasing information gain for increasing target magnitude threshold. We find that the ability to include small earthquakes for simulation of future scenarios is the primary driver of the improvement and that accounting for incompleteness is necessary. Our results have significant implications for our understanding of earthquake interaction mechanisms and the future of seismicity forecasting.

Cited by 35Open access

Embracing Data Incompleteness for Better Earthquake Forecasting

Leila Mizrahi, Shyam Nandan, Stefan Wiemer · ETH Zürich Research Collection · 2021

We propose two new methods to calibrate the parameters of the epidemic-type aftershock sequence (ETAS) model based on expectation maximization (EM) while accounting for temporal variation of catalog completeness. The first method allows for model calibration on earthquake catalogs with long history, featuring temporal variation of the magnitude of completeness, mc. This extended calibration technique is beneficial for long-term probabilistic seismic hazard assessment (PSHA), which is often based on a mixture of instrumental and historical catalogs. The second method jointly estimates ETAS parameters and high-frequency detection incompleteness to address the potential biases in parameter calibration due to short-term aftershock incompleteness. For this, we generalize the concept of completeness magnitude and consider a rate- and magnitude-dependent detection probability – embracing incompleteness instead of avoiding it. Using synthetic tests, we show that both methods can accurately invert the parameters of simulated catalogs. We then use them to estimate ETAS parameters for California using the earthquake catalog since 1932. To explore how the newly gained information from the second method affects earthquakes’ predictability, we conduct pseudo-prospective forecasting experiments for California. Our proposed model significantly outperforms the base ETAS model, and we find that the ability to include small earthquakes for simulation of future scenarios is the main driver of the improvement. Our results point towards a preference of earthquakes to trigger similarly sized aftershocks, which has potentially major implications for our understanding of earthquake interaction mechanisms and for the future of seismicity forecasting.

Cited by 1Open access

Explaining data incompleteness in knowledge aggregation. Under review

Honglei Zeng · 2005

Knowledge aggregation is the problem of taking information from multiple heterogeneous sources and aggregating it into a unified knowledge base. One of the main challenges in that work has been dealing with data incompleteness because data sources seldom contain complete answers to a user’s query. Current approaches leverage users ’ preferences over data sources when trying to aggregate incomplete data. Nevertheless, these approaches are not adequate to satisfy users ’ needs to trust aggregated data before they can use them with confidence in the presence of incomplete information. We believe such trust may be earned by providing users with the explanations for incomplete data. In this paper, we build a decision tree-based classification system to acquire context knowledge about the sources and present techniques for applying the knowledge to explain incomplete data. Our experiments suggest the decision trees we built being 87 % accurate in predicting unseen data. Further, context knowledge provides good characterizations of sources that we show to be valuable and often critical to users.

Cited by 0

Gradient-Aware Data Augmentation for Federated Stress Detection under Data Incompleteness

Woan-Shiuan Chien, Huanyu Chen, Chi-Chun Lee · 2025

Federated learning enables privacy-preserving stress detection when leveraging wearable devices to monitor physiological indicators without transmitting raw data. However, missing data in federated settings remains a critical challenge, disrupting model training, introducing disparities, and leading to degraded performances among clients. In this work, we explore the impact of missing data on stress detection performances and the gradient magnitudes observed during training across two datasets. Our analysis reveals that the bias induced by missing data directly impacts client performance and is closely related to patterns of gradients during training. To mitigate the effects of missing data in FL setting, we introduce a flexible gradient-aware mechanism that dynamically adjusts data augmentation. Our results show the efficacy of our approach in improving overall stress detection performance while reducing performance disparities among clients.

Cited by 0

On the dynamical incompleteness of the Protein Data Bank

Monzón, Alexander, Marino Buslje, Cristina, Zea, Diego Javier et al. · CONICET Digital (CONICET) · 2017

Major scientific challenges that are beyond the capability of individuals need to be addressed by multi-disciplinary and multi-institutional consortia. Examples of these endeavours include the Human Genome Project, and more recently, the Structural Genomics (SG) initiative.The SG initiative pursues the expansion of structural coverage to include at least one structural representative for each protein family to derive the remaining structures using homology modelling. However, biological function is inherently connected with protein dynamics that can be studied by knowing different structures of the same protein. This ensemble of structures provides snapshots of protein conformational diversity under native conditions. Thus, sequence redundancy in the Protein Data Bank (PDB) (i.e. crystallization of the same protein under different conditions) is therefore an essential input contributing to experimentally based studies of protein dynamics and providing insights into protein function.In this work, we show that sequence redundancy, a key concept for exploring protein dynamics, is highly biased and fundamentally incomplete in the PDB. Additionally, our results show that dynamical behaviour of proteins cannot be inferred using homologous proteins. Minor to moderate changes in sequence can produce great differences in dynamical behaviour.Nonetheless, the structural and dynamical incompleteness of the PDB is apparently unrelated concepts in SG. While the first could be reversed by promoting the extension of the structural coverage, we would like to emphasize that furtherfocused efforts will be needed to amend the incompleteness of the PDB in terms of dynamical information content, essential to fully understand protein function.

Cited by 0Open access

On the dynamical incompleteness of the Protein Data Bank

Cristina Marino‐Buslje, Alexander Miguel Monzón, Diego Javier Zea et al. · Briefings in Bioinformatics · 2017

Major scientific challenges that are beyond the capability of individuals need to be addressed by multi-disciplinary and multi-institutional consortia. Examples of these endeavours include the Human Genome Project, and more recently, the Structural Genomics (SG) initiative. The SG initiative pursues the expansion of structural coverage to include at least one structural representative for each protein family to derive the remaining structures using homology modelling. However, biological function is inherently connected with protein dynamics that can be studied by knowing different structures of the same protein. This ensemble of structures provides snapshots of protein conformational diversity under native conditions. Thus, sequence redundancy in the Protein Data Bank (PDB) (i.e. crystallization of the same protein under different conditions) is therefore an essential input contributing to experimentally based studies of protein dynamics and providing insights into protein function. In this work, we show that sequence redundancy, a key concept for exploring protein dynamics, is highly biased and fundamentally incomplete in the PDB. Additionally, our results show that dynamical behaviour of proteins cannot be inferred using homologous proteins. Minor to moderate changes in sequence can produce great differences in dynamical behaviour. Nonetheless, the structural and dynamical incompleteness of the PDB is apparently unrelated concepts in SG. While the first could be reversed by promoting the extension of the structural coverage, we would like to emphasize that further focused efforts will be needed to amend the incompleteness of the PDB in terms of dynamical information content, essential to fully understand protein function.

Cited by 18

Topological effects of data incompleteness of gene regulatory networks

Joaquín Sanz, Emanuele Cozzo, Javier Borge‐Holthoefer et al. · arXiv (Cornell University) · 2012

The topological analysis of biological networks has been a prolific topic in network science during the last decade. A persistent problem with this approach is the inherent uncertainty and noisy nature of the data. One of the cases in which this situation is more marked is that of transcriptional regulatory networks (TRNs) in bacteria. The datasets are incomplete because regulatory pathways associated to a relevant fraction of bacterial genes remain unknown. Furthermore, direction, strengths and signs of the links are sometimes unknown or simply overlooked. Finally, the experimental approaches to infer the regulations are highly heterogeneous, in a way that induces the appearance of systematic experimental-topological correlations. And yet, the quality of the available data increases constantly. In this work we capitalize on these advances to point out the influence of data (in)completeness and quality on some classical results on topological analysis of TRNs, specially regarding modularity at different levels. In doing so, we identify the most relevant factors affecting the validity of previous findings, highlighting important caveats to future prokaryotic TRNs topological analysis.

Cited by 0Open access