Hicham Moad Safhi, Bouchra Frikh, Brahim Ouhbi · 2019
Big Data presents promising technological and economical opportunities. In fact, it has become the raw material of production for many organizations. Data is available in large quantities, and it continues generating abundantly. However, not all the data will have valuable knowledge. Unreliable sources provide misleading and biased information, and even reliable sources could suffer from low data quality.
Cited by 1
Nathalie Goemans (503769), Brenda Wong (495712), Marleen Van den Hauwe (4898152) et al. · Figshare · 2020
Cited by 0Open access
Pascal Molli, Hala Skaf‐Molli, Arnaud Grall · HAL (Le Centre pour la Communication Scientifique Directe) · 2020
As a web search engine is able to find relevant sources for a keyword query, the web of data clearly needs a source selection engine able to find relevant endpoints for a SPARQL query. However, source selection requires to get informations about the content of endpoints and currently, it remains difficult to automatically explore the content of endpoints as web robots explore the content of web servers. Thanks to the web preemption principle, we propose to automatically build RDF summaries of endpoints through SPARQL queries. We propose SemCat, an approach to compute the source selection of a query Q by evaluating a rewriting of Q on summaries. As all queries terminate under the web preemption paradigm, SemCat is able to provide a web automated source selection service relying on preemptable SPARQL servers. We empirically demonstrate that various summaries can be extracted with a data transfer proportional to the size of the summary, and highlight the trade-off between the size of the summaries, the accuracy of source selection and the execution time of source selection.
Cited by 1Open access
Malkov, Saša, Mitić, Nenad · IMAGINE - Repository of the Institute of molecular genetics and genetic engineering (University of Belgrade) · 2024
Bioinformatics experiments often produce large data sets, with lots of samples and many different attributes, which provide high dimensionality of the data. Even if some of the dimensions have little significance for specific data analysis, they can prove useful in complex data processing. Advanced data mining techniques and AI algorithms typically welcome high dimensionality of data. However, if there are too many dimensions, we can run into the curse of dimensionality, because an abundance of dimensions can introduce additional complexity and cost to data handling and processing, as well as overfitting the model to less important dimensions. In order to make data processing more efficient and improve the quality of created models, we often need dimensionality reduction. Single cell transcriptomics (SCT) is one of the most important sequencing technologies today. It enables simultaneous measurement of the activity of thousand of genes in individual cells, resulting in RNA profiles of the cells. Such profiles allow researchers to analyze the physiological activity of cells in different circumstances, including different biochemical conditions but also different health conditions of the subjects. By processing large numbers of cells, SCT provides a lot of sample data. Each detectable RNA represents one dimension of data, which ultimately gives us thousands of dimensions. Moreover, initial cells conditions, complex cell preparation techniques, and RNA measurement methods can vary significantly, resulting in significant differences in data coming from different sources. Here we discuss the feature selection problem on the example of more than 120,000 instances of peripheral blood mononuclear cell (PBMC) SCT data from four different sources, with the detection of 30,698 genes (dimensions). We pay special attention to the imbalanced nature of the data and consider feature selection methods that allow for an unbiased set of significant features to be obtained as a result. We show that statistical correlation-based feature selection, with some support from mutual information-based techniques, can result in a reasonably complex method for high-quality feature set selection.
Cited by 0
Martin Husemann, Norbert Ritter · Lecture notes in computer science · 2010
Cited by 1
Song Deng · 2015
In order to improve the efficiency of cultural information integration in the field of tourism, we propose a new strategy of data source selection based on the association between celebrities and mark words. Firstly, we acquire the sampling data based on character ontology, collect information in relation to the content of celebrity based on related themes, recognize the cultural theme of sentences based on the user's feedback, and finally extend character based on the project evaluation techniques. After the cultural summary is built, a data source scoring approach has been proposed based on the culture content gain of "single celebrity " and "synthetic character". We use domain datasets to do experiments. The results show that the accuracy of our method is higher and our method could provide effective support for characters and information integration of attractions.
Cited by 0Open access
Felix Naumann, Johann Christoph Freytag, Myra Spiliopoulou · 1998
Due to the development of the Internet there is an increasing number of information sources available to users. This makes it necessary to query only the most appropriate sources. The information quality offered by these sources can and must be a criterion for source selection. However, information quality has many dimensions, both subjective and objective, and it is thus difficult to directly compare sources with one another or give a ranking of sources. We propose to use Data Envelopment Analysis (DEA) to solve these shortcomings. This method does not directly compare sources, but focuses on individual sources, determining their efficiency in terms of information quality and cost. We adapt DEA to the task of source selection and discuss its advantages over other methods. Furthermore, we expand the model to include cost criteria and suggest a solution to additionally select sources in a query-dependent way.
Cited by 0
Soufyane El Allali, Daniel Blank, Wolfgang Müller et al. · Lecture notes in computer science · 2008
Cited by 6
Julio Cesar Cortes Rios · Research Explorer (The University of Manchester)
Cited by 0
Thierry Toutin, Danielle Hoja, E. Hoeppner et al. · 1998
GCP selection in mountainous relief is a difficult process. This paper shows methods using topographic, SAR and visible image data to define ridge points as GCPs. The SAR layover and shadow boundaries method is the best method but it depends of the back and fore slopes versus the look angle. Using SPOT ortho-images the results depend mainly on the visibility of the feature between the SAR and visible images. Simulated SAR is not a good solution since the DEM induces too large planimetric errors.
Cited by 2
Song Deng, Changxuan Wan, Xiping Liu · 2011
There are hundreds or thousands of web data sources providing data of relevance to a particular domain on the Web, so how to find a suitable set of sources quickly to integrate from a number of sources is becoming more and more important. Traditional data sources selection methods that are based on data quality do not consider quality characteristics of data sources under the different domains, but select the same quality dimensions for quality evaluation models. As a result, accuracies of a traditional method under different domains are quite different. In light of this, we propose a Web multimedia source selection method based on user feedback. We gain accurate core quality dimensions of data sources under the multimedia domain based on user feedback, and then achieve a quality estimation model of the multimedia data sources. The experiments on hundreds of real-world multimedia data sources have proved that our approach is efficient and accurate.
Cited by 0
Т. А. Воронина, V. V. Voronin · Geosystems of Transition Zones · 2023
The reconstruction of a tsunami source as a solution to the inverse problem in mathematical physics relies on the use of the truncated singular value decomposition method (a variant of the least squares method) for inverting remote records of the tsunami wave. The proposed method allows one to overcome the inevitable instability of the numerical solution. The result of inversion depends on the choice of the observation system, actual bathymetry and data noise level. Within the developed approach, a methodology for choosing key inversion options and an optimal dataset which provide the best accuracy of a tsunami source recovery is discussed. It is based on analyzing the distribution of the specific energy generated by all spatial modes at the locations of the active sensors. The peculiarity of the algorithm is that the use of the most informative data allows one without re-computation of the direct problem to obtain wave amplitudes at the points of interest (let us call them as “fictitious” stations) where there were no observations, but those that were considered in preliminary calculations. Three real-life events, the Chilean Illapel tsunami on September 16, 2015, the tsunami near the Solomon Islands on February 6, 2013, and the Shikotan tsunami on October 5, 1994, are used as examples of the proposed approach. The results obtained allow one hope for using of this approach in practice.
Cited by 1Open access
Kanyu Miyoshi, Ryotaro Shimizu, Linxin Song et al. · SSRN Electronic Journal · 2024
Cited by 0Open access
Elham Azhir, Negin Daneshpour, S. Salmani Ghanbari · Journal of artificial intelligence and data mining · 2016
Technology assessment and selection has a substantial impact on the organization procedures in regard to technology transfer. Technological decisions are usually made by a group of experts, and whereby, integrity of these viewpoints to a single decision can be quite complex. Today, operational databases and data warehouses exist to manage and organize data with specific features, and henceforth, the need for a decisionaid approach is essential. The process of developing data warehouses involves time-consuming steps, complex queries, slow query response rates, and limited functions, which is also true for operational databases. In this regard, fuzzy multi-criteria procedures used to choose efficient data sources (data warehouse and traditional relational databases) based on organization requirements are addressed in this paper. In proposing an appropriate selection framework, the paper compares a triangular fuzzy number (TFN)-based framework and a fuzzy analytical hierarchy process (AHP) based on data source models, business logic, data access, storage, and security. The results obtained show that the two procedures rank data sources in a similar manner and due to an accurate decision-making.
Cited by 2Open access
Kanyu Miyoshi, Ryotaro Shimizu, Lulin Song et al. · Knowledge-Based Systems · 2025
• Domain adaptation information gain-based target-related data selection is proposed. • Proposed DAIG improves target model accuracy via target-related data selection. • DAIG effectively gathers relevant data from source to benefit target tasks. • DAIG extracts “rough prior” from target data for source data pre-training. • DAIG-guided selection outperforms baselines in multiple experimental settings. In recent years, due to the explosive popularity of large-scale pre-trained models such as large language models, pre-training approaches that use a massive amount of source data and can be applied to various target tasks are becoming more popular. Pre-trained models allow us to learn highly accurate target models by fine-tuning them with target data, even when their volume is insufficient. However, the source data used to train a pre-training model is generally a large and miscellaneous data set obtained in the wild without being aware of the target task, and it highly possibly contains much data that does not contribute to relearning the target task. This study defines a novel paradigm as “target-aware source data selection,” which uses the source data itself instead of a pre-training model and selects source data for pre-training and aims to increase its quality, effectiveness, and robustness. Our proposal fundamentally differs from the current studies addressing the lack of target data and conventional transfer learning approaches, improving source data quality using the novel Domain Adaptation Information Gain criteria. Specifically, the target model is pre-trained while actively selecting only informative data from the source data using the “rough-prior knowledge” obtained from the target data training before the pre-training. Finally, fine-tuning the model with the target data results in a highly accurate model for the target (downstream) task. The effectiveness of our proposed paradigm has been demonstrated through multifaceted experiments using multiple pairs of target data and source data with different strengths of their relevance.
Cited by 0Open access
Yuyang Liu, Minchen Wei, Xinchao Qu et al. · Color and Imaging Conference · 2024
The colorimetric calibration of cameras are critical in imaging systems, with the sources used in light booths being widely used in practice. These sources, however, may not good presentations of the sources in real life, which possibly results in poor colors. In this study, we adopted a genetic algorithm and a large dataset of real light sources to identify an optimal set of sources that can better represent the sources in real life. The experiment results suggested that the identified set of sources can result in better color performance. Moreover, the selection of the sources was much less complicated in comparison to manual selections, which can be considered and implemented in practice.
Cited by 0Open access
Henry Assael, David F. Poltrack · Journal of Advertising Research · 1996
Cited by 2
Yiming Lin, Hongzhi Wang, Shuo Zhang et al. · Journal of Systems and Software · 2016
Cited by 12
Xuefeng Xian, Peng-Peng Zhao, Wei Fang et al. · 2009
Deep Web has been an important resource on the Web due to its rich and high quality information, leading to emerging a new application area in data mining and information retrieval and integrates. In Web scale deep Web data integration tasks, where there may be hundreds or thousands of data sources providing data of relevance to a particular domain, It must be inefficient to integrate all available deep Web sources. This paper proposes a data source selection approach based on the quality of deep Web source. It is used for automatic finding the highest quality set of deep Web sources related to a particular domain, which is a premise for effective deep Web data integration. The quality of data sources are assessed by evaluating quality dimensions represent the characteristics of deep Web source. Experiments running on real deep Web sources collected from the Internet show that our provides an effective and scalable solution for selecting data sources for deep Web data integration.
Cited by 6
XIAN Xuefeng,CUI Zhiming,FANG Ligang,GU Caidong,SUN Xun · SHILAP Revista de lepidopterología · 2017
Aiming at the problems that the data source based on the selection method of data source quality exists in selection process are heavy crawling price and high repetition rate,this paper proposes a two-layer selection model for source selection and integration. The selection model is built based on the quality and utility of the data source,and a recursive incremental data source selection and integration strategy is presented based on the model. The strategy adopts a data source quality classifier to filter majority low-grade Deep Web resources,only leaveing several high-quality ones as the input of the second layer utility classifier. The second layer classifier chooses the processed candidate resources recursively,which enables the integrated system to extract as much high qualified resources while escaping to get high coverage over k. Experimental results show that,combined the ascendency of two classifiers,the designed model can reduce the space of candidate data resources while assuring the quality,and it simultaneously avoids processing huge amounts of repeated data and reduces the integrated cost of Deep Web resources extraction effectively.
Cited by 0Open access