María Vargas-Vera · International Journal of Knowledge Society Research · 2016
This paper presents a proposal for a data integration framework. The purpose of the framework is to locate automatically records of participants from the ALSPAC database (Avon Longitudinal Study of Parents and Children) within its counterpart GPRD database (General Practice Research Database). The ALSPAC database is a collection of data from children and parents from before birth to late puberty. This collection contains several variables of interest for clinical researchers but we concentrate in asthma as a golden standard for evaluation of asthma has been made by a clinical researcher. The main component of the framework is a module called Mapper which locates similar records and performs record linkage. The mapper contains a library of similarity measures such Jaccard, Jaro-Winkler, Monge-Elkan, MatchScore, Levenstein and TFIDF similarity. Finally, the author evaluates the approach on quality of the mappings.
Cited by 1
Noël Malod-Dognin (8677110), Gaia Ceddia (15353967), Maja Gvozdenov (15353970) et al. · Figshare · 2023
<p>We used the NMTF method to integrate four different data types: germline mutations of the five subjects presented in , protein-protein interactions (PPIs) from BioGRID presented in , coexpressions (COEXs) from COXPRESdb presented in and genetic interactions (GIs) from [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0284084#pone.0284084.ref014" target="_blank">14</a>] presented in . <i>M</i> is simultaneously decomposed into the product of three lower dimensional matrix factors, <i>P</i>, <i>S</i> and <i>G</i><sup><i>T</i></sup>; molecular networks, <i>R</i><sub><i>i</i></sub>, are simultaneously decomposed into the product of three factors, <i>G</i>, <i>U</i><sub><i>i</i></sub> and <i>G</i><sup><i>T</i></sup>, as detailed in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0284084#sec002" target="_blank">Materials and methods</a>. <i>k</i><sub>1</sub> and <i>k</i><sub>2</sub> are the gene and the patient clusters, respectively.</p>
Cited by 0Open access
Łukasz Faber · 2016
Abstract Combining data from diverse, heterogeneous sources while facilitating a unified access to it is an important (albeit difficult) task. There are various possibilities of performing it. In this publication, we propose and describe an agent-based framework dedicated to acquiring and processing distributed, heterogeneous data collected from diverse sources (e.g., the Internet, external software, re-lational, and document databases). Using this multi-agent-based approach in the aspects of the general architecture (the organization and management of the framework), we create a proof-of-concept implementation. The approach is presented using a sample scenario in which the system is used to search for personal and professional profiles of scientists.
Cited by 0
Kyaw, Ye Lin. · DR-NTU (Nanyang Technological University) · 2010
A data warehouse primarily provides intelligent business facilities such as analytical processing, decision making and data mining. Design and implementation of data warehouses has been well studied and supported by many commercial offerings. However increasing demands for real-time systems fuels the need for changes in existing data warehouse designs and frameworks. One such change required to realise a real-time approach is to redefine the process of synchronization between transactional data and the data warehouse. Current loading processes are typically carried out by using the methods of insert, batch or bulk loading which can deteriorate the real-time performance. Though bulk loading is faster, it needs additional preprocessing of the data, which is not cost effective particularly when transferring large amounts of data. It is thought that real-time data integration can improve the process of frequently loading small amounts of data. This dissertation discusses the performance and functionalities of existing data integration tools along with their associated problems and drawbacks. Current trends in real-time data integration are also analysed and a new real-time data integration framework is introduced. Implementations of the well known real-time data integration method are also presented, as are possible extensions for implementing or introducing the new real-time data integration implementation method to address other problems. An in-depth empirical analysis demonstrates the validity of the proposed approach.
Cited by 0
Maurício Mello Codesso, Paulo Caetano da Silva, Miklos A. Vasarhelyi et al. · Revista Contemporânea de Contabilidade · 2018
A aproximação de áreas de negócio com o uso de novas tecnologias, economia em tempo real, transações com vários países e em vários continentes com diferentes garantias legais são necessárias. Essas garantias podem ser adquiridas por meio da Auditoria Contínua (CA). No entanto, para poder realizar a análise, os auditores precisam ter acesso e extrair os dados. Pesquisadores anteriores enfatizam apenas os benefícios da aplicação dos métodos de AC, mas não explicam como recuperar e organizar os dados. Desta forma, propomos o desenvolvimento de um framework para integrar diferentes sistemas de auditoria contínua. O artigo visa contribuir para a literatura com o aprofundamento das formas de acessar, estruturar e coletar dados críticos e necessários para a AC. Com o aprofundamento do Audit Data Standard e do eXtensible Business Reporting Language (XBRL), além de criar uma base para pesquisas futuras com a integração de algoritmos de extração, análise e detecção de exceção que são utilizados pela CA.
Cited by 3Open access
Lindfors, Erno, Dam, Jesse Van, Lam, Carolyn et al. · Socio-Environmental Systems Modeling · 2018
Abstract Background Systems biology takes a holistic approach by handling biomolecules and their interactions as big systems. Network based approach has emerged as a natural way to model these systems with the idea of representing biomolecules as nodes and their interactions as edges. Very often the input data come from various sorts of omics analyses. Those resulting networks sometimes describe a wide range of aspects, for example different experiment conditions, species, tissue types, stimulating factors, mutants, or simply distinct interaction features of the same network produced by different algorithms. For these scenarios, synchronous visualization of more than one distinct network is an excellent mean to explore all the relevant networks efficiently. In addition, complementary analysis methods are needed and they should work in a workflow manner in order to gain maximal biological insights. Results In order to address the aforementioned needs, we have developed a Synchronous Network Data Integration (SyNDI) framework. This framework contains SyncVis, a Cytoscape application for user-friendly synchronous and simultaneous visualization of multiple biological networks, and it is seamlessly integrated with other bioinformatics tools via the Galaxy platform. We demonstrated the functionality and usability of the framework with three biological examples - we analyzed the distinct connectivity of plasma metabolites in networks associated with high or low latent cardiovascular disease risk; deeper insights were obtained from a few similar inflammatory response pathways in Staphylococcus aureus infection common to human and mouse; and regulatory motifs which have not been reported associated with transcriptional adaptations of Mycobacterium tuberculosis were identified. Conclusions Our SyNDI framework couples synchronous network visualization seamlessly with additional bioinformatics tools. The user can easily tailor the framework for his/her needs by adding new tools and datasets to the Galaxy platform.
Cited by 0Open access
Ward, Roger, Pratt, Nicole, Boyle, Dougie et al. · Zenodo (CERN European Organization for Nuclear Research) · 2024
This is a Consultation Draft of the Australian Research Data Commons (ARDC) Medical Research Data Integration Framework. We welcome high-level feedback on key themes. In this document, the ARDC proposes a framework for the systematic integration and use of data from electronic medical records (EMRs) across Australia for research and analysis. At the core of this proposal is the adoption of the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM), an internationally recognised standard that can harmonise health data across different EMR and health care systems. By implementing the OMOP CDM, this framework aims to standardise EMR data, enhance interoperability, and support local, national, and international collaborative research projects. Such a capability would also provide significant internal analytics and business intelligence opportunities for health service providers and policymakers. We also advocate for the establishment of an Australian Health Data to Evidence Network to coordinate the adoption and implementation of the OMOP CDM across Australia, initially focused on EMR data. This document outlines key governance, security and privacy considerations to ensure the framework’s successful integration into existing health data systems to create an enhanced infrastructure for health data research. Learn more about our Data Integration activity, a part of our People Research Data Commons for health and medical research.
Cited by 0Open access
Mauricio Mello Codesso, Paulo Caetano da Silva, Miklos A. Vasarhelyi et al. · 2018
The approximation of business areas with the use of new technologies, real-time savings, transactions with several countries and on several continents with different law guarantees are necessary. These warranties can be acquired through Continuous Audit (CA). However, to be able to perform the analysis the auditors need to have access and extract the data. Previous researchers only emphasis on the benefits of applying the CA methods, but do not explain how to retrieve and organize the data. The paper aims to contribute to the literature with the deepening of ways to access, structure and collect critical and necessary data for CA. With the deepening of Audit Data Standard and XBRL, as well as creating a basis for future research with the integration of extraction, analysis, and exception detection algorithms that are used by CA.
Cited by 0
Erno Lindfors, Jesse C. J. van Dam, Carolyn Lam et al. · BMC Bioinformatics · 2018
Our SyNDI framework couples synchronous network visualization seamlessly with additional bioinformatics tools. The user can easily tailor the framework for his/her needs by adding new tools and datasets to the Galaxy platform.
Cited by 3Open access
Jianing Wang · Lecture notes in computer science · 2012
Cited by 5
Lindfors, Erno, Dam, Jesse Van, Lam, Carolyn et al. · Figshare · 2018
Abstract Background Systems biology takes a holistic approach by handling biomolecules and their interactions as big systems. Network based approach has emerged as a natural way to model these systems with the idea of representing biomolecules as nodes and their interactions as edges. Very often the input data come from various sorts of omics analyses. Those resulting networks sometimes describe a wide range of aspects, for example different experiment conditions, species, tissue types, stimulating factors, mutants, or simply distinct interaction features of the same network produced by different algorithms. For these scenarios, synchronous visualization of more than one distinct network is an excellent mean to explore all the relevant networks efficiently. In addition, complementary analysis methods are needed and they should work in a workflow manner in order to gain maximal biological insights. Results In order to address the aforementioned needs, we have developed a Synchronous Network Data Integration (SyNDI) framework. This framework contains SyncVis, a Cytoscape application for user-friendly synchronous and simultaneous visualization of multiple biological networks, and it is seamlessly integrated with other bioinformatics tools via the Galaxy platform. We demonstrated the functionality and usability of the framework with three biological examples - we analyzed the distinct connectivity of plasma metabolites in networks associated with high or low latent cardiovascular disease risk; deeper insights were obtained from a few similar inflammatory response pathways in Staphylococcus aureus infection common to human and mouse; and regulatory motifs which have not been reported associated with transcriptional adaptations of Mycobacterium tuberculosis were identified. Conclusions Our SyNDI framework couples synchronous network visualization seamlessly with additional bioinformatics tools. The user can easily tailor the framework for his/her needs by adding new tools and datasets to the Galaxy platform.
Cited by 0Open access
Ward, Roger, Pratt, Nicole, Boyle, Dougie et al. · Zenodo (CERN European Organization for Nuclear Research) · 2024
This is a Consultation Draft of the Australian Research Data Commons (ARDC) Medical Research Data Integration Framework. We welcome high-level feedback on key themes. In this document, the ARDC proposes a framework for the systematic integration and use of data from electronic medical records (EMRs) across Australia for research and analysis. At the core of this proposal is the adoption of the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM), an internationally recognised standard that can harmonise health data across different EMR and health care systems. By implementing the OMOP CDM, this framework aims to standardise EMR data, enhance interoperability, and support local, national, and international collaborative research projects. Such a capability would also provide significant internal analytics and business intelligence opportunities for health service providers and policymakers. We also advocate for the establishment of an Australian Health Data to Evidence Network to coordinate the adoption and implementation of the OMOP CDM across Australia, initially focused on EMR data. This document outlines key governance, security and privacy considerations to ensure the framework’s successful integration into existing health data systems to create an enhanced infrastructure for health data research. Learn more about our Data Integration activity, a part of our People Research Data Commons for health and medical research.
Cited by 0Open access
Paul Chandler · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2024
Cited by 0Open access
Benjamin Besimi, Jaumin Ajdari, Xhemal Zenuni · 2024
In an era where cloud computing is becoming essential, organizations are actively seeking to build cloud-based data solutions or migrate existing on-premises systems to the cloud. This shift is driven by the need to benefit from the scalability, flexibility, and efficiency offered by cloud environments. However, this transition presents a complex challenge: the integration of cloud data practices that are universally applicable yet sufficiently adaptable to domain specific needs. A significant number of cloud data initiatives fail due to poor planning and the absence of appropriate methodologies for effective cloud data integration. This failure often results in issues related to the completeness, accuracy, and consistency of data, which are essential for building reliable and trustworthy data platforms. This paper addresses this challenge by proposing a cloud-agnostic framework for cloud data integration. The paper’s contribution is a comprehensive, adaptable metadata-driven data integration framework that is flexible to be tailored to specific domain requirements and scalable for various cloud service providers. The proposed framework is designed to guide and assist organizations in their cloud data integration efforts, enabling them to focus on deriving organizational value.
Cited by 1
Di Wang, Wei Liu, Xin Nie et al. · Array · 2026
As multimodal information becomes central to complex task understanding, efficiently integrating heterogeneous data remains a challenge. This paper proposes a knowledge-driven hybrid fusion framework that enhances model classification performance by integrating the modeling of images and text. We designed an adaptive gating network to dynamically balance the contributions of visual and textual modalities, enabling flexible and robust knowledge integration across diverse clinical scenarios. The framework incorporates a hybrid fusion strategy, early fusion captures fine-grained semantic associations across modalities, while late fusion preserves modality-specific representations, enhancing model consistency and discriminative capabilities under multi-task optimization. Experimental results on multiple multimodal medical datasets validate the effectiveness and generalizability of the proposed hybrid fusion framework in clinical scenarios. The framework’s near-clinical accuracy in lung disease and cancer classification highlights its potential for clinical decision support. • We propose a local interaction attention mechanism that aligns semantic cues from clinical text with corresponding regions in chest X-ray images, enabling fine-grained diagnostic knowledge modeling. • We design an adaptive gating network that dynamically balances the contributions of textual and visual modalities, allowing flexible knowledge fusion and improving decision-support robustness under varying clinical scenarios. • This is an early research that constructs a hybrid fusion architecture employs early fusion for cross-modal semantic interaction and combines late fusion to preserve modality-specific knowledge. By incorporating a contrastive learning mechanism, enabling the model to simultaneously optimize classification tasks and contrastive tasks within a multi-task learning framework. This approach achieves more accurate, reliable, and comprehensive decision support.
Cited by 0Open access
Wayne Routly, Dalenca Pottas · 2008
The new millennium has brought about an increase in the use of business intelligence and knowledge management systems. The very foundations of these systems are the multitude of source databases that store the data. The ability to derive information from these databases is brought about by means of data integration. With the current emphasis on security in all walks of information and communication technology, a renewed interest must be placed in the systems that provide us with information; data integration systems. In the database environment we are concerned with the database itself and the media used to connect to and from the database. In distributed data integration, the concept of the database is redefined to the source database, from which we extract data and the storage database in which the integrated data is stored. This postulates three distinct areas in which to apply security, the data source, the network medium and the data store. All of these areas encompass data integration and must be considered holistically when implementing security. Data integration is never only one server or one database; it is various geographically dispersed components working together towards a common goal. It is important then that we consider all aspects involved when attempting to provide security for data integration. This paper investigates security issues in the data integration cycle, with special reference to problems when performing data integration in a peer-to-peer environment, as in distributed data integration.
Cited by 0
Pfeifer B, Seger M, Netzer M et al. · 2012
In this work a cellular automaton software package for simulating different infectious diseases, storing the simulation results in a data warehouse system, and analyzing the obtained results in order to generate predictions models or contingency plans is proposed. The Brisbane H3N2 flu virus, which has been spreading this winter season, was used for spreading simulation in federal state of Tyrol. The simulation-modeling framework consists of an underlying cellular automaton. The generated data are stored in the back room of the data warehouse using the Talend Open Studio software package, and subsequent statistical and data mining tasks are performed using the Knowledge Discovery in Database Designer (KD 3). The obtained simulation results were used for generating prediction models for all nine federal States of Austria, and were compared to flu outbreaks in the past. 1.
Cited by 0
Tran Khanh Dang, Manh Huy Ta, Le Hoang Nguyen · Communications in computer and information science · 2020
Cited by 6
DE ANTONELLIS, Valeria, DE CAPITANI DI VIMERCATI, Sabrina, MELCHIORI, Michele et al. · Institutional Research Information System (Università degli Studi di Brescia) · 2000
Cited by 0
Maurício A. Mudadu, Adhemar Zerlotini · bioRxiv (Cold Spring Harbor Laboratory) · 2020
ABSTRACT Background Genome projects and multiomics experiments generate huge volumes of data that must be stored, mined and transformed into useful knowledge. All this information is supposed to be accessible and, if possible, browsable afterwards. Computational biologists have been dealing with this scenario for over a decade and have been implementing software libraries, toolkits, platforms, and databases to succeed in this matter. The GMOD’s (Generic Model Organism Database project) biological relational database schema, known as Chado , is one of the few successful open source initiatives, it is widely adopted and many softwares are able to connect to it. Results We have been developing an open source software named Machado ( https://github.com/lmb-embrapa/machado ), a genomics data integration framework implemented in Python , to enable research groups to both store and browse, query, and visualize genomics data. The framework relies on the Chado database schema and, therefore, should be very intuitive for current developers to adopt it or have it running on the top of already existing databases. It has several data loading tools for genomics and transcriptomics data and also for annotation results from tools such as BLAST , InterproScan , OrthoMCL and LSTrAP . There is an API to connect to JBrowse and a web browsing visualisation tool is implemented using Django Views and Templates . The Haystack library integrated with the ElasticSearch engine was used to implement a google-like search i.e. single auto-complete search box that provides fast results and incremental filters. Conclusion Machado aims to be a modern object-relational framework that uses the latests Python libraries to produce an effective open source resource for genomics research.
Cited by 1Open access