Data Integration Patterns for Data Warehouse Automation

被引：5
|
作者：
Tomingas, Kalle [1 ]
Kliimask, Margus [2 ]
Tammet, Tanel [1 ]
机构：
[1] Tallinn Univ Technol, EE-19086 Tallinn, Estonia
[2] Eliko Competence Ctr, EE-12618 Tallinn, Estonia
来源：
NEW TRENDS IN DATABASE AND INFORMATION SYSTEMS II | 2015年 / 312卷
关键词：
data warehouse; etl; data mappings; template based sql generation; abstract syntax patterns; metadata management;
D O I：
10.1007/978-3-319-10518-5_4
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The paper presents a mapping-based and metadata-driven modular data transformation framework designed to solve extract-transform-load (ETL) automation, impact analysis, data quality and integration problems in data warehouse environments. We introduce a declarative mapping formalization technique, an abstract expression pattern concept and a related template engine technology for flexible ETL code generation and execution. The feasibility and efficiency of the approach is demonstrated on the pattern detection and data lineage analysis case studies using large real life SQL corpuses.
引用
收藏
页码：41 / 55
页数：15
相关论文
共 50 条
  • [1] Data extraction and integration for data warehouse
    Xu, Li-Zhen
    Xie, Hong-Qiang
    Dong, Yi-Sheng
    2003, Shenyang Institute of Computing Technology (24):
  • [2] A Data Warehouse Approach to Semantic Integration of Pseudomonas Data
    Marrakchi, Kamar
    Briache, Abdelaali
    Kerzazi, Amine
    Navas-Delgado, Ismael
    Francisco Aldana-Montes, Jose
    Ettayebi, Mohamed
    Lairini, Khalid
    Rossi Hassani, Badr Din
    DATA INTEGRATION IN THE LIFE SCIENCES, 2010, 6254 : 90 - +
  • [3] A data warehouse to support web site automation
    Domingues, Marcos Aurélio
    Soares, Carlos
    Jorge, Alípio Mário
    Rezende, Solange Oliveira
    Journal of the Brazilian Computer Society, 2014, 20 (01) : 1 - 16
  • [4] On implementing the data warehouse -: GIS integration
    Matousek, K
    Mordacík, J
    Jankú, L
    WORLD MULTICONFERENCE ON SYSTEMICS, CYBERNETICS AND INFORMATICS, VOL 1, PROCEEDINGS: INFORMATION SYSTEMS DEVELOPMENT, 2001, : 206 - 210
  • [5] Integration and Automation of Data Preparation and Data Mining
    Narayanan, Shrikanth
    Jaiswal, Ayush
    Chiang, Yao-Yi
    Geng, Yanhui
    Knoblock, Craig A.
    Szekely, Pedro
    2014 IEEE INTERNATIONAL CONFERENCE ON DATA MINING WORKSHOP (ICDMW), 2014, : 1076 - 1085
  • [6] Semantic integration of medication data into the EHOP Clinical Data Warehouse
    Delamarre, Denis
    Bouzille, Guillaume
    Dalleau, Kevin
    Courtel, Denis
    Cuggia, Marc
    DIGITAL HEALTHCARE EMPOWERING EUROPEANS, 2015, 210 : 702 - 706
  • [7] Data Warehouse Oriented Data Integration System Design and Implementation
    Wang, Xiaoguo
    Shen, Jian
    Sun, Chuan
    MECHATRONICS AND INDUSTRIAL INFORMATICS, PTS 1-4, 2013, 321-324 : 2532 - 2538
  • [8] The data richness estimation framework for federated data warehouse integration
    Kern, Rafaf
    Kozierkiewicz, Adrianna
    Pietranik, Marcin
    INFORMATION SCIENCES, 2020, 513 : 397 - 411
  • [9] BioDWH: A Data Warehouse Kit for Life Science Data Integration
    Toepel, Thoralf
    Kormeier, Benjamin
    Klassen, Andreas
    Hofestaedt, Ralf
    JOURNAL OF INTEGRATIVE BIOINFORMATICS, 2008, 5 (02):
  • [10] The BioKET biodiversity data warehouse: Data and knowledge integration and extraction
    Inthasone, Somsack, 1600, Springer Verlag (8819):
12345