Data Integration Patterns for Data Warehouse Automation

被引：5
|
作者：
Tomingas, Kalle [1 ]
Kliimask, Margus [2 ]
Tammet, Tanel [1 ]
机构：
[1] Tallinn Univ Technol, EE-19086 Tallinn, Estonia
[2] Eliko Competence Ctr, EE-12618 Tallinn, Estonia
来源：
NEW TRENDS IN DATABASE AND INFORMATION SYSTEMS II | 2015年 / 312卷
关键词：
data warehouse; etl; data mappings; template based sql generation; abstract syntax patterns; metadata management;
D O I：
10.1007/978-3-319-10518-5_4
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
The paper presents a mapping-based and metadata-driven modular data transformation framework designed to solve extract-transform-load (ETL) automation, impact analysis, data quality and integration problems in data warehouse environments. We introduce a declarative mapping formalization technique, an abstract expression pattern concept and a related template engine technology for flexible ETL code generation and execution. The feasibility and efficiency of the approach is demonstrated on the pattern detection and data lineage analysis case studies using large real life SQL corpuses.
引用
收藏
页码：41 / 55
页数：15
相关论文
共 50 条
  • [41] MODELING AND INTEGRATION OF INTENSIVE CARE DATA INTO AN openEHR-BASED ENTERPRISE DATA WAREHOUSE
    Haarbrandt, B.
    Marschollek, M.
    CURRENT THERAPEUTIC RESEARCH-CLINICAL AND EXPERIMENTAL, 2016, 78 : S8 - S9
  • [42] Integration of Decentralized Clinical Data in a Data Warehouse A Service-oriented Design and Realization
    Hanss, S.
    Schaaf, T.
    Wetzel, T.
    Hahn, C.
    Schrader, T.
    Tolxdorff, T.
    METHODS OF INFORMATION IN MEDICINE, 2009, 48 (05) : 414 - 418
  • [43] Data Warehouse and Data Virtualization
    Mousa, Ayad Hameed
    Shiratuddin, Norshuhada
    PROCEEDINGS 2015 INTERNATIONAL CONFERENCE ON DEVELOPMENTS IN ESYSTEMS ENGINEERING DESE 2015, 2015, : 369 - 372
  • [44] The data warehouse and data mining
    Inmon, WH
    COMMUNICATIONS OF THE ACM, 1996, 39 (11) : 49 - 50
  • [45] The data warehouse
    Donaldson, WR
    AM/FM INTERNATIONAL CONFERENCE XX, PROCEEDINGS - ENTERING THE MAINSTREAM, 1997, : 21 - 29
  • [46] Cost Effective Framework for Complex and Heterogeneous Data Integration in Warehouse
    Amuthabala, P.
    Mohanapriya, M.
    SOFTWARE ENGINEERING PERSPECTIVES AND APPLICATION IN INTELLIGENT SYSTEMS, VOL 2, 2016, 465 : 93 - 104
  • [47] Metadata integration of engineering data warehouse system based on metamodel
    Zhao, Xiaofei
    Huang, Zhiqiu
    Shen, Guohua
    Yang, Tao
    Nanjing Hangkong Hangtian Daxue Xuebao/Journal of Nanjing University of Aeronautics and Astronautics, 2006, 38 (03): : 341 - 346
  • [48] Data Warehouse
    Peter Gluchowski
    Informatik-Spektrum, 1997, 20 (1) : 48 - 49
  • [49] From Traditional Data Warehouse To Real Time Data Warehouse
    Bouaziz, Senda
    Nabli, Ahlem
    Gargouri, Faiez
    INTELLIGENT SYSTEMS DESIGN AND APPLICATIONS (ISDA 2016), 2017, 557 : 467 - 477
  • [50] TOWARDS MODEL-DRIVEN DATA WAREHOUSE AUTOMATION USING MACHINE LEARNING
    Essaidi, Moez
    Osmani, Aomar
    ICEC 2010: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON EVOLUTIONARY COMPUTATION, 2010, : 380 - 383
12345