A Service-Oriented Framework for ETL Implementation

被引：2

作者：

Oliveira, Bruno ^{[1
]}

Leite, Mario ^{[1
]}

Oliveira, Oscar ^{[1
]}

Belo, Orlando ^{[2
]}

机构：

[1] Porto Polytech, Sch Management & Technol, CIICESI, Felgueiras, Portugal

[2] Univ Minho, ALGORITMI R&D Ctr, Campus Gualtar, Braga, Portugal

来源：

PROGRESS IN ARTIFICIAL INTELLIGENCE, EPIA 2022 | 2022年 / 13566卷

关键词：

Analytical systems; ETL patterns; Service-oriented approach; ETL as a service;

D O I：

10.1007/978-3-031-16474-3_52

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The development of analytical systems imposes several challenges related not only to the amount and heterogeneity of the involved data but also to the constant need to readapt and evolve to overcome new business challenges. The data modelling layer represents the mapping between the domain and technical knowledge, however, to organize raw data into a form that can be used for analytics, specific Extract, Transform and Load (ETL) processes should be applied. ETL systems are recognized as a critical and tightly coupled system component that encapsulates data-level requirements that are hard to implement and maintain. In a Big Data era, adaptability and extensibility are important characteristics to hold when developing analytical systems. Thus, to provide more consistent, reliable, flexible, and reusable ETL processes, a service-oriented implementation for ETL development is proposed.

引用

页码：636 / 647

页数：12

共 23 条

[11]

Lewis James, Microservices

[12] Graph Data Warehouse: Steps to Integrating Graph Databases into the Traditional Conceptual Structure of a Data Warehouse [J].

Liu, Yunkai ;

Vitolo, Theresa M. .

2013 IEEE INTERNATIONAL CONGRESS ON BIG DATA, 2013, :433-434

[13] DOD-ETL: distributed on-demand ETL for near real-time business intelligence [J].

Machado, Gustavo V. ;

Cunha, Italo ;

Pereira, Adriano C. M. ;

Oliveira, Leonardo B. .

JOURNAL OF INTERNET SERVICES AND APPLICATIONS, 2019, 10 (01)

[14]

Newman Sam, 2015, Building Microservices-Design FineGrained Systems

[15] ETL Development using Patterns: A Service-Oriented Approach [J].

Oliveira, Bruno ;

Oliveira, Oscar ;

Santos, Vasco ;

Belo, Orlando .

PROCEEDINGS OF THE 21ST INTERNATIONAL CONFERENCE ON ENTERPRISE INFORMATION SYSTEMS (ICEIS), VOL 1, 2019, :216-222

[16] On the specification of extract, transform, and load patterns behavior: A domain-specific language approach [J].

Oliveira, Bruno ;

Belo, Orlando .

EXPERT SYSTEMS, 2017, 34 (01)

[17] An Ontology for Describing ETL Patterns Behavior [J].

Oliveira, Bruno ;

Belo, Orlando .

DATA: PROCEEDINGS OF THE 5TH INTERNATIONAL CONFERENCE ON DATA MANAGEMENT TECHNOLOGIES AND APPLICATIONS, 2016, :102-109

[18] Foundry: a message-oriented, horizontally scalable ETL system for scientific data integration and enhancement [J].

Ozyurt, Ibrahim Burak ;

Grethe, Jeffrey S. .

DATABASE-THE JOURNAL OF BIOLOGICAL DATABASES AND CURATION, 2018,

[19]

Ralph Kimball e., 2013, The Data Warehouse Toolkit. The Complete Guide to Dimensional Modelling

[20] Towards a Concept for Building a Big Data Architecture with Microservices [J].

Shakir, Aamir ;

Staegemann, Daniel ;

Volk, Matthias ;

Jamous, Naoum ;

Turowski, Klaus .

24TH INTERNATIONAL CONFERENCE ON BUSINESS INFORMATION SYSTEMS (BIS): ENTERPRISE KNOWLEDGE AND DATA SPACES, 2021, :83-94

← 1 2 3 →