Reducing ETL Load Times by a New Data Integration Approach for Real-time Business Intelligence
Research Area: | Volume 1 Issue 2, March 2012 | Year: | 2012 |
---|---|---|---|
Type of Publication: | Article | Keywords: | Business Intelligence, Dynamic Warehouse, Real-time Data Integration, Change Data Capture |
Authors: |
|
||
Journal: | IJEIR | Volume: | 1 |
Number: | 2 | Pages: | 56-60 |
Month: | March | ||
ISSN: | 2277-5668 | ||
Abstract: | Reducing business latency is essential in today’s
competitive and demanding environments. This means
responding immediately to new information as it arrives, and
having the right information in time to make the best
decision. Integrating, processing and delivering results in
real-time is a huge challenge, particularly as data volumes
continue to increase dramatically and sources of data are
ever more distributed and varied.
The decision making process in traditional data warehouse
environments is often delayed because data cannot be
propagated from the source system to the data warehouse in
time. The typical update patterns for traditional data
warehouses on an overnight or even weekly basis increase
this propagation delay. Keeping data current by minimizing
the latency from when data is captured until it is available to
decision makers in this context is a difficult task. A real-time
data warehouse aims at decreasing the time it takes to make
business decisions and tries to attain zero latency between the
cause and effect of a business decision.
An ETL process that periodically copies a snapshot of the
entire source consumes too much time and resources.
Alternate approaches that include timestamp columns,
triggers, or complex queries often hurt performance and
increase complexity. What is needed is a reliable stream of
change data that is structured so that it can easily be applied
by consumers to target representations of the data.
To keep up with the market competition, there is increased
need to minimize ETL load times. In this paper I have
introduced an approach for data integration that deals with
reducing ETL load times. |
||
Full text:
![]() |
Indexed By:
Our Journals
IJECCE International Journal of Electronics Communication and Computer Engineering ISSN(Online): 2249 - 071X ISSN (Print) : 2278 – 4209 www.ijecce.org Submissions open |
IJAIR International Journal of Agriculture Innovations and Research ISSN(Online) : 2319 – 1473 www.ijair.org Submissions open |
IJISM International Journal of Innovation in Science and Mathematics ISSN : 2347 – 9051 www.ijism.org Submissions open |
IJEIR International Journal of Engineering Innovations and Research ISSN(Online) : 2277 – 5668 www.ijeir.org Submissions are open. |
IJAIM International Journal of Artificial Intelligence and Mechatronics ISSN(Online) : 2320 – 5121 www.ijaim.org Submissions open |
IJRAS International Journal of Research in Agricultural Sciences ISSN(Online) : 2348 – 3997 www.ijras.org Submissions open |