Qlik Compose for DWH compared to Microsoft SSIS

Published on 21.04.2020
News
BI-Blog

An overview of the benefits of Qlik Compose for DWH compared to Microsoft SSIS

Qlik Compose DWH versus SSIS

 

Qlik Compose for DWH (Qlik Data Integration)

 

  1. With the Qlik Data Integration Platform, you are uniquely agile (by combining automated data loading and data processing/preparation) to quickly start with initial test cases and incrementally continue from there (DevOps / DataOps approach).
     
  2. By means of our Change-Data-Capture (CDC) approach we not only understand all data sources through permanent, consistent data flow, but can also automatically propose a data mapping in Compose or point out data mapping "breaks". Thereby the operative databases are not directly loaded (no-footprint architecture).
     
  3. Through the graphical, web-based interface, completely agile adjustments can be made to the data model or individual data marts at any time and, in combination with our versioning, tests can be rolled out and deployed, for example, and then transferred to the production environment if necessary.
     
  4. DDL changes of any kind are automatically recorded and event-based notifications are issued in Compose about structural changes and their impact on the data model or affected data marts - this factor is an immensely high added value, since the DWH is thus reliable and "always up to date" - it can be decided in Compose how to deal with the structural change (e.g. Dev - Test - Deploy in Prod within hours).
     
  5. Adding new data sources / requirements is reduced from weeks/days to hours/minutes and Qlik Compose helps to add new requirements to the model.
     
  6. No more scheduled jobs - we focus on setting up a data landing zone that is updated in real time (within seconds) and from there we process the data according to the use case/data mart / requirements using Compose and prepare it ready for analysis as required.
     
  7. With Replicate + Compose + Enterprise Manager from Qlik Data Integration, the customer gets a software from one source with a central "no footprint architecture" for seamless end-to-end management of all this data management via our central interface.

 

 

Die Qlik Data Integration Platform (Attunity)

as

The Qlik Data Integration Platform is a complete solution with all necessary functions for DataOps for Analytics. Attunity technology is the core of Qlik's advanced data integration capabilities.

To the Qlik Data Integration Platform

 

SQL SERVER INTEGRATION SERVICES (SSIS)

  1. Classic ETL approach for Analytics Architecture / DWH data flow requires a clearly defined requirements profile in advance, so that the requirements engineering effort is high.

  2. Especially when combining several different systems, such as ERP, PLM, CRM etc., a lot of data mapping takes place, which generates additional transformation effort.

  3. There is little room for agility in the data model or data mart structure, as many adaptations would require changes to the entire DWH and DB structure.

    • Example: A Finance Data Mart would access data from 2-3 different source systems and many of the ETL sets created for these systems would be affected by adjustments.

  4. Changes in the data structure (DDL) of the source systems, e.g. by adapting the application (also in the future - not only during implementation) result in the fact that many ETL routes have to be adapted (mostly manually) -> high follow-up costs.

  5. Adding new data sources / data sets or integrating new requirements often resembles a whole project cycle and generates a lot of waiting time in the business.

  6. This rather rigid data management is based on "scheduled jobs" (scheduled scripts that trigger data loading routes / transformation routes), which often run once per night - 3 challenges:

    • Often it is only noticed the next day that certain system pulls / loading cycles were not successfully performed, which leads to analysis problems / failures.

    • High monitoring effort, as errors can occur in many different ways within these often complex data loading processes.

    • No possibility to actively obtain data for time-critical business cases faster or to work with even more current data.

  7. Not only in the implementation but also in the operation of the DWH / architecture SSIS / the ETL approach offers only limited possibilities to manage, maintain and further develop the data model end-to-end via a central user friendly interface in order to keep data marts "always-up-to-date".

 

Microsoft SQL-Server Integration Services Data Flow

SSIS

 

Leave a comment

Plain text

  • No HTML tags allowed.
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.