Verify Cloning Table Contents via Delta Load

 Posted on 2021-12-04  |  3 minutes  |  457 words  |  Kazuhiko Takata

Following last week’s Minimize Extraction Time by Delta Load Option, today I would like to insert new record to Postgres table then try Delta Load again to extract it. To do this, I will start from operating pgAdmin, that is already ready for my loal machine after docker-compose. First step is to enter localhost:5050 to my browser, then at the login screen enter pgadmin@celonis.cloud as email and pgadmin as password then click login button. [Read More]
Data Integration  Extractor  Extraction  Transformation  Data Job 

Minimize Extraction Time by Delta Load Option

 Posted on 2021-11-27  |  3 minutes  |  454 words  |  Kazuhiko Takata

Last week I extracted Postgres table and looked at the log to understand mechanism of data transfer. At that time I used Full Load option to extract data, that is to replace all table contents and schema to latest version. That is easiest way to synchronize tables between source system (Postgres) and Celonis, but it takes a lot of time to complete this task. So that I should also use second option Delta Load to minimize extraction time. [Read More]
Data Integration  Extractor  Extraction  Transformation  Data Job 

Look at Data Transfer Process by Data Job Log

 Posted on 2021-11-20  |  3 minutes  |  607 words  |  Kazuhiko Takata

Last week I posted Connect to Celonis and Bring Back Instruction to look at how Extractor works to connect between Celonis and Postgres. This week I would like to extract data from Postgres and look at data transfer process by data job log. In the Data Integration, I create new Data Job with Data Connection I created last week, then create new extraction task. In the next screen I add new table public. [Read More]
Data Integration  Extractor  Extraction  Data Job 

Connect to Celonis and Bring Back Instruction

 Posted on 2021-11-13  |  3 minutes  |  433 words  |  Kazuhiko Takata

From last week I started Data Integration series and posted Run Extractor on Your Local Machine to prepare for my Extractor and Postgres database. Today I will start using Extractor and show you the mechanism to extract data safely. From this week I will start Extractor and Postgres by next two steps (same as last week). Open VS code and open terminal at celonis-postgres folder. Enter docker-compose up in VS code terminal. [Read More]
Data Integration  Extractor 

Run Extractor on Your Local Machine

 Posted on 2021-11-06  |  3 minutes  |  457 words  |  Kazuhiko Takata

From this week I would like to explain my experience regarding Data Integration functions (Extraction, Transformation, Load etc.). To do this, I try to create sample source systems and build code in Celonis training environment. As first topic, I would like to explain on premise Extractor, that is in the middle between your source systems (SAP, Oracle etc.) and Celonis EMS and support transferring data. By the way, because I do not want to pay licence of source systems for this blog, I would like to use open source Postgres database. [Read More]
Data Integration  Extractor 

Categorize and Name Activity

 Posted on 2021-07-24  |  3 minutes  |  471 words  |  Kazuhiko Takata

In the previous post Transform Source System Tables to Minimize Data Model Tables, I recommended to convert some kind of source system tables to Activity. Today I focus on Activity and would like to share my way how to categorize and name Activity. First point is to split Activity name to two parts, more general part and detail part. For example, in SAP ECC or S4HANA Order to Cash process, general Activity name is Create Sales Order when data committed in VA01 transaction. [Read More]
Data Model  Data Integration  Transformation  PQL  Process Analytics  Studio  Analysis  Pull Up Function  Process Explorer 

Transform Source System Tables to Minimize Data Model Tables

 Posted on 2021-07-17  |  2 minutes  |  338 words  |  Kazuhiko Takata

In the last part of previous post Utilize N-M relationship between Activity and Dimension Tables, I said just mimicing source system’s table structure to data model is not useful for Celonis data model. Also I feel loading time to data model is exponentially long when one more table is added. So minimizing data model table is good practice especially for real time analytics. In case of SAP Order to Cash (O2C) scenario, case table is sales order item (VBAP). [Read More]
Data Model  Data Integration  Transformation