SCD Type2 in Talend Open Studio

SCD Type2 in Talend Open Studio:

Talend Open Studio for Data Integration Version: 6.3.1
Java Compiler: 1.7
OS: Windows 8

In the previous post I had demonstrated the mapping between oracle to oracle with simple transformation. In this post, I will try to implement SCD Type2. If you want to know the implementation in ODI then refer this post. We will be using the same set of tables as discussed in previous post.

Source Table: Employee
Target Table: Employee1.

Open Talend Open Studio. Right click on Job Design or demo folder and create a new job. Give the name as job_Oracle_SCD_Type2. Drag tOracleInput, tMap and tOracleSCD component. Either you search them in the palette or start typing directly in the design workspace, a drop down will be popped up immediately.


Configure the tOracleInput like we had done in previous post and connect it to tMap. If you dont see the source schema in tMap Map Editor, come back to tOracleInput properties and click on guess schema to refresh all the attributes. There are some cases where you will be asked to install some of the drivers which are missing and required for current operation. Go ahead and install them right away. Notice here that we dont need to bounce the studio as we used to do in case of ODI.


Now connect tMap to tOracleSCD and give the output name as OUTPUT. If it says to get the target schema then click on yes.


Now go back to the Map Editor again and see if the mapping is properly done or not.


Close this window and click on tOracleSCD component. In the properties click on SCD Editor.


Setup your parameters as per below screenshot. Drag the EMPNO to Source Keys, Name to type 2 fields and rest of the columns to Type 0. Leave the END_DATE, IS_ACTIVE, SG_KEY, START_DATE inside unused panel. Type the details manually in the versioning section.


Now we are done with configuration. Click ok and press F6 to execute the job. Verify the target table to see if all the columns are populated properly or not.


Since we have configured NAME field as type 2, we will do some changes in the source side. I will go ahead and modify KING to “KING KING” and  execute the same job again. Now notice the change.


Isn’t it pretty simple. If you personally ask me, I would say this is really really quick in terms of  configuration and setting up the parameters than ODI. I would recommend you to implement SCD type 3 in similar fashion and let me know if you are stuck.

Thats all for today.

Thank you!!!

About Bhabani 86 Articles
Bhabani has 12 plus years of experience in Data warehousing and Analytics projects that has span across multiple domains like Travel, Banking and Financial, Betting and Gaming Industries. Solution areas he focuses on designing the data warehouse and integrating it with cloud platforms like AWS or GCP. He is also a Elite level contributor at OTN forum more than 9 years. He loves to do experiment and POC on different integration tools and services. Some of his favorite skills are Redshift, Big Query, Python, Apache Airflow, Kafka, HDFS, Map Reduce ,HIVE, Habse, Sqoop, Drill, Impala.

1 Comment

Comments are closed.