Module 6: Creating Batch Job

Module Objectives

After completing this unit , you should be able to understand about:

–Create a Project

–Creating a Job

–Explain source & target objects

–Explain what a transform is

–Understand the Query transform

–Understand Job Execution

INTRODUCTION

• Create a project

• Create a job

• Create Data Flow

• Add, connect and delete objects in workspace

• Using Query Transforms

CREATE DATA FLOW

SOURCE OBJECTS & TARGET OBJECTS

•Define sources from which to read & write data

–Table

–Template table

–File

–Document

–XML file

–XML message

TRANSFORM AT GLANCE

•Transform manipulate input sets and produce one or

more output sets

•Sometimes transforms such as Date_generation and SQL transform can also be used as source objects.

•Use operation codes with transforms to indicate how each row in the data set is applied to target table

•Most commonly used transform is Query transform

Data Integrator Transform

Data Quality Transform

Plat form (Commonly used in DQ & DI) Transform

UNDERSTANDING QUERY TRANSFORM

• Query transform can perform following operations:

–Choose (filter) the data to extract from sources

–Join data from multiple sources

–Map columns from input to output schemas

–Perform transformations and functions on data

–Add new columns to output schema

–Assign primary key to output columns

QUERY EDITOR WINDOW

UNDERSTANDING TARGET TABLE EDITOR

• Target table loader offers different tabs where you can set database type properties, different table loading options, and use different tuning techniques for loading a job

USING DESCRIPTIONS & ANNOTATIONS

• Use description with objects

–Add description to an object :

Designer determines when to show objects descriptions based

on system level setting and an object level setting

–Display a description in the workspace

Display the object by right click the object in the workspace and select View Enabled Descriptions.

• Use annotations to describe job, work and data flows

–Add annotations to job, work flow, data flow or diagram

in the workspace.

USING VIEW DATA AND THE INTERACTIVE DEBUGGER

•Sources

View data allows you to see source data before you execute a job.

Using data details you can:

–Create higher quality job designs

–Scan and analyse imported table and file data from object library

–See the data for those same objects with in existing jobs.

–Refer back to the source data after you execute the job.

•Targets

View data allows you to check target data before you executing a job, then look at the changed data after the job executes. In a data flow, you can use one or more View data panels to compared at a between transforms and within source and target objects

TEMPLATE TABLES

•During the initial design of an application, you might find it convenient to use template tables to represent data base tables. With template tables, you do not have to initially create a new table in your DBMS and import the meta data into Data Integrator. Instead, Data Integrator automatically creates the table in the data base with the schema defined by the data flow when you execute a job. After creating a template table as a target in one data flow, you can use it as a source in other data flows. Though a template table can be used as a source table in multiple data flows, it can only be used as a target in one data flow.

•Template tables are particularly useful in early application development when you are designing and testing a project.

•When the job is executed, Data Integrator uses the template table to create a new table in the data base you specified when you created the template table. Once a template able is created in the data base, you can convert the template table in the repository to a regular table.

•Once a template table Is converted, you can no longer alter the schema.

+VV Satyanarayana G 

+Sairam Jalluru 

Print this post