发布时间 : 星期日 文章ETL开发指南(DataStage EE)V2.0更新完毕开始阅读19b0c57da26925c52cc5bf0a
DataStage Enterprise
Edition
开发指南v2.0
DataStage EE开发指南
目 录
目 录 .................................................................................................................................................. I 1. 引言 ..................................................................................................................................................... 1 1.1 编写目的 ....................................................................................................................................... 1 1.2 帮助使用 ....................................................................................................................................... 1 2. 产品概述 ............................................................................................................................................. 2 3. 常规应用 ............................................................................................................................................. 3 3.1 常用组件使用方法 ....................................................................................................................... 3 3.1.1 Sequential file .......................................................................................................................... 3 3.1.2 Annotation ............................................................................................................................... 7 3.1.3 Change Capture Stage ............................................................................................................. 8 3.1.4 Copy Stage ............................................................................................................................ 10 3.1.5 Filter Stage ............................................................................................................................ 11 3.1.6 Funnel Stage .......................................................................................................................... 12 3.1.7 Tansformer Stage .................................................................................................................. 13 3.1.8 Sort Stage .............................................................................................................................. 14 3.1.9 LookUp Stage ........................................................................................................................ 15 3.1.10 Join Stage ............................................................................................................................ 16 3.1.11 LookUp Stage 和 Join Stage的区别 ................................................................................. 17 3.1.12 Merge Stage ........................................................................................................................ 18 3.1.13 Modify Stage ....................................................................................................................... 19 3.1.14 Data Set Stage ..................................................................................................................... 20 3.1.15 File Set Stage ....................................................................................................................... 22 3.1.16 Lookup File Set Stage ......................................................................................................... 23 3.1.17 Oracle Enterprise Stage ....................................................................................................... 26 3.1.18 Aggregator Stage ................................................................................................................. 28 3.1.19 Remove Duplicates Stage .................................................................................................... 30 3.1.20 Compress Stage ................................................................................................................... 31 3.1.21 Expand Stage ....................................................................................................................... 32 3.1.22 Difference Stage .................................................................................................................. 33 3.1.23 Compare Stage .................................................................................................................... 36 3.1.24 Switch Stage ........................................................................................................................ 37 3.1.25 Column Import Stage .......................................................................................................... 39 3.1.26 Column Export Stage .......................................................................................................... 41 3.1.27 Teradata Enterprise Stage .................................................................................................... 43 3.2 常用数据库的连接 ..................................................................................................................... 45 3.2.1 Informix数据库连接 ............................................................................................................. 45 3.2.2 Oracle数据库连接 ................................................................................................................ 46 4. 高级应用 ........................................................................................................................................... 48 4.1 DATASTAGE BASIC接口 ............................................................................................................. 48 4.2 自定义STAGE TYPE ..................................................................................................................... 49 4.2.1 Wrapped Stage ...................................................................................................................... 49 4.2.2 Build Stage ............................................................................................................................ 49 4.2.3 Custom Stage ........................................................................................................................ 49 4.3 性能调优 ..................................................................................................................................... 49 4.3.1 优化策略 .............................................................................................................................. 49 4.3.2 关键问题分析 ...................................................................................................................... 54
DataStage EE开发指南
4.3.3 并行度 .................................................................................................................................. 54 4.3.4 处理建议 .............................................................................................................................. 55 4.3.5 其它 ...................................................................................................................................... 56 4.3.6 机器的对称性 ...................................................................................................................... 56 4.3.7 并行调度测试说明： .......................................................................................................... 56 5. 开发经验技巧汇总 ........................................................................................................................... 57
5.1.1 问题 ...................................................................................................................................... 57
DataStage EE开发指南
1. 引言
1.1 编写目的
IBM DataStage作ODS项目所使用的主要开发工具，在项目中得到了比较充分的应用，对IMB DataStage产品方方面面的功能，都有所涉及。作为对我们使用本产品的一个技术沉淀，我们编写了这个总结性质的手册，旨在综合描述该产品的安装、常规应用、高级开发等等项目所涉及到的各个方面。为项目组内部或其他使用该产品的项目组提供一个全面而综合的产品操作指导，帮助使用者能够在最短的时间内了解该产品，并上手使用。
1.2 帮助使用
由于DataStage产品功能强大，配置复杂，我们不可能在本手册中涉及到所有功能，也无法对每一个描述的能够都给出详尽的例子。能够为阅读者穿针引线的整理产品的开发思路，通过对常用、基本的功能的描述，让大家掌握到该产品的使用精髓，并能够举一反三的掌握其它本手册没有介绍的功能。这就是我们的目的。
我们通常使用的帮助有如下两个途径。
a. 智能化的帮助功能；产品在几乎所有的操作窗口都有一个Help键，点击该键可以显示出当前使用的界面的功能和各项选项的具体的说明和操作方法。DataStage独有的智能化的帮助能够指引你方面的查找到你想要得到的帮助。
b. Online Manuals；就是产品安装后程序组中的DataStage Documents，里面更加综合、全面的对整个产品从普通到高级，从Server版到Enterprise Edition版，从For Windows到For Unix等等方面的详尽叙述。能够帮助更加系统、全面的掌握该产品。