Informatica映射的性能调整

问题描述 投票:0回答:1

我必须调整informatica映射性能。源和目标是sql server表。映射包含SQ断开连接的lookup-exp-target。源表和目标表相同。查找查询在数据库中耗时7-8分钟,并且根据会话日志,查找缓存的创建也需要时间。我可以看到,创建查找缓存需要花费一些时间,但我无法获得确切的问题。

您能否建议如何调整此会话日志或查找查询

下面是会话日志:-

Severity    Timestamp   Node    Thread  Message Code    Message
INFO    1/2/2020 2:38:17 AM WRITER_1_*_1    WRT_8158    

*****START LOAD SESSION*****

Load Start Time: Thu Jan 02 02:38:17 2020

Target tables:

     Dim_Cust_Ord


INFO    1/2/2020 2:38:17 AM WRITER_1_*_1    WRT_8005    Writer run started.
INFO    1/2/2020 2:38:17 AM READER_1_1_1    RR_4049 RR_4049 SQL Query issued to database : (Thu Jan 02 02:38:17 2020)
INFO    1/2/2020 2:38:17 AM READER_1_1_1    RR_4029 SQ Instance [SQ_Dim_Cust_Ord] User specified SQL Query [SELECT 
DISTINCT Dim_Cust_Ord.Business_Unit, 
Dim_Cust_Ord.PO_Num,
Dim_Cust_Ord.Account_Num 
FROM 
Dim_Cust_Ord Dim_Cust_Ord with(NOLOCK)
WHERE 
(Dim_Cust_Ord.PO_IND IS NULL OR Dim_Cust_Ord.PO_IND='O') 
AND Dim_Cust_Ord.Business_Unit IN ('C','W','R','U')]
INFO    1/2/2020 2:38:17 AM     WRITER_1_*_1    WRT_8270    Target connection group #1 consists of target(s) [Dim_Cust_Ord]
INFO    1/2/2020 2:38:17 AM WRITER_1_*_1    WRT_8003    Writer initialization complete.
INFO    1/2/2020 2:38:17 AM READER_1_1_1    BLKR_16007  Reader run started.
INFO    1/2/2020 2:38:19 AM READER_1_1_1    RR_4050 RR_4050 First row returned from database to reader : (Thu Jan 02 02:38:19 2020)
INFO    1/2/2020 2:38:19 AM LKPDP_1 TM_6660 Total Buffer Pool size is 609824 bytes and Block size is 65536 bytes.
INFO    1/2/2020 2:38:19 AM LKPDP_1:READER_1_1  DBG_21438   Reader: Source is [BIPAOSQL@SLM], user [informat]
INFO    1/2/2020 2:38:19 AM LKPDP_1:READER_1_1  BLKR_16007  Reader run started.
INFO    1/2/2020 2:38:19 AM LKPDP_1:READER_1_1  BLKR_16003  Initialization completed successfully.
INFO    1/2/2020 2:38:20 AM LKPDP_1:READER_1_1  RR_4049 RR_4049 SQL Query issued to database : (Thu Jan 02 02:38:20 2020)
INFO    1/2/2020 2:41:49 AM LKPDP_1:READER_1_1  RR_4050 RR_4050 First row returned from database to reader : (Thu Jan 02 02:41:49 2020)
INFO    1/2/2020 2:41:49 AM LKPDP_1:TRANSF_1_1  DBG_21079   DBG_21079 Creating Lookup Cache : (Thu Jan 02 02:41:49 2020)
INFO    1/2/2020 2:41:49 AM LKPDP_1:TRANSF_1_1  TT_11195    Warning: Unsorted input found when building the cache for the Lookup transformation [LKP_FACT_BOOK]. The current number of entries in the index cache is 2. For optimal performance, use sorted input.

INFO    1/2/2020 2:59:16 AM LKPDP_1:READER_1_1  BLKR_16008  Reader run completed.
INFO    1/2/2020 2:59:16 AM LKPDP_1:TRANSF_1_1  DBG_21641   LKP_FACT_BOOK: Index cache size = [178960000], Data cache size = [357916672]
INFO    1/2/2020 2:59:16 AM LKPDP_1:TRANSF_1_1  DBG_21297   Lookup cache row count : 30101473

INFO    1/2/2020 2:59:16 AM LKPDP_1:TRANSF_1_1  DBG_21294   DBG_21294 Lookup cache creation completed : (Thu Jan 02 02:59:16 2020)
INFO    1/2/2020 2:59:16 AM LKPDP_1:TRANSF_1_1  DBG_21682   Lookup table row count : 30101473
INFO    1/2/2020 2:59:16 AM LKPDP_1:TRANSF_1_1  DBG_21216   Finished transformations for Source Qualifier [LKP_FACT_BOOK{{DSQ}}]. Total errors [0] 
INFO    1/2/2020 2:59:16 AM LKPDP_1:TRANSF_1_1  CMN_1671    Created new cache files PMLKUP1301_4_0_2704068W64.[dat/idx] in directory F:\Informatica\InfoCache for Lookup [LKP_FACT_BOOK].
INFO    1/2/2020 2:59:17 AM WRITER_1_*_1    WRT_8167    Start loading table [Dim_Cust_Ord] at: Thu Jan 02 02:59:16 2020
INFO    1/2/2020 2:59:36 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 02:59:35 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 10296      Applied: 10296      Rejected: 0          Affected: 25585     

Severity    Timestamp   Node    Thread  Message Code    Message
INFO    1/2/2020 2:59:53 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 02:59:52 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 20592      Applied: 20592      Rejected: 0          Affected: 49811     

INFO    1/2/2020 3:00:13 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:00:12 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 30888      Applied: 30888      Rejected: 0          Affected: 74052     

INFO    1/2/2020 3:00:34 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:00:34 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 41184      Applied: 41184      Rejected: 0          Affected: 101520    

INFO    1/2/2020 3:00:56 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:00:56 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 51480      Applied: 51480      Rejected: 0          Affected: 129791    

INFO    1/2/2020 3:01:18 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:01:18 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 61776      Applied: 61776      Rejected: 0          Affected: 157788    

INFO    1/2/2020 3:01:41 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:01:41 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 72072      Applied: 72072      Rejected: 0          Affected: 189511    

INFO    1/2/2020 3:02:05 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:02:05 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 82368      Applied: 82368      Rejected: 0          Affected: 215331    

INFO    1/2/2020 3:02:23 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:02:23 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 92664      Applied: 92664      Rejected: 0          Affected: 243820    

INFO    1/2/2020 3:02:47 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:02:47 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 102960     Applied: 102960     Rejected: 0          Affected: 271753    

INFO    1/2/2020 3:03:11 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:03:10 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 113256     Applied: 113256     Rejected: 0          Affected: 297470    

INFO    1/2/2020 3:03:35 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:03:34 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 123552     Applied: 123552     Rejected: 0          Affected: 324556    

INFO    1/2/2020 3:03:59 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:03:59 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 133848     Applied: 133848     Rejected: 0          Affected: 351867    

INFO    1/2/2020 3:04:25 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:04:24 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 144144     Applied: 144144     Rejected: 0          Affected: 377844    

INFO    1/2/2020 3:04:34 AM READER_1_1_1    BLKR_16008  Reader run completed.
INFO    1/2/2020 3:04:34 AM READER_1_1_1    BLKR_16019  Read [158430] rows, read [0] error rows for source table [Dim_Cust_Ord] instance name [Dim_Cust_Ord]
INFO    1/2/2020 3:04:39 AM TRANSF_1_1_1    DBG_21216   Finished transformations for Source Qualifier [SQ_Dim_Cust_Ord]. Total errors [0] 
INFO    1/2/2020 3:04:43 AM WRITER_1_*_1    WRT_8161    
TARGET BASED COMMIT POINT  Thu Jan 02 03:04:43 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 154440     Applied: 154440     Rejected: 0          Affected: 401641    

INFO    1/2/2020 3:04:48 AM WRITER_1_*_1    WRT_8168    End loading table [Dim_Cust_Ord] at: Thu Jan 02 03:04:47 2020
INFO    1/2/2020 3:04:48 AM WRITER_1_*_1    WRT_8035    Load complete time: Thu Jan 02 03:04:47 2020

LOAD SUMMARY
============

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 158430     Applied: 158430     Rejected: 0          Affected: 412362    


INFO    1/2/2020 3:04:48 AM WRITER_1_*_1    WRT_8141    
Commit on end-of-data  Thu Jan 02 03:04:47 2020
===================================================

WRT_8036 Target: Dim_Cust_Ord (Instance Name: [Dim_Cust_Ord])
WRT_8041 Updated rows  - Requested: 158430     Applied: 158430     Rejected: 0          Affected: 412362    

INFO    1/2/2020 3:04:48 AM MANAGER PETL_24031  
***** RUN INFO FOR TGT LOAD ORDER GROUP [1], CONCURRENT SET [1] *****
Thread [READER_1_1_1] created for [the read stage] of partition point [SQ_Dim_Cust_Ord] has completed.
    Total Run Time = [1577.617025] secs
    Total Idle Time = [1574.481379] secs
    Busy Percentage = [0.198758]
Thread [TRANSF_1_1_1] created for [the transformation stage] of partition point [SQ_Dim_Cust_Ord] has completed.
    Total Run Time = [1579.801052] secs
    Total Idle Time = [1574.840183] secs
    Busy Percentage = [0.314019]
    Thread work time breakdown:
        EXP_DIM_CUST_ORD: 100.000000 percent
Thread [WRITER_1_*_1] created for [the write stage] of partition point [Dim_Cust_Ord] has completed.
    Total Run Time = [1531.502834] secs
    Total Idle Time = [1199.889383] secs
    Busy Percentage = [21.652813]

INFO    1/2/2020 3:04:48 AM WRITER_1_*_1    WRT_8043    *****END LOAD SESSION*****
INFO    1/2/2020 3:04:48 AM WRITER_1_*_1    WRT_8006    Writer run completed.

查询查询:-

SELECT 
SUM(Fact_Book.Qty) as Qty, 
Fact_Book.Num as Num, 
Fact_Book.Sold_To as STo, 
(case when Fact_Book.Div = '50'
then 'C'
when Fact_Book.Div = '40'
then 'R'
when Fact_Book.Div = '98'
then 'U'
else 'W'
end ) as Business_unit
FROM 
Fact_Book with(NOLOCK)
GROUP BY
Fact_Book.Num,
Fact_Book.STo ,
(case when Fact_Book.Div = '50'
then 'C'
when Fact_Book.Div = '40'
then 'R'
when Fact_Book.Div = '98'
then 'U'
else 'W'
end )```




sql sql-server informatica informatica-powercenter
1个回答
0
投票

如果源和目标是相同的物理表,并且查找使用相同的数据库,则可以利用Pushdown Optimization的优势-只需在会话属性中启用它并将其设置为Full

如果无法,请尝试将查询更改为连接器,对Source Qualifier上的数据进行适当排序,然后启用Sorted input连接器属性。

让我知道是否会有任何问题-还有更多可用的选项。

© www.soinside.com 2019 - 2024. All rights reserved.