Incorrect Special Character Handling in Informatica Powercenter 9.1

Question

I am currently working on a project in my organisation where we are migrating Informatica Powercenter in our application from v8.1 to v9.1.

Informatica PC is loading data from datafiles but is not able to maintain certain special characters present in few of the input dat files.

The data was is getting loaded correctly in v8.1.

Tried changing characterset settings in Informatica as below -

CodePage movement = Unicode
NLS_LANG = AMERICAN_AMERICA.UTF8 to ENGLISH_UNITEDKINGDOM.UTF8
"DataMovementMode" = Unicode

After making the above settings I am getting the below error in the in Informatica log:

READER_1_2_1> FR_3015 Warning! Row [2258], field [exDestination]: Data [TO] was truncated. 
READER_1_2_1> FR_3015 Warning! Row [2265], field [exDestination]: Data [IOMR] was truncated. 
READER_1_2_1> FR_3015 Warning! Row [2265], field [parentOID]: Data [O-MS1109ZTRD00:esm4:iomr-2_20040510_0_0] was truncated. 
READER_1_2_1> FR_3015 Warning! Row [2268], field [exDestination]: Data [IOMR] was truncated.

The special character that are being sent in the data are and not being handled correctly -

Ø
Ù
Ɨ
¿
Á

Can somebody please guide how to resolve this issue? What else is required at Informatica end to be changed. Does it need any session parameters to be set in database?

Answer 1

Also set your integration service (IS) to run in Unicode mode for best results apart from configuring ODBC and relational connections to use Unicode

Details for Unicode & ASCII

a) Unicode - IS allows 2 bytes for each character and uses additional byte for each non-ascii character (such as Japanese/chinese characters)

b) ASCII - IS holds all data in a single byte

Make sure that the size of the variable is big enough to hold the data. Some times the warnings mentioned will be received when the size is small to hold the incoming data.