8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 1/10

Playing Sherlock Holmes todetect CONVT_CODEPAGEruntime error mysteryTOPICS: Chinese Characters

Conversion Of A Text From Codepage '4110' To Codepage

'4102

CONVT_CODEPAGE Runtime Error

CX_SY_CONVERSION_CODEPAGE

POSTED BY: SAP YARD AUGUST 15, 2015

Recently we had our unicode upgrade along withEhp7. One of our batch job, which is used foruploading mass data from a file in application serverto SAP system, intermittently starting giving runtimeerror.

Enter email

Subscribe

RECENT POSTS

Playing Sherlock Holmes todetect CONVT_CODEPAGEruntime error mysteryDELETING rows of theinternal table within theLOOP. Is it a Taboo? A bigNO NO?Quick Reference for Vistex

SAP YARDYOUR BACKYARD FOR SAP TECHNICAL TIPS AND SOLUTIONS

HOME SEE ALL POSTS ASK YOUR QUESTIONS ABOUT ME CONTACT ME

You and 92 other friends like this

SAP Yard172 likes

Liked

SEARCH …

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 2/10

Exception : CX_SY_CONVERSION_CODEPAGE

Reason: During conversion of a text from code page‘4110‘ to code page ‘4102‘, one of the followingoccurred:– characters were discovered that cannot be dislpayed inone of the code pages, or– the conversion could not be performed for some otherreason

The dump looked like below.

The dump took place at READ statement.

TechnicalOffshore DevelopmentModel in 10 StepsReady Reckoner for SAPDevelopers

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 3/10

The issue was strange, as it worked for 99% of filesbut dumped for some very few. It means, the codeworks good for maximum cases but could not handlesome specific scenarios. I looked at the text file whichwas loaded to application server very minutely.But, nothing suspicious was visible to open eyes.

Since most of the files were loaded successfully, Iplanned to investigate this file more to figure outwhat was different in this file from rest which loadedcorrectly. To my good luck, this sample file had just 3lines, for me to scan through. But still, I did not findanything wrong.

Not sure why, may be flash of Mr Holmes , I thoughtof opening this file using Microsoft Excel. Opened ablank excel file. File -> Open -> txt file.

The moment I hit ‘Open’, look what I found!! Specialcharacters treasure which I was hunting for. Somehow,the creator of this file ended up putting some specialcharacters (may be Chinese) which our program wasnot able to handle.

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 4/10

Once you know the root cause, finding solution was nobrainer. Our business analyst quickly deleted the specialcharacters and re-ran the job. Boom!! everything loadedsuccessfully.

As an ABAPer, I thought, our program should havebeen smart enough to handle such simple exception.I, did an ‘F4‘ on ‘OPEN DATASET‘ key words andfound the below documents. I must have seen ithundreds of time, but for the first time I went througheach words to understand what it meant. I wanted tounderstand, how the system would behave when weincorporate the key words specified in sap document.

Alternative 1.I tried explicitly specifying UTF-8, but got the samedump.

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 5/10

OPEN DATASET p_file FOR INPUT IN TEXT MODEENCODING UTF-8 SKIPPING BYTE-ORDER MARK.

Alternative 2.Then I tried replacing ‘ENCODING DEFAULT’ by‘ENCODING NON-UNICODE‘.OPEN DATASET p_file FOR INPUT IN TEXT MODEENCODING NON-UNICODE.

Bingo.. It worked!!!

Debug screen shows that SAP considers the specialcharacters as space. So, no issue.Is this a right way? I was not sure and I am not sure tilldate. May be some expert in this area can throw somelight. What I could do best was run Extended SyntaxCheck and Code Inspector. None of them gave anywarning or error. Hopefully SAP likes it..

But already being in UNICODE system, is it goodpractice to OPEN file using NON-UNICODE encoding?Not sure..

Alternative 3.I also tried adding the key word ‘IGNORINGCONVERSION ERRORS‘.

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 6/10

OPEN DATASET p_file FOR INPUT IN TEXT MODEENCODING DEFAULT IGNORING CONVERSIONERRORS.

The file did not dump in this case as well. SAPhandled the conversion.

Debug screen shows that the special character isread as ‘#’ at runtime. Also note that the specialcharacter ‘#’ and the Tab separator ‘#’ looks thesame.

Now, I was wondering, will the SPLIT AT TAB(separator) statement give wrong output. Look how itbehaves in Debug.

Although it did not dump, but the IGNORINGCONVERSION ERRORS would add those unwanted ‘#’,if we load the data to SAP System.

For now, I believe Alternative 2 is OK, though I am notfully aware if we should use it or not.

If you can correct your file i.e remove unwantedcharacters and load them, that is even better. The best

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 7/10

4 COMMENTS

ON "PLAYING SHERLOCK HOLMES TO DETECT CONVT_CODEPAGE

alternative

By any chance, have encounter this issue earlier?Please share how you resolved it. If you want to addsome more information on this topic, please feel freeto email me at mailsapyard@gmail.com or leave yourexplanation in the comment section.

Till, someone provides the right answer with goodjustification, the mystery continues…

Sample text file with the special characters.

Code snippet used for analyzing the above scenarios.

If you have 5 minutes, you can upload the above textfile to application server using t-code CG3Z and thencreate the program using the code snippet above andtest it.

If you liked this page, you might like to check ourother post on Unwanted Error.

Please subscribe to get updates about our new post.

Thank you very much for your time!!

 

 

Image source : www.flickr.com

Previous post

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 8/10

RUNTIME ERROR MYSTERY"

Thank you so much for sharing this with me. Ihave never come across that error but am surehave learn and if i get some infor willdefinitely share with you as well. Thanks forsuch helpful information. I keep learning fromyou every day.

Regards

William Mutero

Thanks William..Appreciate your visit..

Regards,Raju.

It’s really useful!As you said, ” I must have seen it hundreds oftime, but for the first time I went througheach words to understand what it meant” – so

Mutero william | August 17, 2015 at8:42 am | Reply

SAP Yard | August 17, 2015 at3:23 pm | Reply

Le Van Son | August 18, 2015 at 4:31 am| Reply

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 9/10

do i.

Dear Le Van – Glad thatyou think it is useful. We generallyglance through the SAP document, without analysing what it actually means..Whenever we are in trouble, we tend toread more carefully.. .. Glad that I amnot the only one..

Regards,Raju.

Leave a commentYour email address will not be published.

Name *Raju

Email *respond2raju@gmail.com

Website

Comment

SAP Yard | August 18, 2015 at4:39 am | Reply

8/18/2015 Playing Sherlock Holmes to detect CONVT_CODEPAGE runtime error mystery | SAP Yard

http://www.sapyard.com/playingsherlockholmestodetectconvt_codepageruntimeerrormystery/ 10/10

Post Comment

COPYRIGHT 2015 | SAPYARD BY WWW.SAPYARD.COMALL PRODUCT NAMES ARE TRADEMARKS OF THEIR RESPECTIVE COMPANIES. SAPYARD.COM IS NOT AFFILIATED TO SAP AG.