Question Invalid Character data found in MEMPTR for codepage ISO8859-1

I am reading data from a https socket response into a memptr and would like to copy the data from the memptr to a longchar. However I keep getting the error

Invalid Character data found in MEMPTR for codepage ISO8859-1

The code to read the response looks as follows

PROCEDURE getResponse:
DEFINE VARIABLE vcWebResp AS LONGCHAR NO-UNDO.
DEFINE VARIABLE viBytesAvail AS INTEGER NO-UNDO.
DEFINE VARIABLE mResponse AS MEMPTR NO-UNDO.

ASSIGN viBytesAvail = vhSocket:GET-BYTES-AVAILABLE().

IF viBytesAvail = 0 THEN RETURN.

SET-SIZE(mResponse) = 0.
SET-SIZE(mResponse) = viBytesAvail.
vhSocket:READ(mResponse,1,viBytesAvail,READ-EXACT-NUM).

COPY-LOB FROM OBJECT mResponse TO vcWebResp.

ASSIGN vcFinalResp = vcFinalResp + vcWebResp.
SET-SIZE(mResponse) = 0.

END PROCEDURE.

I have tried to replace the line
COPY-LOB FROM OBJECT mResponse TO vcWebResp.
with
COPY-LOB FROM OBJECT mResponse TO vcWebResp CONVERT SOURCE CODEPAGE "iso8859-1" TARGET CODEPAGE "UTF-8".

but that did not help.

Any ideas what I can do to successfully complete the copy?

Have you tried

COPY-LOB FROM OBJECT mResponse TO vcWebResp CONVERT SOURCE CODEPAGE "utf-8" TARGET CODEPAGE "iso8859-1".

What is your session:cpinternal when running this code ?
What codepage is the data that you read from the https socket ?

Hi Cecil,
I agree with your solution but seems like he may be loosing the actual character that he need during this conversion (UTF-8 to ISO8859-1). Please correct me if am wrong.

Rather than coding it this way, I would recommend you to dig and see, When you know your DB is in ISO8859-1 code page format, why are we getting characters outside the character set? Is it right?

Regards,
Saravanakumar B

UTF-8 is _THE_ code page widely used on the internet whereas most classic GUI or CHUI applications stick to a localized code page. iso8859-1 is just one of them - it is used mostly in the US and Western Europe. As soon as your application has to deal with UTF-8 encoded data ( socket connections to other applications, possibly web servers, is just one of them, parsing XML documents from 3rd party providers might be another one ) one should consider to use UTF-8 for the database also. Even more so if that data needs to be stored in your database.

The good news is that Progress ( almost ) fully supports UTF-8. The bad thing is that UTF-8 does not come for free - storage and coding wise.

Heavy Regards, RealHeavyDude.

Hey, I just had a similar problem today where I was posting a JSON string (UTF-8) to WebSpeed and was then saving the the results to a CSV file and I was getting a  character randomly appear where there should be spaces. My resolution was to annoyingly replace the offending character with a blank.

My first solution was to try and use the CODEPAGE-CONVERT() function from UTF-8 to iso8859-1 as the all the characters where are ANSI text. But that failed to do anything.