Writing on sockets : Problem !

[DevTeam]

Hi everyone,

First of all, please forgive my English writing, I will try to do my best.

Well, I am currently trying to make Progress & Java talk together through a socket.

I have a Java socket server listening on a port, upon which a Progress client gets a connection, succesfully.


But then I try to write a string (zipped through the zlib library) on a memptr and then send this memptr on the socket.

But the Java server seems to hang on the socket.readLine() instruction (it does not seem to guess the client finished to write on the socket). So I added the '~n' character to the zipped string, but then the unzipping (on the Java side) bugs ("invalid distance code" exception)...

Here is the Progress code :

DEFINE VARIABLE hndl AS HANDLE.
DEFINE VARIABLE bufSource AS MEMPTR.
DEFINE VARIABLE bufCible AS MEMPTR.
DEFINE VARIABLE bufCible2 AS MEMPTR.
DEFINE VARIABLE monIP AS CHARACTER INIT "" NO-UNDO.
DEFINE VARIABLE monPort AS CHARACTER INIT "" NO-UNDO.
DEFINE VARIABLE msocket AS HANDLE    NO-UNDO.
DEFINE VARIABLE connex  AS CHARACTER NO-UNDO.
DEFINE VARIABLE varStr  AS CHARACTER NO-UNDO.

CREATE SOCKET msocket.
msocket:CONNECT("-H" monIP "-S" monPort.).

IF msocket:CONNECTED()
THEN DO :
    RUN zlib.p PERSISTENT SET hndl.
END.
ELSE DO :
    MESSAGE "Impossible de se connecter à l'hote".
    RETURN.
END.

SET-SIZE(bufSource) = 0.
SET-SIZE(bufCible) = 64.
SET-SIZE(bufCible2) = 64.
varStr = "aaaaaabbbbbbcccccccdddddddeeeeeeefffffff".
SET-SIZE(bufSource) = LENGTH(varStr) + 1.
PUT-STRING(bufSource,1) = varStr.
PUT-STRING(bufCible2,1) = "~n".
RUN zlib_compressBuffer IN hndl (INPUT bufSource, OUTPUT bufCible).
mSocket:WRITE(bufCible,1,GET-SIZE(bufCible)).
mSocket:WRITE(bufCible2,1,GET-SIZE(bufCible2)).
MESSAGE "Compression envoyée".
WAIT-FOR READ-RESPONSE OF mSocket.

Is there any better way to tell Java that the writing is done ?

I'm sure this message is far from being clear, so do not hesitate to send me Private Messages for further explanations.

Thanks in advance for your help...

Julien

PS : the Progress zipping / Java unzipping works well when I make this through files instead of memptrs...

 

[FrancoisL]

Your problem seem to be that Readln blocks till it gets the end of line character but if you add it then your compress string becomes invalid to decompress because of the EOL character at the end. Even if you do 2 writes on the progress side the Java side will read the EOL char so your compress string contains more information then the original.

Did you try to remove the EOL character from the received string before doing a decompression?

Another solution would be to create your compressed buffer, get it length then do a send of the LENGTH first in a fix size string like 4 character then send the compress buffer. On the java side you do a Read of 4 to get your length and then you use this information to do another Read of the Length of the buffer.

 

[DevTeam]

Dear François,

First of all, thank you for answering.

Then, your first name is a common one in France, so I am wondering whether you are French (or speak French), which would be much easier for me, I must say !

Let's get back to the point...

I tried to retrieve 1 or several characters from the incoming string in order to get rid of the end-of-line sign, but I still had the exception about invalid distance code...

Your second proposition is a good idea, even if it implies one more sending on the network, which means dropping performances. But it's worth a shot !

Regards,

Julien

 

[FrancoisL]

I am from Quebec Canada and yes my primary language is French .

Je crois pas que cette solution va avoir un gros impact sur les performances de ton programme car tu ne ferme pas le socket entre les 2 envois . Tu envoi les 4 premiers et les reste en sequence . Le serveur recois presque tous en même temps mais tu fais juste lire les 4 premiers pour savoir la longueur de ta chaine et tu lis le reste apres . Envoyer 4 + X en un envoi ou 4 et X en 2 envoi sur le meme Socket demandera pas plus de performance. Ce qui prends de la performance c'est envoyé 4 , fermé le socket , ouvrir un nouveau Socket et envoyé X.

Le socket agis comme un tampon et garde se qu'il recois en attendant un Read pour lire le tampon mais tu n'es pas obligé de tous lire ce qui a sur le tampon en une lecture. Ce qui bloque les programmes c'est quand tu dits de lire X mais qu'il y a moins de X sur le tampon alors le programme attends que X soit disponible. C'est ce qui appelle un blocking Read car le programme attends de pouvoir lire ce que tu lui demande de lire.


Hesite pas si tu as plus de questions

 

[DevTeam]

Bonjour François,

(english traduction of this message in the following post)

Pardon de te répondre un peu en retard, mais j'ai préféré pousser mes tests au maximum avant de re-poster sur le forum.

Voici donc l'état de mes recherches :
J'ai mis en place ton idée d'envoyer d'abord la taille annoncée puis les données, et le problème de dézipping côté Java persiste.

Aussi ai-je essayé d'isoler la vraie cause du problème.

Il semblerait que j'aie un problème de conversion des bytes... Je m'explique.

• TEST 1

1. J'ai une chaine de caractères coté Progress, que je mets dans un memptr que je passe en paramètre à une procédure utilisant la librairie zlib. Cette procédure me retourne les données zippées dans un nouveau memptr.
2. J'utilise ce memptr pour alimenter un stream pointant vers un fichier sur le disque.
3. Mon programme Java lit ce fichier byte par byte, les mets dans un tableau. Je relève alors les valeurs des bytes de ce tableau : [120, -100, 75, 76, 4, -127, 36, 48, 72, -122, -128, 20, 8, 72, -123, -128, 52, 8, 0, 0, 60, 64, 15, -111]
4. Mon programme décompresse parfaitement les données qui s'y trouvent :: je récupère ma chaine de caractères d'origine.

• TEST 2

1. Idem que test1
2. J'écris ce memptr sur la socket (instruction mSocket:WRITE(bufCible,1,GET-SIZE(bufCible))
3. Mon programme Java reçoit les bytes et les met dans un tableau de bytes : [120, 83, 75, 76, 4, -3, 36, 48, 72, 32, -84, 20, 8, 72, 38, -84, 52, 8, 96]
4. Mon programme crashe lors de la décompression ("incorrect header check")

On s'aperçoit que les 2 tableaux ne sont pas identiques (taille différente, quelques bytes différents).


Dans le test1, j'alimente le fichier via un "EXPORT .... BINARY NO-CONVERT", donc je me demande si mon problème lors de passage par la socket n'est pas due à un codepage différent, ou quelquechose dans le genre.

Si jamais tu as une idée, je suis preneur !

Merci encore pour ton soutien.

Julien

 

[DevTeam]

For non-French readers

While I was trying François' propositions about the way of reading the socket in Java, I found something rather strange :

• CASE #1

1. My Progress program puts a string into a memptr, which is passed as a parameter to the zlib_compressBuffer function using zlib library. This function returns a second memptr containing zipped data.
2. I use this memptr to feed a stream in order to create a file.
3. My Java program reads this file and puts every byte in a byteArray, which contains at the end : [120, -100, 75, 76, 4, -127, 36, 48, 72, -122, -128, 20, 8, 72, -123, -128, 52, 8, 0, 0, 60, 64, 15, -111]
4. The Java program successfully uncompresses data contained in the file : the string retrieved is the same as the original one in Progress.

• CASE #2

1. The same as in case #1
2. I write the second memptr on the socket : mSocket:WRITE(bufCible,1,GET-SIZE(bufCible)
3. The Java program reads every byte on the socket and stores them in a byte array, which contains at the end : [120, 83, 75, 76, 4, -3, 36, 48, 72, 32, -84, 20, 8, 72, 38, -84, 52, 8, 96]
4. Java crashes when uncompressing ("incorrect header check" exception)

The exception seems quite normal, as the 2 arrays differ...


In case#1, the file is fed through the "EXPORT .... BINARY NO-CONVERT" instruction. That's the reason why I am wondering if writing on the socket implies codepage problems, or something like that.

Any advice or suggestion is welcome !
Thanks in advance.

Julien

 

[Casper]

With the export function you are writing to a file. You want the memptr not converted to the cpstream of your application that's why you use .... binary no-convert.

When writing to a socket you just write the binary data from the memptr directly to the socket. So you don't have any codepage issues there.

HTH,

Casper.

 

[DevTeam]

(Resolved) Writing on sockets : Problem !

Thanks for your help.

I will put the emphasis on the Java side, maybe my socket reading does not work properly...

I'll let you know when it's done.

Julien

EDIT : well, my bad... The problem was on the socket.readline().getBytes() instruction in Java, which uses the default charset (iso8859-1). This method provides another signature : socket.readline().getBytes("UTF-16BE"), which works perfectly

Sorry for the bothering...

 

[FrancoisL]

Glad you were able to find the problem , charset can be a real pain sometimes.