innovaphone Apps SDK Developer Forum

Registered 3 years 291 days

mitop Wednesday, 9 February 2022, 10:25 AM

httpclient and json_io class

Hi everybody,

maybe a mindless question, but what is the easiest and most performant way to send a self assembled (and encoded) json_io class in the body of the httpclient with POST method?
The Session class for example offers the "SendResponse" method and can directly accept a json_io object. In the httpclient we have to provide byte* and the length of the message when using the Send method.

Do i need to cast it myself? What is the correct way? (have several procedures in mind) or is there a build in function that i should use?

Thank you very much and best regards

Mladen

Moderator

Registered 15 years 121 days

Daniel Deterding (innovaphone) Wednesday, 9 February 2022, 12:36 PM

1 of 1 users consider this post helpful

Re: httpclient and json_io class

Hi Mladen,

the HTTPClient has a SetRequestType method. Here you can either use chunked encoding or a fixed content length.

If you just send a simple JSON which you can encode with one json_io::encode call, simply set the content length to the return value of econde (take care, that encode in 13r1/13r2 just returns a 16bit value, in 13r3 it will be 32bit!).

The send the data with Send((byte *)data, length, true) in one piece. The socket layer itself will handle the rest for you.

Casting a char * buffer to byte * doesn't matter in this environment, as you hand the length and the buffer doesn't need to be null terminated.

Greetings,

Daniel

Registered 3 years 291 days

mitop Wednesday, 9 February 2022, 12:41 PM

Re: httpclient and json_io class

Thank you Daniel, i will do that and let you know if it worked out

Best regards,

Mladen

Registered 3 years 291 days

mitop Thursday, 17 February 2022, 05:08 PM in response to Daniel Deterding (innovaphone)

1 of 1 users consider this post helpful

Re: httpclient and json_io class

Hi Daniel,

it works

And i realized, the chunked based transport style is the better option, will implement that tomorrow

Thanks again and greetings

Mladen

Registered 3 years 291 days

mitop Friday, 18 February 2022, 03:43 PM in response to Daniel Deterding (innovaphone)

Re: httpclient and json_io class

Hi Daniel,

one last question: No matter if using chunked based transport style or not, every time when i send out one request and finish it by receiving the HTTPClientRequestComplete event with http_status_ok, i cannot use the httpclient anymore to do a second send out. I mean, i can call the send, but no events are fired and the data isn´t transported. Do i need to destroy the object or perform a manual shutdown of the client and do a second connect? Seems not to be the right way. I tried a reconnect, but i am still connected so i get an error.

What am i doing wrong? I want to send out one datapackage, check the result in HTTPClientRequestComplete (http_status_ok hopefully) and immediatelly send the next datapackage and wait for the HTTPClientRequestComplete event.

Regards

Mladen

Moderator

Registered 13 years 52 days

Andreas Fink (innovaphone) Friday, 18 February 2022, 11:22 PM

Re: httpclient and json_io class

Hello Mladen,

HTTPClientRequestComplete callback means only that request is complete, but do not mean the complete response was received. First when HTTPClientRecvResult with transferComplete=true is called the buffer is empty.

First with transferComplete=true you can call Shutdown and use Reconnect.

Until this happens Recv must be called triggered by HTTPClientRecvResult with transferComplete=false.

Best Regards

Andreas Fink

Registered 3 years 291 days

mitop Tuesday, 22 February 2022, 10:25 AM

Re: httpclient and json_io class

Hi Andreas,

i played around with it...

This is how the data is send out (headers set before and everthing else prepared):

- "tanssHttpClient->Send((byte *)data, len, true);"

I do receive a HTTPClientRequestComplete and i can see that my data is send out and proceded (i can see it on the other side), without HTTPClientRecvResult called at all. Calling:

"this->httpClient->Recv((byte *)malloc(1024), 1024);" at the End of HTTPClientRequestComplete will results in a transferComplete=true message inside HTTPClientRecvResult.

So i do:

- if (transferComplete) {
free(buffer);

Calling "tanssHttpClient->Send((byte *)data, len, true);" a second time, results in a HTTPClientShutdown.

I want to be sure:

- After getting "HTTPClientRecvResult" what is neccessary at all to proceed with the client, i cannot send out data anymore, except i shutdown the http client and perform a reconnect?

- So if my task is, to make sure every single json message was send out by the http client and received correctly by the other side, and let us say there could be a couple of houndered json messages a minute, i do need to perform a reconnect a couple of hundered times?

-- Chunk based is not a option, because important is that we receive a positive feedback from our receiver before proceeding with next data entry

But maybe such a client can't work at all because of the nature of http protokoll and I'm wrong.

Moderator

Registered 15 years 121 days

Daniel Deterding (innovaphone) Tuesday, 22 February 2022, 10:45 AM

1 of 1 users consider this post helpful

Re: httpclient and json_io class

Hi Mladen,

the flow must be like this:

callback HTTPClientRequestComplete

call client->Recv()

callback HTTPClientRecvResult

a) call client->Recv if transferComplete==false and wait for another HTTPClientRecvResult

b) call client->SetRequestType four your second request and client->Send

So as long as you do further requests, you do not call client->Shutdown, as this will also close the TCP socket.

Of course the HTTP server must support multiple requests on the same socket (aka HTTP 1.1 ;) ).

Greetings,
Daniel

Registered 3 years 291 days

mitop Tuesday, 22 February 2022, 04:39 PM

Re: httpclient and json_io class

Hi Daniel,

seems that i am lost, cannot get it running. Is your described solution meant for the chunk based transport or does it fit the usual one too?

Here is my flow, maybe there is an error:

A From Main App on startup connect to databases and via http client

A1 HTTPClientConnectComplete called

B Calling task on db if dataentry list is empty, read next dataentry, perform a second task with values from dataentry and preparing the sendout with the data

B1 Give data to httpclient

B11 SetRequestType(HTTP_POST, uriToRequest, contentType)

B12 SendContentForAuthentication(true);

B13 SetCustomHeaderField("name", value)

B14 Send((byte *)data, len, true)

C HTTPClientRequestComplete called

C1 Calling "Recv((byte*)malloc(1024), 1024)"

D HTTPClientRecvResult called

D1 If transferComplete=true (tried with and without free(buffer))

D11 GoTo B and check if there is more from dataentry to prepare for send out

Calling the "Send((byte *)data, len, true)" a second time with new data results in a HTTPshutdowm Event.

So i have still an error in this flow, or it only works with chunck based transport or as you mentioned the receiving server does not support multiple requests. Checked the server, the usual requests from the server are HTTP 1.1 version, and i can see that other requests are using keep-alive, so i added SetCustomHeaderField("Connection", "Keep-Alive") but with no effect.

No idea whats wrong here...

Moderator

Registered 15 years 121 days

Daniel Deterding (innovaphone) Wednesday, 23 February 2022, 07:16 AM

1 of 1 users consider this post helpful

Re: httpclient and json_io class

Hi Mladen,

Step B14:

here you call Send with last=true. Is this really the case?

a) Do you want to POST more data inside the same HTTP request if you get more data from your dataset or

b) do you want to establish a SECOND HTTP request where you send new data?

If a):

you cannot then call Send with last=true as long as you do not know if you'll send out more data if more data comes from the DB. Just set last=true for your last send call within the same HTTP request

If b):

in D11 you say that you just call Send(...) again after transferComplete==true. But you must prepare a second HTTP request first by calling SetRequestType again before calling Send!

Btw.:

SetCustomHeaderField("Connection", "Keep-Alive") is not neccessary, it's set automatically
If you call Recv mit malloc, you must free the buffer within RecvResult or remember the the buffer somewhere to free it later, otherwise you'll get leaks inside your app

Greetings,

Daniel

Edit: if you still do not get this working, please attach a log file with HTTPClient trace flag enabled. Then we should see what's happening from the view of the HTTPClient.

Registered 3 years 291 days

mitop Wednesday, 23 February 2022, 10:04 AM

Re: httpclient and json_io class

Hi Daniel,

i tried to realize a) and b) everything is better than reconnecting all the time after every successfully made request. To your "If b):": This is what the flow will do, explained in my previous post. I also tried immediatelly to SetRequestType inside transferComplete=true, but no matter what i do, chunk or usual, call the functions here or there, in trace log files there is only:

HTTPClient(00e71630,00e26280)::HTTPClientShutdown reason:HTTP_SHUTDOWN_NORMAL

If the client is working correctly, then the only conclusion that remains is that my remote station does not accept multiple requests on one connection, nor does it support chunk-based transport (I could just see that only the first value sent was displayed).

I apologize for wasting your time

... reconnecting all the time after every dataentry send seems to be the only solution

Thank you very much and greetings

Mladen

Moderator

Registered 13 years 52 days

Andreas Fink (innovaphone) Wednesday, 23 February 2022, 11:16 AM

1 of 1 users consider this post helpful

Re: httpclient and json_io class

Hello Mladen,

> If the client is working correctly, then the only conclusion

> that remains is that my remote station does not accept

> multiple requests on one connection, nor does it support

> chunk-based transport (I could just see that only the first

> value sent was displayed).

this would be easy to check with a Wireshark trace. Could you post a complete trace of at least two sequential requests?

Best Regards

Andreas Fink

Registered 3 years 291 days

mitop Wednesday, 23 February 2022, 11:35 AM

Re: httpclient and json_io class

Hi Andreas,

yes i can do that, my wireshark version needs an update first, after lunch i will post the trace. Is there a possibility to send you the trace on another way?

It will contain information that we are still keeping confidential.

Greetings

Mladen

Registered 3 years 291 days

mitop Thursday, 24 February 2022, 01:15 PM in response to Andreas Fink (innovaphone)

1 of 1 users consider this post helpful

Re: httpclient and json_io class

Hi all,

after Andreas and Daniel supported me in troubleshooting and Andreas took the trouble to search my trace for the possible error, it turned out in the meantime that the remote station (the receiving HTTP server/software) can only process one request per established connection. No sequential or chunck-based transport is supported. In our case it is even the case that the remote station accepts only one json object per request, so it is not even possible to send a list of json objects in one request.
Unfortunately it was not enough to call e.g. with Powershell:
- (Invoke-WebRequest https://yourserverUri -UseBasicParsing).headers
and conclude from that information what the remote station can do or not. If the manufacturer of the software does not provide any documentation about the functionality of the Api (how it works, not what it can do), the technical support of the remote station must be contacted to clarify these questions.

Thanks again to Daniel and Andreas (and the persistence of all of us

)

Moderator

Registered 13 years 52 days

Andreas Fink (innovaphone) Thursday, 24 February 2022, 01:21 PM

1 of 1 users consider this post helpful

Re: httpclient and json_io class

Thank you for follow-up, Mladen.

May be you can implement multiple requests in parallel, as workaround, in order to speed-up the communication.

Best Regards

Andreas Fink

Registered 3 years 291 days

mitop Thursday, 24 February 2022, 01:20 PM in response to mitop

Re: httpclient and json_io class

02-23 12:16:54.924 yourapp@yourdomain.de OpenSSLInstance::HandleSSLCallResult OpenSSL close 0 6
02-23 12:16:54.924 yourapp@yourdomain.de TLSSocket(01a29f98,01a69d28)::ReadSSLData ssl->ReadDecoded() close
02-23 12:16:54.924 yourapp@yourdomain.de TCPSocket::Shutdown(01a47da0,01a29f98)
02-23 12:16:54.924 yourapp@yourdomain.de TCPSocket(01a47da0,01a29f98)::SocketShutdown reason:0
02-23 12:16:54.924 yourapp@yourdomain.de TLSSocket(01a29f98,01a69d28)::SocketRecvCanceled
02-23 12:16:54.924 yourapp@yourdomain.de TLSSocket(01a29f98,01a69d28)::SocketShutdown reason:0
02-23 12:16:54.924 yourapp@yourdomain.de HTTPClient(01a69d20,01a1eff0)::HTTPClientShutdown reason:HTTP_SHUTDOWN_NORMAL

... my first request while doing sequential or chuck based works. Trying the second request on the same connection (with a recevier not supporting it) will lead to the above output in your log files...

You are here