Performance issues with chunked response

[darklow]

We are building autocompletion using nodejs, sphinx and trying out limestone.

Autocompletion containts only 6 items
Everything works perfect except one performance issue.

When searching we use limit: 6, so we need quite small amount of data to return.
Sphinx returns results in average time from 1 - 9ms

However with header Transfer-Encoding': 'chunked. it takes browser 180-200 ms to receive data. And it is huge time loss for us.
If i turn off chunked header, then browser recieved data in 10-16ms, which id perfect and desired result, but the response is not complete if headers ar not specified as chunked.

If i take full response data as string from chunked result and send it directly as body, then it also takes 10-16ms. So the problem is definately in that chunked data output or buffering.

Is there any way to output straight and not chunked data for small amounts of items? Otherwise these 180-200ms are huge performance loss for us.

Thanks

[darklow]

Well after lot of debugging finaly found nodejs parameter and it resolved all the chunk and performance issues

response.chunkedEncoding = false;

If there are no comments on this, please close issue and sorry for bothering :)
Thanks for such a great code!

[JoeZ99]

@darklow I'm on a somehow similar thing. That thing about the "chunks", never heard of it, but I'll check and maybe I can gain a few ms , too!

[darklow]

If anyone is interested here is how i got performance i wanted, avoiding chunked response. Now it takes maximum 10-15ms until request is received by browser:

1. Set chunkedEncoding= false, so that it don't use chunked encoding

response.chunkedEncoding = false;

2. Don't specify header:
   'Transfer-Encoding': 'chunked'

3. For content-length header use: buffer.byteLength and not data.length:

var buffer = require('buffer');
...
"Content-Length": buffer.Buffer.byteLength(data, 'utf8'),

This works if you have small amount of data, i guess if you have lot of data, then chunked encoding might be faster