Why esp_http_server answers take 250 ms

A page on the ESP32 that should load instantly takes a quarter of a second for every request, even a tiny JSON. The CPU is idle and the WiFi is fine. The time is spent waiting, on purpose, in TCP.

The symptom

Each response from esp_http_server takes about 250 ms, whatever its size. A page that makes ten requests feels sluggish, and a page that polls feels laggy. Measured from a browser, most of the time is between the first byte and the last, not before the first byte.

What happens

httpd_resp_send() writes the headers and the body in two separate sends. Two TCP rules then meet:

Nagle's algorithm, on the ESP32: while a small segment is still unacknowledged, hold the next small one back, to send fewer packets. The headers go out; the body waits for their ACK.
Delayed ACK, on the other side: do not acknowledge a lone segment at once, wait a little in case there is data to send back with it. The receiver waits for more; the ESP32 waits for the ACK.

Neither moves until the receiver's delayed-ACK timer fires. How long that is depends on the receiver: in lwIP it is a timer of 250 ms, on Windows about 200 ms, on Linux usually less. That is why it can look fine from one computer and slow from another, and slow through any proxy or tunnel that sits in between.

The fix

Turn Nagle off on the server's connections, so the body follows the headers at once. esp_http_server calls open_fn for every new connection, which is the place to do it:

#include "lwip/sockets.h"

static esp_err_t on_open(httpd_handle_t server, int fd)
{
    int one = 1;
    setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one));
    return ESP_OK;
}

httpd_config_t config = HTTPD_DEFAULT_CONFIG();
config.open_fn = on_open;
httpd_start(&server, &config);

That is all. Responses then take as long as the network and your handler take: a few milliseconds on the LAN.

Does it cost anything?

Slightly more, smaller packets when a handler writes many tiny pieces. For a web server that sends a header and a body, or a few chunks, there is no measurable cost. If a handler builds a response from many small writes, collect it into one buffer first; that is faster with or without Nagle.

Other servers

Arduino's WebServer also writes the headers and the body of server.send() separately, with Nagle on by default, so the same delay can appear there. Sending the whole response in one write avoids it.
The ESP-IDF example that comes with Pinscope sets TCP_NODELAY in open_fn. Pinscope's own connection to your server does not need it: it reaches local servers through lwIP's callback API and decides itself when to send.

Try it

Three devices are free, and no card is needed. Create an account, or read the five-minute guide first.