A strategy sends a 40-byte order over TCP and measures 40 milliseconds of added delay on some sends and none on others. Nagle's algorithm is on and delayed acknowledgement is on at the receiver. Explain the exact interaction that produces 40 ms and give the two ways to fix it.

A strategy sends a 40-byte order over TCP and measures 40 milliseconds of added delay on some sends and none on others. Nagle's algorithm is on and delayed acknowledgement is on at the receiver. Explain the exact interaction that produces 40 ms and give the two ways to fix it.

Approach: State the rule Nagle enforces on small segments and the rule the receiver's delayed acknowledgement timer enforces, then find the state where each is waiting on the other.

Nagle holds a small segment until the previous small segment is acknowledged, and delayed ack holds the acknowledgement for up to 40 ms waiting for a response to piggyback on, so the two deadlock for one timer period even though the network round trip is microseconds. Nagle's rule is that at most one unacknowledged sub-MSS segment may be outstanding, which exists to stop a telnet session sending one byte per packet. The receiver's stack does not acknowledge immediately; it waits up to 40 ms, or 200 ms on some stacks, hoping to combine the ack with outbound data. If the application writes a small order, then writes a second small order before the first is acknowledged, the second sits in the sender's buffer until the delayed ack timer expires. The pattern is intermittent because it only bites when a write follows an unacknowledged small write, which depends on message timing. The fixes are to set TCP_NODELAY on the socket, which disables Nagle outright and is standard for every trading connection, or to write each logical message in exactly one write call so a second small segment is never queued. Setting TCP_QUICKACK on the receiver removes the other half but it is not sticky on Linux and must be reasserted.

Follow-up: With TCP_NODELAY set, what does your application now have to do that Nagle was previously doing for you?

Key concepts: nagle, delayed ack, tcp_nodelay, round trip.