TCP/IP stack fingerprinting: TTL, window size, and MSS as OS identity

Before a browser sends a single byte of HTTP, before the TLS ClientHello, before anything you can edit in code, the operating system has already announced itself. The first packet of any TCP connection is a SYN, and that SYN carries a handful of fields the application never chose. The initial time-to-live. The advertised window. The maximum segment size. A list of TCP options in an order the kernel fixed years ago and never bothered to expose as a setting. None of these say “I am Windows” in plain language, but together they say it clearly enough that a passive observer can read the operating system off the wire with no cooperation from the host and no extra packets sent.

That is the uncomfortable part for anyone trying to disguise a client. You can set the User-Agent to whatever string you like. You can spoof the TLS fingerprint with a library that mimics Chrome down to the cipher order. But the SYN was built by the kernel of the machine you are actually running on, and the kernel does not consult your User-Agent before it fills in the TTL. If the header says Windows and the SYN says Linux, the two halves of the same connection disagree, and that disagreement is one of the oldest and most stubborn signals in bot detection. This post is about the fields that carry the signal, where their default values come from, how the classic and modern tools turn them into a label, and why the transport layer is so much harder to forge than the layers above it.

The sections below run roughly bottom to top. First the IP header field that does the most work, the initial TTL, and the arithmetic of hop distance. Then the TCP window size and the scaling option that complicates it. Then the MSS, and why a value a few bytes below 1460 betrays a tunnel. Then the single most distinctive field of all, the ordering of TCP options, and how p0f and JA4T encode the whole SYN into a signature. A closing section on the cross-layer mismatch that catches proxies, and what the transport layer can and cannot tell a defender in 2026.

The field that survives the trip: initial TTL

The time-to-live is an eight-bit field in the IPv4 header, hop limit in IPv6, and its job is mundane. It stops packets from circling a routing loop forever. Every router that forwards the packet decrements it by one, and when it hits zero the packet is dropped and an ICMP time-exceeded message goes back to the sender. That is the whole mechanism, and traceroute is built entirely out of abusing it.

What makes TTL useful for fingerprinting is that the initial value is not specified by the protocol. RFC 1700 recommended 64 as the default, and that recommendation was widely ignored. Operating systems picked their own starting points and have mostly kept them for decades. Linux and the BSDs and macOS start at 64. Windows starts at 128. Network gear like Cisco IOS routers starts at 255. Those three values, 64, 128, and 255, cover almost everything you will see on the open internet, which is convenient because they are spaced far enough apart that a few router hops cannot blur one into another.

The blurring is the catch. By the time a SYN reaches a server, its TTL has already been decremented once per hop along the path. A packet that left a Windows box at 128 might arrive at 117 after eleven hops. So the observer does not read the OS off the raw TTL. It reads the nearest initial value above the observed one. An arriving TTL of 117 was almost certainly 128 to start with, eleven hops back. An arriving 54 was almost certainly 64, ten hops back. This is the reasoning p0f encodes when it writes an initial TTL as 54+10, meaning observed 54, distance 10, inferred initial 64. When p0f cannot be sure of the distance it writes nnn+? and tells you to confirm with traceroute.

TTL alone is weak. It splits the world into three buckets and nothing finer, and a router or NAT box in the path can rewrite it. But it is cheap, it survives the entire trip to the server unencrypted, and it is the first thing every passive fingerprinter looks at. The classic Netresec write-up on passive OS fingerprinting reduces the whole technique, in its simplest form, to two fields read off the first packet: the initial TTL and the TCP window size. Get both and you have already narrowed the OS family considerably.

The window size and the scaling trick

The TCP window is the receiver’s flow-control promise: how many bytes it is willing to accept before the sender must stop and wait for an acknowledgment. The field in the TCP header is sixteen bits, so the raw value tops out at 65,535 bytes. On the SYN packet that opens a connection, this is the initial receive window the host advertises, and like the TTL it is a value the OS picks rather than the application.

The defaults vary enough to be useful. Netresec’s table lists Linux 2.4 and 2.6 kernels advertising 5840, Google’s customized Linux a slightly different 5720, FreeBSD and Windows XP both at 65535, Windows 7 and Vista and Server 2008 at 8192, and a Cisco IOS 12.4 router at 4128. Those are old systems, but the principle holds on current ones: each stack has a characteristic starting window, often computed as a small multiple of the MSS. The Pydoll network-fingerprinting notes describe modern Linux kernels advertising an initial window of 29200, which is exactly twenty times an MSS of 1460, with newer 5.x and 6.x kernels sometimes using 64240, which is forty-four times 1460 minus a bit. Windows 10 and 11 sit up around 65535 with auto-tuning. macOS defaults to 65535 as well.

Sixteen bits was never going to be enough for a fast modern link, where you want megabytes in flight, not kilobytes. RFC 1323, later folded into RFC 7323, added the window scale option to fix this. It is a three-byte TCP option sent only on the SYN, and it carries a single shift count from 0 to 14. The real window is the sixteen-bit field shifted left by that many bits, so a scale of 7 multiplies the advertised window by 128, and a scale of 14 by 16,384. Because the option appears only in the SYN, the scale factor for each direction is locked in at connection setup and never renegotiated.

That scale factor is itself a fingerprint. It is derived from the receive-buffer configuration, so it tracks the OS and its tuning. Linux commonly advertises a scale of 7. Windows often uses 8. The incolumitas VPN-detection research caught a clean example of this: an Ubuntu 18.04 host advertising window scale 7 against an Android 9 device advertising scale 9, the two stacks distinguishable on that field alone even when their MSS matched. The p0f signature format treats an excessively large scale, anything above 14, as a quirk worth recording, because it is out of spec and only a handful of stacks emit it.

The window deserves its reputation. Nmap’s OS-detection chapter notes that the sixteen-bit window value alone is effective because more than eighty distinct values are known to be emitted by at least one operating system. Combine the raw window, the scale factor, and the fact that the window is often a clean multiple of the MSS, and you have a field that carries far more identifying information than its sixteen bits suggest. The companion piece on TCP timestamp and window-scaling fingerprints goes deeper on the scaling and timestamp angle.

MSS, the MTU, and the tunnel tell

The maximum segment size is a TCP option, present on the SYN, that tells the other side the largest TCP payload it is willing to receive in a single segment. It exists to keep segments from being fragmented at the IP layer. A host computes its MSS from the MTU of the interface the connection will use, subtracting the IP and TCP header overhead. On standard Ethernet the MTU is 1500 bytes, and after taking off 20 bytes of IPv4 header and 20 bytes of TCP header you are left with 1460. So 1460 is the overwhelmingly common MSS on the open internet, and the FoxIO write-up on JA4T calls it exactly that, the most common MSS, based on an Ethernet MTU of 1500.

The interesting MSS values are the ones that are not 1460. When the path includes a tunnel, the tunnel’s own headers eat into the usable payload, and the effective MTU drops. A VPN, a PPPoE link, a mobile carrier’s infrastructure, all of these shrink the MTU and therefore the MSS. The result is a value a few bytes or a few dozen bytes below 1460. FoxIO points out that an MSS slightly under 1460, such as 1436, suggests a network element sitting in-line before the host. The incolumitas measurements show MSS values like 1412 and 1424 in the wild, both below the Ethernet baseline, both consistent with a tunnel or a carrier in the path. Older guidance on proxy detection notes that PPTP, L2TP, and IPsec IKE all lower the MTU, and that comparing the segment sizes in an intercepted connection against the standard MTU and MSS can reveal a tunnel.

There is a more aggressive version of this where a middlebox actively rewrites the MSS, called MSS clamping. A router terminating a tunnel will catch passing SYN packets and lower their MSS option to whatever fits inside the tunnel, so that no segment ever exceeds the path MTU and triggers fragmentation. This is common on consumer routers and VPN gateways. The visible effect at the server is the same: an MSS that is suspiciously below 1460, on a connection whose User-Agent claims a plain desktop on Ethernet. The exact clamp value depends on the tunnel; WireGuard, OpenVPN, and IPsec each carry different overhead, and a specific clamp number is a property of the deployment rather than a fixed constant, so a server reads the shape of the anomaly more than any single magic number.

This is also where p0f’s handling of the MSS gets clever. The window size in a signature is often expressed relative to the MSS, as a multiple like mss*20, precisely because so many stacks size their initial window as a function of the MSS. If a stack derives the MSS from the interface MTU, and the window from the MSS, then a single odd MTU ripples through both fields in a correlated way that is hard to fake piecemeal. The companion post on MTU and path MTU discovery follows that thread further.

The most distinctive field: TCP option order

Everything so far narrows the OS to a family. The field that often pins it down is the order of the TCP options on the SYN. RFC 793 lists the options a TCP implementation may include but does not mandate any particular ordering, and Nmap’s OS-detection chapter makes the consequence explicit: because no ordering is required, implementations come up with their own, and those orderings are stable per stack. The kernel emits its options in a fixed sequence, version after version, and almost never exposes that sequence as something an application can change.

The common options on a SYN are MSS (option kind 2), window scale (kind 3), SACK-permitted (kind 4), timestamp (kind 8), and the structural fillers NOP (kind 1) and end-of-options (kind 0). NOP exists mostly to pad options so they align on four-byte boundaries, and where a stack inserts its NOPs is itself part of the signature. The orderings differ cleanly across the major systems. The Pydoll notes give three sequences that match what p0f’s database has encoded for years:

1
Linux:    MSS, SACK_PERM, TIMESTAMP, NOP, WSCALE
2
Windows:  MSS, NOP, WSCALE, NOP, NOP, SACK_PERM
3
macOS:    MSS, NOP, WSCALE, NOP, NOP, TIMESTAMP, SACK_PERM

Two things jump out. First, Windows in its default configuration does not send the timestamp option at all, while every Unix-derived stack does. FoxIO states this plainly: Microsoft Windows does not use TCP option 8, whereas all Unix-based operating systems do. That single presence-or-absence bit separates the Windows family from everything else before you even look at ordering. Second, the placement of the NOP padding and the relative position of SACK-permitted and the timestamp differ between Linux and macOS even though both are Unix, which is what lets a fingerprinter tell a Mac from a Linux box when the TTL and window alone would not.

iOS adds its own tell. FoxIO notes that iOS ends its option list with a TCP option 0, the explicit end-of-options-list marker, where other systems simply stop. Small structural quirks like a trailing EOL, an unusual NOP count, or a SACK-permitted that sits before rather than after the timestamp are exactly the kind of detail that no application-level disguise touches, because the application never built the options block in the first place.

MSS NOP WS NOP NOP SACK

MSS NOP WS NOP NOP TS SACK

1
ver : ittl : olen : mss : wsize,scale : olayout : quirks : pclass