Measurements

The goal of this experiment is to determine whether packet tailgating has accuracy comparable to previously known techniques with a significant reduction in the number of packets sent and received. For these measurements, we use a prototype implementation of nettimer. The results in this section are preliminary and are intended as a proof of concept and not an evaluation of the full potential of the technique.

**Table:** Program Versions: This table lists the versions of the programs we used.
Program	Version	Date
`pathchar`	alpha	April 21, 1997
`clink`	1.0	August 14, 1998
`pchar`	1.1.1	January 24, 2000
`nettimer`	1.0.6	May 30, 2000

We compare the results of the latest publicly available versions (listed in Table 2) of pathchar [7], clink [4], pchar [11], and nettimer. pathchar, clink, and pchar implement the one-packet technique described earlier. Although 1.0 is the latest publicly available version of Downey's clink program, it does not incorporate the adaptive algorithms described in [4]. However, those techniques are complementary with packet tailgating.

**Table:** Short Path: This table lists the results of running the link bandwidth programs on the short path. TTL is the distance of the link from the source. C is the number of channels in the link. BW/C is the actual physical bandwidth per channel. Columns 3-6 are bandwidths given in Mb/s.
TTL	C	BW/C	`pathchar`	`clink`	`pchar`	`nettimer`
1	1	10	7.9	7.8	7.9	6.7
2	1	100	34	35	34	62
3	1	100	31	32	32	21
4	1	100	9.6	7.9	7.9	38

**Table:** Network Load: This table lists the total number of packets transferred during the short and long path probes.
Program	Short Path Packets	Long Path Packets
`pathchar`	11562	31782
`clink`	6002	16400
`pchar`	11732	32417
`nettimer`	982	6663

**Table:** Long Path: This table lists the results of running the link bandwidth programs on the long path. TTL is the distance of the link from the source. C is the number of channels in the link. BW/C is the actual physical bandwidth per channel. Columns 3-6 are bandwidths given in Mb/s.
TTL	C	BW/C	`pathchar`	`clink`	`pchar`	`nettimer`
1	1	10	8.0	7.9	7.9	7.6
2	1	100	35	32	34	36
3	1	100	77	100	88	100
4	1	622	345	223	222	244
5	1	622	145	173	330	239
6	1	622	289	508	183	286
7	1	622	207	168	224	277
8	1	155	55	60	49	45
9	2	100	36	35	39	55
10	2	100	73	44	66	34
11	1	100	36	52	43	66

We ran each program using default settings from
tnt.stanford.edu to statistics.stanford.edu (4 hops) and to www.berkeley.edu (11 hops). The short path has a RTT of 1.0ms and the long path has an RTT of 4.2ms. The actual number of channels of the links and their bandwidths are listed in the 2nd and 3rd columns of Table 3 and Table 5. We chose these paths because we know that both endpoints are well connected and that their administrators would be likely to tell us the link bandwidths of these links. In addition, both the one-packet and tailgating techniques can measure these paths because the destinations do not rate-limit or filter ICMP packets and there is no very slow link followed by a very fast link.

The measurement source node, tnt.stanford.edu, is an Intel Pentium II 266MHz with 256MB of main memory. It is running Redhat 6.1 with a GNU/Linux kernel 2.2.12-20. The results were gathered from 1:21 to 2:13 AM PST on May 30. We chose this time because there would be relatively little traffic in the network. We used tcpdump to determine the number of packets sent and received.

The short path results are summarized in Table 3 and Table 4. All of the programs have approximately the same accuracy, but nettimer uses an order of magnitude fewer packets than the others. In bandwidth measurements, pathchar, clink, and pchar are mostly in agreement, which is not surprising considering their measurement technique is similar. In particular, all of these programs under-estimate the last link on this path. This is because the destination host responds to probes with an ICMP port unreachable message which is up to 584 bytes. This is within the RFC standard because it sometimes allows the sender to make a more accurate correlation between the ICMP packet and the UDP packet that caused it. However, pathchar, clink, and pchar apparently do not take into account how much longer the large ICMP packet takes to travel back to the source, causing an under estimation of the bandwidth.

The long path results are summarized in Table 5 and Table 4. These results show that there is not significant accumulation of error in the tailgating implementation compared to the others. However, the results of all the programs deviate by as much as 68% from the nominal values, even though there was relatively little traffic during this time.

We believe that the four most likely contributors to error for all four programs are 1) unreachable nominal values, 2) errors in implementation, 3) noise, 4) timing resolution, and 5) multi-channel links.

Although some of the links have a nominal value of 622Mb/s, in practice a router may not actually be able to forward packets this quickly. We are attempting to investigate this issue through simulation.

Another possible source of error is the implementation. The problem with not accounting for large ICMP packets is an example of such an error. If three independent implementations of the same algorithm all have the same error, then there are likely even more errors in our singular implementation.

The problem with noise is that we are trying to measure differences in packet delays that are several orders of magnitude smaller than delays caused by other traffic. In our experiment, to measure the 622Mb/s links, we have to detect differences in delays of ${\frac{1500 \textrm{bytes}}{622 \textrm{Mb/s}}}$ = 19 $\mu$ s. Just one packet queued at the wrong time could result in a delay of ${\frac{1500 \textrm{bytes}}{10 \textrm{Mb/s}}}$ = 1.2ms. This is almost 100 times larger than the delay we are trying to measure. One solution is to deploy nettimer software at the destination instead of relying on TCP RST packets. This would eliminate the interference of noise along the reverse path. We are implementing this in our next version of nettimer.

The timing resolution of the host machine and operating system can also cause a large error in the bandwidth estimate. We measured the timing resolution of the source machine by sending small packets to the loopback interface and measuring the smallest non-zero inter-packet delay. This gives an upper bound on timing resolution. The timing resolution of tnt.stanford.edu is no worse than 20 $\mu$ s. Nonetheless, this could cause us to measure a 622Mb/s link as a 414Mb/s link or a 1333Mb/s link. The GNU/Linux 2.3 kernel has a more efficient method of conveying packet timings to applications, and we hope to test this shortly.

Finally, while we are adapting nettimer to detect multi-channel links, the one-packet techniques will have difficulty measuring the multi-channel links 9 and 10 because of the inability of their fundamental model to consider such links.