Aug 2005
Most of the NS scenarios in CUBIN Lab's tests show this problem.
We did NS simulations with various noise level from 2% ~ 80% of capacity. In these cases, noise traffic with 5% of the capacity is enough to let the new flows see baseRTT.
CUBIN Lab's low buffer test and most of the Hamilton Insititute's test.
MI is the suspected component in this issue.
Suggestion: change MI to other fairer rapid increment such as HSTCP like increment when queue is not detected.
Hamilton Institute's test
Alpha-tuning is the suspected compenent in this issue since several HI's figures show that one flow may suddenly drop its cwnd to be much lower than the other flows, while everyone is still stable.
I believe we disabled this version of alpha-tuning already?
Hamilton Institute's test, NCSU's test, SLAC's test on production link in Apr 2003.
FAST congestion control keeps up to alpha packets when it sees the queueing delay generated by Reno. If buffer size is small, or bandwidth delay is large, this 200pkt is more aggressive to Reno. When BDP is small or buffer size is huge, this 200pkt is less aggressvie than Reno. So in general, the behaviour is complicated. Alpha-tuning usually makes FAST much less aggressive.
If the alpha can be tuned to occupied a proportion of the buffer size (if using maxRTT is not very risky), this problem can be solvable.
SLAC's test on Aug 2003 (reported PFLDNet 2004)
FAST see the reverse traffic as congestion and reduce the rate to very very low.
Most of SLAC's test
The reason is not clear. May be the followings:
CUBIN Lab uses NS2 version of the FAST. The NS2 version is developed in CUBIN Lab. Some bug was found and fixed for loss recovery after the tests were done. Hopefully, these bugs did not affect the test results since most of the scenarios have enough buffer, except one.
There is still some problem in the implementation when I did the test recentely (Aug 12). I saw a zero value of cwnd. I need to invetigate further to identify the problem.
The case shows that FAST could not get correct RTT estimation for flows join the network later. This is done with NS2 without noise.
FAST works as expected, also revealing the problem of base RTT estimation.
FAST has two problems:
1. baseRTT estimation is broken
2. The algorithm is unfair.
I guess the first problem is due to the fact that the simulation code does not have SACK implemented. Also, the loss recovery seems to be Reno such that the burstiness after loss is very significant. See the Random Loss case for more clear picture.
The second problem seems to be the MI part. I did another simulation without MI and did not see such unfairness (But that test is less congested in terms of sigma alpha).
Work as expected, also with the baseRTT problem.
Work as expected. But I would say FAST reduces its congestion window too aggressively.
Hamilton Inistitute uses the FAST beta 2 version, with alpha tuning turned on.
Gamma is set to be 50 pkts but the smallest alpha is 8 pkts. Not sure if gamma is adaptive as alpha changes.
FAST has a very bad fairness. Also, the fairness oscillates randomly when RTT changes.
The alpha tuning is highly suspectible. The buffer size in these tests are set to 20% of BDP. The bandwidth is 10Mbps (0.87 pkt/ms) or 250Mbps ( 22 pkt/ms). The RTT is from 15ms to about 162ms. In most of these experiments, the buffer size is smaller than 400pkts (the buffer size is from 3pkt to 675pkt) which are the queue size to hold intial value of alphas from each flow. Hence the alpha tuning is activated and drops the alpha value according to the loss signal. This also happens when RTT=162ms and bandwidth=250Mbps. In that case, alpha tuning is triggered, probably by a random loss in one flow. And the throughput of that flow drops significantly.
In the very low end, both flows have the smallest alpha value and should be fair if there were no MI. But in these cases, FAST works as MIMD due to the MI component. MI=1/8 and MD=1/2. In this case, FAST has similar problem with Scalable TCP. This is the case with 10Mbps flows and small buffer.
Again, FAST has not good fairness as expected. In the low end, when buffer size is very small (for 10Mbps, the buffersize is from 3pkt to 27pkt), FAST behaviours like MIMD with MI=1/8 and MD=1/2. So, it is unfair among flows with different RTTs.
In the high end, when the buffer size is large enough, alpha tuning makes the flows unfair. In most of the scenarios, alpha tuning trigged by random loss is observed.
FAST is more aggressive in the beginning (due to large value of alpha) and less aggressive than Reno after a while (due to the change of alpha).
BW=100Mbps, Delay=82ms, queue size=0.01 ~ 1 BDP, which is equivalent to 7pkt ~713.4 pkt.
FAST works worse than others when the buffer size is smaller than 0.05 BDP (35pkts) and works as good as others when buffer size is larger than that.
NCSU uses the FAST beta 2 version, with alpha tuning turned on.
The only material availabe from NCSU is the ppt in PFLDNet 2005.
Page 14 and 15 showed that Reno is using up all the bandwidth (80% of the 800Mbps) while FAST is not using any bandwidth. This is contradicting to the experiment results where Reno alone cannot use 80% of the 800Mbps with 100ms or 200ms delay. The author acknowledge that there may be some problems in the data.
However, we note that with alpha tuning turned on, Reno does have more advantage over FAST in these cases. But Reno should not be performing so well in these cases.
Note: The testbeds in SLAC are usually refered to the production links that are lightly loaded. The experiments are not exactly reproducable.
SLAC->Caltech cannot reach full utilization. From the figure, it seems that a single flow stablizes around 400Mbps, not sure why. And oscillaiton happens when more than one flow is present -- probably because the buffer is small since the BDP is small. It is likely that the buffer size is so small that the smallest alpha value is not small enough.
The same observation is in SLAC->UFL with more than two flows.
SLAC->CERN has a severe fairness problem. The second (green) flow never get a fair share. Alpha tuning is highly suspected here. Also, the first 120 seconds have several throughput drops that take several seconds to recover. Not sure why. Timeout? but why so severe timeout?
FAST works poorly when reverse traffic is present. (The reverse traffic is 16 Reno flows. which is strong enough to kill FAST in the forward path).
The full available capacity is about 400Mbps. RTT is about 100ms. The reverse traffic adds 200ms additionally. alpha/queue is about 1pkt/ms which is 10Mbps.
Note that all other variants suffer from the reverse traffice, but FAST is very badly affected.
The path RTT is about 49.2 ms and the bottleneck is about 550Mbps.
FAST has worse stability with the window size is 8MB than when the window size is 4MB. This is strange. (4MB is about the BDP. That means FAST overshoots. That is single link! The buffer size in the path seems to be 50ms (from STCP and Reno16 and HSTCP got 80ms queueing delay)
net.ipv4.tcp_vegas_alpha = 200
net.ipv4.tcp_vegas_beta = 250
net.ipv4.tcp_vegas_gamma = 200
The target should be 4.6ms of equilibrium.
| 4MB | 8MB | |
| ping delay | 52.79 ms ~ 53.73 ms | 53.18ms ~ 53.9 ms |
| ping variation | 2.6ms ~ 3.18ms | 4.39ms ~ 7ms |
Shall we react slower? Or is the NIC card similar to the old SysKonnect cards?
The same path, against Reno and against Scalable and against BIC. FAST loses out almost completely. One reason is the delay vs loss signal. (Alpha tuning seems to be off in these tests.)
Again FAST is not stable when the sender buffer size is large.
Inter-protocol performance is better since the delay is larger. The loss-based protocol takes longer time to fill the buffer and the buffer size is smaller (max rtt is around 100ms)
FAST is not getting full bw probably because the noise is too much. (RTT is only 11 ~ 12ms in these cases).
Sender's buffer is not enough (8MB); The experiments are incomplete. Skipped.
FAST is oscillating wildly, esp when the UDP traffic is present. Not sure why. But other protocol cannot get high queueing delay most of the time. So I guess the random loss rate is high in this path.
For competition, FAST is better in this case since the RTT is long.
This is a competition test between Reno and FAST (and other protocols), with paths of different delay.
The results are consistent with the expectation: The larger the RTT, the better the FAST against Reno.
The worst case is SNV-Caltech where RTT<26ms but the maximum RTT >120. In this over-over-buffered network, FAST works the worst.
Single FAST vs Reno (from single flow to 8 flows). FAST works as expected but I suspect the sender's buffer of FAST is constrained. (8MB is only enough to support about 35pkt/ms over the 150ms path)
FAST is aggressive against Reno on the path of SLAC-CERN. The problem is probably that the RTT is huge (200ms) and the buffer size is very small -- I don't see queue built up in this path at all.
/proc/sys/net/ipv4/tcp_vegas_alpha = 400
/proc/sys/net/ipv4/tcp_vegas_beta = 250
I think the parameters' value should be switched.
FAST works well.
FAST worked not very well with GVA2. Explanation: "There was a transient problem with the machine at GVA2, as explained after the result."
1. FAST takes around 100 second to reach equilibrium. 100 second = 500 RTT. On average, each RTT grows the window by 32 pkt. This is very small. I expected the queueing delay is very noisy in this path, or the burstiness effect is very significant.
2. Sometimes. FAST takes a long time to converge. The web page explains that it is possibly due to congestion from other users. Another possible contributor is the noisy queueing delay caused by burstiness.
FAST works well between SNV and GVA. But has some oscillation between SNV and Amsterdam (Other protocols are oscillating lower than 800Mbps too).