Fastpass

A Centralized "Zero-Queue" Datacenter Network

Fastpass is a datacenter network framework that aims for high utilization with zero queueing. It provides low median and tail latencies for packets, high data rates between machines, and flexible network resource allocation policies. The key idea in Fastpass is fine-grained control over packet transmission times and network paths.

A logically centralized arbiter controls and orchestrates all network transfers.

Read the paper Get the code

Note (August 2017): Flowtune is developing network monitoring and scheduling based on principles derived from this work. See the website for more details.

Zero network queues

Because the arbiter has knowledge of all current and scheduled transfers, it can choose slots and paths that yield the "zero-queue" property: the arbiter arranges for each packet to arrive at a switch on the path just as the next link to the destination becomes available.

Linux endpoint support

A loadable kernel module implements a Linux qdisc (queueing discipline) that intercepts outgoing packets just before they are passed to the NIC. The qdisc sends requests to the arbiter using specialized control sockets also in the kernel module, implemented as a Fastpass transport protocol.

Arbiter

The arbiter is implemented on the DPDK framework. The arbiter processes each request, performing two functions:

Timeslot allocation: Assign the requester a set of timeslots in which to transmit this data. The granularity of a timeslot is the time taken to transmit a single MTU-sized packet over the fastest link connecting an endpoint to the network. The arbiter keeps track of the source-destination pairs assigned each timeslot.
Path selection. The arbiter also chooses a path through the network for each packet and communicates this information to the requesting source.

Improved fairness

Five rack servers each send a bulk TCP flow to a sixth receiving server. The experiment begins with one bulk flow; every 30 seconds, a new bulk flow arrives until all five are active for 30 seconds, and then one flow terminates every 30 seconds. Even with 1s averaging intervals, baseline TCP flows achieve widely varying rates. In contrast, for Fastpass (bottom), with 3, 4, or 5 connections, the throughput curves are on top of one another. The Fastpass max-min fair timeslot allocator maintains fairness at fine granularity.

Reduced latency

Histogram of ping RTTs with background load using Fastpass (blue) and baseline (red). Fastpass’s RTT is 15.5× smaller, even with the added overhead of contacting the arbiter.

Clone the Fastpass git repository:

git clone https://github.com/yonch/fastpass.git
export FASTPASS_DIR=`pwd`/fastpass

Prerequisites

The DPDK framework for the arbiter (version 16.07 is known to work):
```
git clone git@github.com:DPDK/dpdk.git
pushd dpdk
git checkout v16.07
popd
```
After compiling DPDK (can use tools/setup.sh), set environment variables RTE_SDK and RTE_TARGET, for example:
```
export RTE_SDK=`pwd`/dpdk
export RTE_TARGET=x86_64-default-linuxapp-gcc
```

The Linux kernel, v4.1. Configure and modify it with new pkt_sched.h:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
pushd linux-stable
git checkout v4.1

Create a .config file here (there is an example for Linux v3.10.25 in $FASTPASS_DIR/src/kernel-mod/kernel-config-3.10.25).

unset KBUILD_OUTPUT # build in the source directory
unset KDIR          # build in the source directory
make oldconfig
cp $FASTPASS_DIR/src/kernel-mod/pkt_sched.h include/uapi/linux/
make -j22
popd

Then set environment variables:

export KBUILD_OUTPUT=`pwd`/linux-stable
export KDIR=$KBUILD_OUTPUT

The iproute2 package contains the tools to manipulate qdiscs. It requires flex, bison, iptables-dev, and Berkeley DB. On Ubuntu:
```
sudo apt-get install flex bison iptables-dev libdb6.0-dev
```
Now it's time to get the iproute2-fastpass repository and compile. The tc utility will be in iproute2-fastpass/tc/tc.
```
git clone http://github.com/yonch/iproute2-fastpass
pushd iproute2-fastpass
make -j22
popd
```

Fastpass: A Centralized “Zero-Queue” Datacenter Network, to appear in SIGCOMM'2014, Chicago, IL, August 2014 (bibtex).

No-wait data centers, Larry Hardesty, MIT News Office
MIT invention to speed up data centers should cheer developers, Stephen Lawson, TechWorld
Kill queues for fast data centres: MIT boffins, Richard Chirgwin, The Register
MIT, Facebook Create Low-Latency Fastpass TCP Replacement, George Leopold, EnterpriseTech
Fastpass: Centralized network arbitration eliminates data-center latency, Michael Kassner, Tech Republic
This could be the data centre breakthrough network admins have craved, Vawn Himmelsbach, expertIP
10 cool network and computing research projects, Bob Brown, Network World
Estrategia para una navegación más rápida en grandes webs, Noticias de la Ciencia y la Technologia (Spanish)
Hacker News discussion
Solidot post (Chinese)
Habrahabr post (Russian)

fastpass@mit.edu
Fastpass developers.

Fastpass

A Centralized "Zero-Queue" Datacenter Network

Zero network queues

Linux endpoint support

Arbiter

Improved fairness

Reduced latency

Download

Prerequisites

Papers

In the news

The Team

Contact