Session 7, Paper 1: RotorNet: A Scalable, Low-complexity, Optical Datacenter Network

William M. Mellette, Rob McGuinness, Arjun Roy, Alex Forencich, George Papen, Alex C. Snoeren, and George Porter (UC San Diego)

[Link to Paper]

William Mellette proposes a new switching model called RotorNet, which is a "future-proof" switching model. Bandwidth has been increasing in production data-centers, and 100+ petabit/second networks are projected to be needed in coming years. The challenge is to deliver low-cost bandwidth at scale. This has spurred work for new protocols, new topologies, and new hardware. However, all this work has fixed the switching model. Bandwidth demands for data-centers are doubling every year, and our current switches are not improving fast enough (doubling every 2 years). Electronic packet switches have I/O limits, whereas optical circuit switches can scale to Tb/s. 

However, optical switches have significant control complexity. The "Rotor" switching model eliminates this complexity. Rotor switching makes two changes to the crossbar-switching model:

1. For an N-port switch, the switch can only select from N-1 matchings.
2. The switch's matchings are run on a fixed schedule.

The implementation of an optical rotor switch is simpler than an optical crossbar, as mirrors only have to select from hard-coded matchings rather than implementing the matchings themselves. For a 2048-port switch, an optical crossbar switch would need 4,096 mirrors that could tilt in 2,048 directions while an optical rotor switch would need 2 mirrors that could tilt in 16 directions. This does come with a concern for network underutilization. Mellette proposes RotorLB as an adaptation between 1- and 2-hop forwarding, that defaults to 1-hop forwarding and sends traffic over 2-hops when there is extra capacity. Mellette goes on to show how Rotor switching performs:

This work is significant as it proposes a model that greatly simplifies the implementation of optical switching networks. To address the issue of scaling, Rotor matchings are distributed among multiple switches, making rotor switches easier to build. Below is an image of a prototype Rotor switch built from commodity hardware:

To wrap up, RotorNet is a promising way to scale datacenter networks to the bandwidth that we will need in the next few years.

Q: How do the switches stay in sync?

A: We just need to ensure synchronization between the switches, and they follow the set schedule for rotating matchings.

Q: What is the ordering of packets that are transmitted?

A: There will be reordering (due to traffic taking 1 or 2 hops), our approach is to have a reorder buffer at the receiver.

Q: What is the format that packets are stored in buffers?

A: We hash the packets to be stored on the end hosts.

Q: How do you do 2000 ports? Do you still only use 2 mirrors?

A: There are a number of different ways to implement the rotor switch at the physical layer. At 2000 ports  you can have a large mirror or a mirror array. Options are being investigated.

Session 5 Paper 2: Neural Adaptive Video Streaming with Pensieve

Presented by: Hongzi Mao

Authors: Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh (MIT Computer Science and Artificial Intelligence Laboratory)

Today’s video streaming relies on adaptive bitrate (ABR), e.g. 240P or 1080P, which is selected based on the network condition. The quality of a video is lower with low ABR but an ABR that is too high for the network condition to support would result in video pause. The authors proposed Pensieve, which adjusts ABR based on reinforcement learning on the network conditions and the resulted video quality under the selected ABR.

In this reinforcement learning problem, the action space is the ABR selections, e.g. 240P or 1080P. The reward function considers bitrate, rebuffering, and smoothness. For the state space, many features are considered, including chunk throughput, chunk download time, next chunk size, current buffer size, and past chunk bitrate, etc. These diverse features in the state space would be more helpful than mere throughput prediction and/or buffer occupancy in prior works.

They trained and tested over real network traces and find Pensieve would deliver 12-25% better QoE,  and 10 - 30% less rebuffering than previous ABR algorithms.

Q: How do you explain and understand where the benefits of your reinforcement learning algorithm come from?

A: Explaining the neural network remains a hard problem. We find Pensieve benefits from better control on rebuffering.

Q: What is the cost of computation?  

A: Storage cost is small. Training requires expensive computation but not much computation is needed for ABR selection based on the trained model.

Q: Do you compare with past works on model based congestion control?

A: It is hard to model the network and therefore we propose data driven congestion control.

Q: Did you try user satisfaction for the reward function?

A: No, because it is harder and slower to quantify user satisfaction than our simulation strategy, but it would be trivial to replace the function.

Q: How does it scale to many clients?  

A: We can learn for different clients and maybe coordinate the ABR for multiple clients.

Paper 3: Re-architecting datacenter networks and stacks for low latency and high performance

Paper 3: "Re-architecting datacenter networks and stacks for low latency and high performance" - authors are Mark Handley (University College London), Costin Raiciu, Alexandru Agache, and Andrei Voinescu (University Politehnica of Bucharest), and Andrew Moore, Gianni Antichi, and Marcin Wójcik (University of Cambridge)


The paper is presented by Prof. Mark Handley (University College London). This paper has received a SIGCOMM 2017 Best Paper Award.

The talk starts by describing three goals in DC networks:

1. low latency between hosts
2. receiver prioritisation
3. predictable high throughput

Although, achieving these simultaneously is challenging. The authors present NDP, a DC protocol architecture that simultaneously achieves low latency and high throughput.

One of the key ideas in this paper is the use of packet trimming when a switch queue fills up. This idea proposed as Cut Payload (CP) in this paper (worth checking out):
P. Cheng, F. Ren, R. Shu, and C. Lin. Catch the whole lot in an action: Rapid precise packet loss noti cation in data centers. In Proc. Usenix NSDI, 2014.

In NDP, the authors addressed the shortcomings of CP  with the following changes:

1. An NDP switch maintains separate queues for data packets and higher priority trimmed header packets. This provides low latency similar to lossless Ethernet, without the collateral damage caused by pausing.
2. An NDP switch performs weighted round robin between the high and low priority queues. This can eliminate congestion collapse.
3. An NDP switch decides wether to trim a newly arrived packet or the data data packet at the end of the low priority queue. This breaks up phase effects.

On top of presenting a switch queuing algorithm, NDP proposes a per-packet multipath forwarding and a novel transport protocol.

Implementation: the authors presented NDP in Linux hosts with DPDK, in a software switch, in a NetFPGA-based hardware switch, and in P4.

Results:

• NDP provides very low flow completion time.
• NDP provides much better isolation between different workloads than mechanisms such as DCQCN (that rely on lossless Ethernet for low delay).
• Incast: While other solutions have also tackled this problem well (see figure below), the important part is that NDP solves the incast problem using only 8 packet buffers.

You can check out the great animations that were presented during the talk here.


The talk was followed by a lively Q&A, with questions on fairness (NDP ensures fairness), network topology (NDP requires a Clos topology) and failure awareness (NDP handles link failures by maintaining scores for links).

Technical Session 5 Paper 3: Disk | Crypt | Net: rethinking the stack for high-performance video streaming

Authors:  Ilias Marinos, Robert N.M. Watson, Mark Handley and Randall R. Stewart

Presented by: Ilias Marinos (Cambridge)

    

Ilias starts the presentation by emphasizing the increase in popularity of video traffic in the internet, stating it accounts 50% of all North American traffic. He switches the focus from video popularity to how most of the video being delivered is now encrypted. The authors study the various kernel optimizations made by Netflix and show using the figure below how the throughput for encrypted video is 30% less for when all the content is in the buffer cache compared to when all the data is fetched from disk . The authors state using the buffer cache entirely causes core saturation leading to drop in performance. The reason disk is better is due to the decrease in storage latencies due to SSDs. 

This motivates the author to build a system called Atlas which ideally does not use any memory and fetches the content straight from disk to the NICs. Atlas puts SSDs directly in TCP control loop by processing disk reads to completion and then transmiting. Atlas uses diskmap (kernel bypass framework) to achieve this design. Ilias states that this design is slightly far off from the ideal case due to diskmap limitations.

 Atlas outperforms the Netflix stack by 15% for unencrypted and 50% for encrypted traffic in terms of throughput with half the number of cores used by Netflix.  The number of memory reads is half for each packet sent. Their solution does involve some memory use particularly due to inefficient re-use of buffer cache limited by diskmap.

Question & Answers: 

Q. Did you consider cache partitioning? offloading encryption on to NICs? What would be the next important optimization?
A: Offloading to NICs would help but the next thing for us to look would be how we can get pass the dismap limitations to re-use buffers more efficiently then waiting for the content to be copied to and from the memory due to buffer evictions. 

Session 5 Paper 1: The QUIC Transport Protocol: Design and Internet-Scale Deployment

Presented by: Jana Iyengar

Authors: Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, and Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, Jeff Bailey, and Jeremy Dorfman, Jim Roskind, and Joanna Kulik, Patrik Westin, Raman Tenneti, Robbie Shade, Ryan Hamilton, Victor Vasiliev, Wan-Teh Chang, and Zhongyi Shi (Googlers or ex-Googlers)

QUIC is Google’s HTTPS protocol since 2014. It is between Google services and the Chrome/mobile apps. QUIC improves application performance, reducing youtube video re-buffering by 15 - 18% and Google's search latency by 3.5 - 8%. QUIC handles 35% Google’s egress traffic, or 7% internet traffic. QUIC stopped servicing traffic during December 2015 - January 2016 due to a security bug, but the amount of traffic served increased drastically in August 2016 when mobile Youtube started to run on QUIC. Google has also founded an IEFT working group formed in Oct 2016 to modularize and standardize QUIC.

QUIC is an integrated replacement for TCP + TLS + HTTPS. A lot of cross-layer optimization strategies have been adopted to improve performance and reduce overhead, such as leveraging flow control over a new transmission abstraction (QUIC stream) to avoid Head-of-line blocking on the TCP layer. QUIC is also developed in user space for better tools and practices, for better integration with tracing and logging, and for quicker deployment and evolution.

The performance improvement of QUIC mainly comes from the tail: for good connections with low latency, the space for improvement is small. However, for poor connections with larger latency, QUIC helps. The benefits come from reduced handshake latency and faster loss recovery, which results from the cross-layer design of QUIC.

Q: QUIC completely removes TCP. Should congestion control be moved from TCP to QUIC?

A; QUIC already has the Cubic congestion control algorithm.

Q: How about fairness? What happens when a QUIC session and a TCP session coexist?

A: Fairness depends on the congestion control algorithm (e.g. CUBIC). QUIC should be TCP friendly since QUIC and TCP both use the Cubic congestion algorithm.

Q: What is the cost of infrastructure to deploy QUIC? Some TCP features are already enabled on NICs. However, QUIC may not be able to leverage these existing features in the hardware.

A; We find the performance improvement overweights.

Q: Can a QUIC session be broken up to utilize between WiFi and cellular network?

A: This would be future work.

A Formally Verified NAT

Authors:

Arseniy Zaostrovnykh*

Solal Pirelli*

Luis Pedrosa*

Katerina Argyraki*

George Candea*

*EPFL, Switzerland

http://delivery.acm.org/10.1145/3100000/3098833/p141-Zaostrovnykh.pdf?ip=128.97.8.14&id=3098833&acc=OPEN&key=CA367851C7E3CE77%2E79535EF926D6BC05%2E4D4702B0C3E38B35%2E6D218144511F3437&CFID=975190330&CFTOKEN=93017280&__acm__=1503375257_d9b9bc2b9428feb87b2eb0fdf116d348

As Arseniy stated, this is a work that applies directly to the software network functions rather than assuming models of them.

The contribution of the paper is a Network Address Translator (NAT) function (NF) written in C whose correctness is formally verified as being in line with the semantics specified in RFC3022.

This approach uses different verification techniques for different parts of the code (a new combination of symbolic execution and formal proof checking using separation logic), and combines the results through so-called "lazy proofs" (a lazy proof consists of sub-proofs structured in a way that top-level proofs proceed assuming lower level properties, and the latter are proven lazily a posteriori). All these make it a novel work which aims to improve the state of the art on two fronts: (1) verify high-level semantic properties, such as the correct implementation of an RFC, and (2) verify the stateful property (that Network Functions that are stateful). 

To verify this NAT, the authors developed a verification toolchain, called Vigor.

Experimental results demonstrate the practicality of the in question approach: the verified NAT box performs as well as an unverified Data Plane Development Kit (DPDK) NAT and outperforms the standard Linux NAT. 

———————

Questions & Answers:

Q: How difficult is to generalise for mor RFCs?

A: While it took us 3 man days for the RFC3022 specifically, can't say for sure about how such generalisation could be implemented. May be some future work could consider a kind of automatic parsing of these specifications.

Q: How long would take to verify the whole data plane 

A: Most of the network verification projects use SAM simplified representation of middleboxes which take these models and specifications and verify the network function code. However these models are just abstractions of the real network and not the "real code" and this is where we can fit.

Q:  Do the programs have loop in them or short bounded loops?

A: Yes, it is important that they might have single loops, however we can deal with other data structures and apply our algorithm again.

Paper 3: Pretzel: Email encryption and provider-supplied functions are compatible

Paper 3: "Pretzel: Email encryption and provider-supplied functions are compatible" by Trinabh Gupta, Henrique Fingler, Lorenzo Alvisi, and Michael Walfish.

Trinabh Gupta presented the Bretzels that proposes an end-to-end encryption mechanism between email servers while not compromising essential functionalities such as spam filtering.

Gupta et. al. claim that emails today are not encrypted, e2e, end-to-end (between clients), however  intermediate servers are able to handle these emails in plain text. This has been accepted to offer well-run services. The presenter says that e2e encryption breaks the businesses model (extract user interests, make targeted adv). On the other hand if mail servers can access email then hackers can access emails.

Pretzel establishes end-to-end encryption without compromising the benefits of such services. Their main design objective include e2e encryption, enabling basic services, and achieving low resource costs. Pretzels proposes 2PC solution that  protects both the user's and the server's content (the filter and the email content) Gupta says.

Authors gave the example of sharing salary between two entities. both users gives their salaries to a black box that will allow the exchange.

However, Existing 2PC solutions such as Yao 2PC are very costly mainly because of the size (1 Million of rows and probabilities to compute). Pretzel reduces this cost by 100x.

They test two function services spam filter and topic extraction which implements
linear classifiers (extract words, add properties, compare probabilities). Pretzel does such classification privately.

To reduce the cost, Pretzel adapts packing to reduce client storage cost. It concatenates probabilities before encrypting. It also implements a decomposed classification.

Authors compare Pretzel to a no private system and Yao+GLLM. They measure resource cost. They show that Pretzel achieves 100x less (compare to Yao) CPU-time at the server, 100x less traffic at the network, but with a cost of 3x more storage at the client side. 

Q&A session (2 questions)

Q1: If I organize a protest, I don't want the email provider to know about my topic. How can your model preserve privacy or what's the middle ground.
A1: Client can state that they do not allow topic extraction. Client preferences can be set at the end hosts. Author said that their solution is much better than existing but doesn't fix all issues.

Q2: In your work, you had to change the classifier, how different is your classifier from existing services and what's the impact / tradeoff on performance.
A2: We can implement better classifier using neural networks but performance remain similar (80 to 90% similar).

Keynote: Hitting the Nail on the Head: Interdisciplinary Research in Computer Networking -- Prof. Jennifer Rexford

Jennifer Rexford

2016 ACM Athena Lecturer Award Winner

• The internet is one of the most influential inventions.
• However, we cannot do it alone.
• Hammers (techniques) and Nails (problems).

• Project 1: Protocols as Distributed Optimizers
• Traditional traffic management has architectural limitations -- slow adaption
• TCP
• Forward engineering of traffic management
• Decompose to generate a distributed solution
• Lesson: 
• Start with a well-stated problem, then decompose into a distributed solution
• Research as decomposition
• People and timing

• Project 2: Composition of Network Policies
• Simple, open data-plane API -- openflow
• Moduler controller applications -- each module partially specifies the handling of the traffic
• Abstract openflow -- policy as a function
• Lesson: 
• Abstraction and decomposition; 
• Embedded people (and code)

• Project3: Traffic monitoring in the data plane
• Traffic analysis in the data plane
• Streaming algorithms
• A great opportunity
• Example: heavy-hitter detection
• Approximating and approximation
• Lesson: 
• Getting concrete
• Computational model
• Example problem
• Strawman solution
• Iteratively designing
• Striving for general understanding

Should we all do interdisciplinary research?

• Interdisciplinary fun
• Bigger impact & social fun
• Managing risks
• One vs many hammers
• Steep learning curve
• Learn both the hammer and the nail
• Learning the culture of both research fields
• Join an emerging community
• Credit for the work
• Healthy interdisciplinary collaborations

Q: We have various network devices, what is the network constrained by now? What is the next problem in the network?

A: There are still a lot of things to understand. For example, which things should be where.

Session 3, Paper 3: SketchVisor: Robust Network Measurement for Software Packet Processing (Huang, Jin, Lee, Li, Tang, Chen, Zhang)

Network measurement is important for identifying heavy hitters, traffic anomalies, flow distribution,
and traffic entropy. Previous approaches (sketches) offer a way to summarize traffic statistics of all packets within a fixed-size structure, at the cost of small errors. However, when implemented in practice they consume thousands of CPU cycles.

Huang proposes to separate the control plane and data plane. The idea is to keep user-defined sketches which achieve high accuracy but are relatively slow, and add a fast path. The fast path is high speed, is general for multiple sketches, but is relatively less accurate. The control plane is used to recover the information lost on the fast path. This recovery is transparent to users, who simply deploy the desired sketches. Since an ideal Fast Path algorithm is infeasible with limited resources, SketchVisor offers a practical algorithm that achieves near-perfect accuracy.

The authors compare SketchVisor to the Misra Gries top-k algorithm. SketchVisor has much fewer kick-outs (most expensive operation), resulting in much faster throughput. While error rates increase for Misra Gries as the number of flows increases, it stays the same with SketchVisor. A prototype was implemented based on Open vSwitch, and reached speeds of ~10Gbps in testbed, and ~20Gbps in simulation.

Q: The solution is proposed for switches with limited memory. If we are not concerned about memory, what's the motivation for SketchVisor?
A: SketchVisor also achieves high accuracy and high throughput (due to less CPU usage).

Q: Fast path is able to detect heavy hitters. Why not get rid of sketches altogether? 
A: Fast path is less accurate than sketches and is used only to compliment sketches.

Q: The authors make some assumptions about matrix sparsity in the paper. How does the recovery algorithm work without this assumption?
A: We evaluated many matrices and showed that most of them fall into our assumption.

Session 3 Paper 2: Quantitative Network Monitoring with NetQRE (Yifei, Dong, Ankit, Sajal, Rajeev, Boon)

Presenter: Yifei Yuan
Authors: Yifei Yuan, Dong Lin, Ankit Mishra, Sajal Marwaha, Rajeev Alur, Boon Thau Loo

Dynamic updates are important to today's network management because both of traffic engineering and security events require timely response. To achieve this, quantitative network monitoring(including monitoring metrics and computation of values) is required. However, today's tools suffer from some limitations, such as focusing on low-level measurements, only providing ad-hoc solutions, or requiring significant programming expertise from network operators.

Yifei Yuan presented NetQRE, a language which can provide high-level abstractions. In the NetQRE language, expressions in stream functions are based on quantitative regular expressions(QRE). He also presented the compiler for NetQRE, which generates efficient NetQRE implementations automatically. In order to run with low memory footprint, they address challenges for evaluating each incoming packet without storing the history.

Yifei Yuan also presented several use cases including:
1. Flow-level traffic measurements
2. TCP state monitoring
3. application-level monitoring

Yuan et al. evaluated several aspects of the NetQRE prototype including the expressiveness of the language, performance in terms of throughput and memory footprint, and its capability for real-time monitoring setting. By comparing NetQRE with other tools, it is shown that NetQRE can effectively express a wide range of quantitative monitoring applications. It outperforms equivalent implements in other tools. And it can also support timely monitoring requirements.

You can check out this paper here.

Q&A:
Q: Can you compare your paper with the previous paper(Language-Directed Hardware Design for
Network Performance Monitoring)?
A: The Marple language uses key-value stores, but they still keep packet states. However in our case, we keep states across a sequence of packets. This is the benefit of using quantitative expressions instead of key-value.

Session 3 Paper 1: Language-Directed Hardware Design for Network Performance Monitoring (Srinivas, Anirudh, Vikram, Prateesh, Venkat, Mohammad, Vimalkumar, Changhoon)

Presenter: Srinivas Narayana

Authors: Srinivas Narayana, Anirudh Sivaraman, Vikram Nathan, Prateesh Goyal, Venkat Arun, Mohammad Alizadeh, Vimalkumar Jeyakumar, Changhoon Kim






Performance monitoring is important to quickly localize many network problems. While endpoints-based approach can be used for network monitoring, it lacks visibility to localize problems at links deep. Switch-based approach, however, can provide more direct visibility. But traditional mechanisms can't collect enough information or provide relevant performance data, and are thus restrictive.

To achieve more flexible performance monitoring, Srinivas Narayana presented a language-directed hardware design. They designed the language Marple to express monitoring use cases and switch hardware primitives for this language. The operator writes query in Marple language, which will be compiled into a switch program. The program will run on the programmable switches. The operator can get results from the collection servers which receive streams from switches.

Narayana et al. evaluated Marple in terms of hardware compute resources, memory and bandwidth overheads. Srinivas also presented two use cases of Marple: debugging microbursts and flowlet size distributions. It is shown that Marple is effective for fine-grained and programmable network monitoring. And it only requires modest amount of hardware resources. 

This paper received Best Paper Awards this year. You can check out this paper here. 

Q&A:

Q: When you finish processing the query, do you explicitely deallocate things from the backing store.
A: That could be part of the system, like allocating or deallocating memory for each query. If you have a continues query system, then potentially you need to have some way to archive that. If you know you just want to run it for a specific period of time, you could deallocate them once you write them off.

Q: Have you thought about security problems of programming inside network? Because having stuff out there that can be programmed without control over who writes what programs can lead to all sorts of nasty problems.
A: In this case, the operators who run the network are the ones running the programs. This is very different from a user, or a public customer writing programs on switches. But in general if you have a more flexible system in the network, there is a chance that you are exposed to attacks.

Session 3, Paper 4: Constant Time Updates in Hierarchical Heavy Hitters (Basat, Einziger, Friedman, Luizelli, Waisbard)

DDoS are increasing, so an efficient way to identify and act upon them is needed. Previous work, namely Hierarchical Heavy Hitters (HHH), propose algorithms whose update complexity increases with the hierarchy’s size. Specifically, for each packet, the approach is to look at the source IP and compute all IP prefixes. Such a solution is not ideal as hierarchy size will keep on increasing with the adoption of IPv6. Other emerging trends such as NFV further motivate the need for fast software
based measurement algorithms.

In light of this, Basat proposes to compute a random prefix only. This reduces the run-time since you only count 1 prefix, regardless of the hierarchy size. An additional speedup is gained via sampling. For instance, each incoming packet is ignored with some probability, and the rest of the packets proceed to have their prefixes computed. The idea is that if enough packets are seen from attacking networks, they will be sampled enough times for accurate detection.

The system was implemented in Open vSwitch (OVS) and the results showed that after enough packets there are no false negatives, no counting errors, and only a few false positives.

Accuracy improves with the number of packets. Converges is reached after ~32M packets. If sampling is used, convergence is reach in ~128M packets. After this many packets, the proposed solution performs on par with previous approaches in terms of accuracy. However, Basat's solution achieves up to 62 times speedup. Their performance is close to the vanilla OVS and provides a 250% throughput improvement compared to other approaches.

Q: How do you do randomization between different hierarchies? Is uniform at random the best option?
A: We need the same accuracy for every hierarchy. We thought about variant randomization, but could not justify pursuing this direction is we could not find any data found to convince us that this is needed. Perhaps this is a good direction for future work.

Q: What happens if you add sampling to previously proposed algorithms? Does the throughput performance improve?
A: It is true that most of the throughput improvement is achieved thanks to sampling. However, previous approaches also suffer from non-constant worst-case update times.

Session 2, Paper 1: NFP: Enabling Network Function Parallelism in NFV (Sun, Bi, Zheng, Yu, Hu)

Presenter: Chen Sun

Chen sun introduces how networks have changed from the usage of middleboxes to the use of service chains. Sun et al, argues that sequential service chaining in network function introduce latency and overhead in NFV. He proposes the use of virtualization techniques that exploits low cost, flexibility, and scalability. There have been recent networks that reduce NFV latency such as ClickNP NetBricks and OpenBox via a horizontal sequence.

Sun et al implements an NFP framework that enables network function parallelism to improve NFV performance.  The framework consists of three main components: policy specification scheme, NFP orchestrator and NFP infrastructure.

The Policy Specification Scheme applies an intuitive graph description. This involves a step-by-step process to order the packets, set the priority of the packets and position the packets. The Orchestrator uses a three-stack workflow in handling the packets. This involves NF Dependency Identification (Add/drop, read, write, drop a packet), Resource Overhead Optimization and Service Graph Construction. The NFP Infrastructure aspires to overcome four main challenges: Packet Copying, Packet Delivery, Packet Merging and Packet Dropping. For example, to tackle packet delivery, a distributed packet delivery process is deployed to deliver more packets (example four packets) at the same time as opposed to central packet delivery system. This results in a higher number of packets being delivered.

The authors have implemented a prototype of the NFP framework called OpenNetVM, which showcases the benefits and limitations of NF parallelism. The NFP framework results in a slightly higher latency rate than that of the service chain but has an improved rate of distributed packet delivery.

There is a 53.5% of NF pairs that can run in parallel and 41.5% of NF pairs can be parallelized without causing extra resource overhead. Therefore, from these calculations, we can extract the opportunities posed by the NFP framework to increase the performance of NFV.

Questions posed:

Q: In NF Parallelism with branching, when one packet is dropped by one branch, what are the actions of the other branch?

A: The NF Analysis process is used to detect which packet has been dropped. From there, the framework considers what actions to take and solve the problem. This is done via the identification algorithm.

Q: With branching in NF Parallelism, how does the system handle packets with branches operating at different speeds?

A: Through Packet Tagging, we identify, copy, merge and determine if we should drop the packets.

In the future, we will add branching policies and implement optimization.

Q: How do NF read/write information to packets with the use of 3^rd parties?

A: The programming interface can be used to identify the actions and detect network functions.

Related work:

• ·         Bath processing (e.g NetVM [NSDI’14], Intel DPDK)
• ·         Parallel processing pf NF building blocks (e.g CLickNP [SIGCOMM’16])
• ·         Parallelism between match-action tables (e.g P4, RMT [SIGCOMM’13])
• ·         Module composition in parallel in SDN (e.g Pyretic [NSDI’13])

Session-2, Paper-3, NFVnice: Dynamic Backpressure and Scheduling for NFV Service Chains

Authors: Sameer G Kulkarni, Wei Zhang, Jinho Hwang, Shriram Rajagopalan, K. K. Ramakrishnan, Timothy Wood, and Mayutan Arumaithurai and Xiaoming Fu.

Presenter: Sameer G Kulkarni

Presentation: Presenter first motivates by stating that NFV applications are growing and popular and then highlights the problem as they do not scale. These NFs are diverse that require fair scheduling and efficient service chaining. Their diversity stems from their processing and performance requirements. Some middleboxes operate in Mbps speed, while others can only process in Kbps speed. It has been found that these NFs cannot achieve (1) fair scheduling: standard CPU schedulers do not have sufficient information to allocate resources in a way that provides rate-cost proportional fairness. CPU schedulers usually provide fair allocation of processing time, but if computation costs vary between NFs this cannot provide rate-cost fairness. (2) efficient service chaining: Combination of a number of NFs into service chains demands careful resource management across the chain to minimize the impact of bottlenecks. Processing a packet only to have it dropped from a subsequent bottleneck’s queue is wasteful, and a recipe for receive livelock

To address these issues, this paper proposes NFVnice (a user space control framework for scheduling NFV chains). It provides fair and efficient resource allocations to NF service chains. The idea is based on assisted preemptive scheduling, where network functions provide hints to the underlying OS with regard to their utilization. To address efficient service chaining, NFVnice leverages queues between NFs in a service chain to know when NFs in a chain are overloaded or blocked in their operations.

The key components are (1) cgroup: that limits the resources for each user. This paper uses weigh computation algorithm and updates the results every 10ms. (2) Backpressure: that does selective per chain backpressure marking. The proposed mechanism is different from traditional back-pressure in a way that the this paper marks the packet flow (rather individual packets) to drop it at the source.(3) ECN: that signals the congestion, and (4) I/O management library that abstracts implementation complexities from the NF implementation.

The results show that NFVnice can achieve throughput improvement upto 2 times, and avoids CPU wastage through backpressure; and improves resource utilization through scheduling.

No question from audience.

Session 1, Paper 2 - SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs. (Miao, Zeng, Kim, Lee, Yu)

Rui Miao states that layer 4 load balancing is critical to ensure timely service availability and 40% of the datacenter traffic needs load balancing. Current software load balancers incur high cost, need more servers to scale out, experience high latency for high traffic volume even with optimization techniques like kernel bypassing. They also have poor isolation performance in the face of an attack. These software based load balancers can provide Per Connection Consistency (PCC) but cannot scale to traffic growth. On the other hand, partial offloading based load balancing can scale to traffic growth but cannot guarantee PCC. The authors propose Silkroad that provides the best of both worlds; it is built on switching ASICs providing multi-Tbps and also guarantee PCC.

Silkroad stores the mapping of Virtual IP addresses (VIP) of services to Direct IP addresses (DIP) of the servers in the switching ASICs SRAM as a connection in a connection table. Since DIP updates are pretty common i.e. 100 updates a minute, the challenge is how to store millions of connections in a connection table with limited SRAM. Silkroad employs a novel hashing design to compress a connection table and optimize the usage of SRAM. The other challenge for Silkroad is to do all operations and ensure PCC in just a few nanoseconds; it uses hardware primitives to handle connections, their state and dynamics. 

Silkroad is implemented in a programmable switching ASIC with 400 lines of P4 code.  The control plane functions are written with 1000 lines of C code on top of switch driver software. The authors also prepared a demo on a Tofino programmable switch. They are able to achieve full 6.5 Tbps line rate by replacing 100s of software load balancers with one Silkroad. The processing ingress to egress latency is in sub microseconds while guaranteeing PCC. They also establish resilience against attacks and hardware rate limiters for performance isolation.

In short, Silkroad was able to,

• provide a direct hardware path to the application traffic towards the application servers
• scale for traffic growth with switching ASICs
• optimize the usage of SRAMs to store connections
• ensure PCC under frequent DIP pool updates
• 100 to 1000 times savings in power and capital cost

Some of the questions that the author encountered are,

Q: How do you remove connections from a connection table?

A: Use a hardware timer timeout to scan all entries in the connection table and report the deleted entry to the software where software can delete the entry. This process has fewer bits per entry as a  memory footprint.

Q. If a new rack is introduced in a server, how would the existing connections change?

A. Different load balancers can route the exiting connections to the new racks where they will be treated as new connections, and PCC can be violated just as it can happen with software load balancers.

Session 1, Paper 1 - dRMT: Disaggregated Programmable Switching (Sharad Chole, Andy Fingerhut, Sha Ma, Anirudh Sivaraman, Shay Vargaftik, Alon Berger, Gal Mendelson, Mohammad Alizadeh, Shang-Tse Chuang, Isaac Keslassy, Ariel Orda, Tom Edsall)

Anirudh  proposes a new architecture for programmable hardware, dRMT (disaggregated Reconfigurable  Match-Action Table).  dRMT design aims to overcome the two restrictions and its drawbacks of RMT:

1. Each match-action pipeline stage in RMT can only access memory local to it, implying that the memory not used by a stage cannot be allocated to other match-action stage, therefore poor memory utilization.
2. RMT execute operation only in a fixed order: a match followed by an action in each stage. The authors argue, such fixed functionality lead to under utilisation of hardware resources for programs where matches and actions are imbalanced. 

The dRMT's key idea is to disaggregate hardware resources i.e., memory and compute of a programmable switch.

1. dRMT separates memory for tables from the processing stages and allows to access them using cross bar. This cross bar carries information such as search keys and search results between match/action stages and memory. 
2. Pipeline stages in RMT are replaced with a set of match-action processors. Similar to RMT, these processors has match-action units. But, unlike RMT,  packets do not move between pipeline stages. Instead, a packet is sent to dRMT match-action processor in round robin fashion, and entire p4 program for that packet is executed to its completion.

Then, Anirudh address the following 3 questions:

1. How to schedule entire system (memory and processors) at compile time that guarantees a deterministic throughput and latency?
2. How does dRMT compare with RMT on real programs?
3. Is dRMT feasible?

For more details, please check out the paper.

dRMT is evaluated using 3 switch.p4 programs and one p4 program from a large switch asic manufacturer. The results shows significant  improvements over RMT --- dRMT chip requires 4.5%, 16%, 41%, and 50% fewer processors without compromise on line rate. Even for 100 randomly generated p4 program that has similar switch.p4 program characteristics,  dRMT chip require 10% lesser number of processors. With less number of processors, the throughput degradation of dRMT is graceful, where as RMT's performance falls off a cliff if a program need more pipeline stages.

Though the work has no working chip, it presents a design for dRMT chip and analysed its feasibility and chip area cost. In specific, dRMT require some additional chip area to implement a crossbar and match-action processors. For example, a 32 processor dRMT chip need an additional 5 mm^2, a modest increase when compared to 200 mm^2 typical switching chip.

The bottom line,  Anirudh says:
- Disaggregation of hardware resources improves utilization and throughput.
- dRMT require same chip area supporting even more complex operations.
- More details at http://drmt.technion.ac.il/

Q&A:

Q1: While scaling wiring may not be a problem. It was observed that the problem is with latency introduced by the cross bar. Is there any study on latency?
Anirudh: Yes, latency does goes up with cross bar -- by a few clock cycles (4) in dRMT.

Q2: Whether RMT was just a reference model, which could be implemented in hardware using memory disaggregation,  just like dRMT does?
Anirudh: High performance programmable switching chips that the authors know of do not disaggregate memory and have memory that is local to each stage.

Q3: Whether RMT could benefit from having heterogeneous stages, where each stage had a different ratio of match:action:memory capacity?
Anirudh: There is significant hardware design effort involved in designing a single RMT stage. So you want to amortise that design cost by replicating the same design for all stages and making the stages homogeneous.

Opening Session + Awards

Opening

• Event is a success. A lot of people registered for the conference. 
• All proceedings are online. Papers will be available online for perpetual access.
• Big thanks to the supporters. Visit the Illumination Rooms for exhibits and demos.
• Thanks to NSF for student grants, UCLA and the organizing committee.
• Thanks to authors, TPC members, tutorial speakers, and participants.

PC Chairs:

• Conference went back to single-tier PC model.
• 30% of the members were first-timers.
• Members reviewed 20-25 submissions and attended PC meetings.
• Submissions:
• 250 submissions (10% more than in 2016)
• 15 authors submitted more than 4 papers. 1 author submitted 7 papers.
• Reviewing process:
• Round 1: received 3 reviews. Submission moved on if more than 1 positive review.
• Round 2: 127 Submission received additional 3 reviews. 
• Round 3: 70 submissions received additional review by an expert (>=3 expertise).
• In-person PC meeting
• 1.5 day PC meeting at UCSD
• 65 papers discussed with 6-8 reviews for each paper
• 36 accepted papers (3 experience track papers)
• Each paper was shepherded by PC

Score vs #Authors:

• Single authored paper had a very hard time.

Conference:

• 215 authors got their submissions accepted.
• 40 academic institutes and 20 companies, 15 different countries on 5 continents
• 36 accepted papers. 2 CCR papers

Awards

Best Paper Awards:

(Chair: Dina Papagiannaki)

PC members nominate papers. Chair authored papers are not eligible.

The following papers received the best paper awards

• Re-architecturing datacenter networks and stacks for low latency and high performance
• Handley et al.
• Language-directed hardware design for network performance monitoring
• Narayana et al.

Test of time awards:

Has to have been accepted at an ACM conference before. Awarded to papers that have impacted a lot in the area.

      2017 Test of Time Awards:

• Ethane: Taking control of the Enterprise (SIGCOMM 2007)
• Casado et al.
• impacted the software-defined networking paradigm
• Measurement and analysis of online social networks (IMC 2007)
• Mislove et al.
• made an early contribution in the field of social networks

      2016 Test of Time Awards:

      Committee: Ratul Mahajan, Dina Papagiannaki, Jennifer Rexford, Vyas Sekar.

• Link-level measurements from an 802.11b mesh network (SIGCOMM 2004)
• Aguayo et al.
• A first-principles approach to understanding the Internet's router-level topology (SIGCOMM 2004)
• Li et al.

SIGCOMM Doctoral Dissertation Award:

for dissertations that were completed in 2016

• Middleboxes as a cloud service
• Justine Sherry
• Power, Communication and Sensing Solutions for Energy Constrained Platforms
• Vamsi Talla

SIGCOMM Rising Star Award:

Committee is still working on the nominations from last year.

Conference Organization Awards:

SIGCOMM 2017 General Chairs:

K. K. Ramakrishnan, Lixia Zhang

SIGCOMM 2017 Program Chairs:

Alex Snoeren, Walter Willinger

SIGCOMM Award Chair 2011-2017:

Bruce Maggs

SIGCOMM Conference Coordinators 2013-2017:

Yashar Ganjali

SIGCOMM Information Services Director:

Hamed Haddadi, 2013-2017

Recognizing exceptional service to the SIG:

Chris Edmondson-Yuranan

        Chris Edmondson-Yuranan Travel Grant awarded to a woman whose interest in and dedication            to SIG best exemplifies Chris

Recognizing a lifetime of contributions:

Prof. Raj Jain

        Industrial researcher, entrepreneur, teacher, academic mentor

        Jain fairness index

        Rate control algorithms

        Best selling book "Art of Computer Systems Performance Analysis"

Keynote by Prof. Jain

A pretty good joke:
         Professor traveled a lot before. He is now Touring complete

The Catch-up Game:

• Is networking still hot or should I change? Yes it is.
• Almost all areas of computing and most-valued companies are network-based.
• Is the technology I am working on, succeed? 
• To succeed: Low cost, Killer Application, Coexistence with legacy, Timely completion,                       Promised Performance, Manageability, Interoperability 
• Transition strategy is very important for any tech. to succeed. Clean slate solutions are not ideal.
• Professor's own initial work on Congestion Control
• What to do when a packet is dropped?
• How often do you go up?
• What is a fair allocation?
• How to achieve fairness and efficiency? AIMD.
• What is required to make an impact?
• Select the right research problem. Watch for and adapt to paradigm shifts and hype cycles (Gartner's Hype Cycle for Emerging Tech 2017). Bring it to completion (from development to product).
• Every person is a company. You need an idea, development, marketing and sales capabilities. Diversify your research and measure success by adoption. 
• Don't be let down by failures. Rejections lead to improvements.
• If you are too deep in one area, you cannot move fast enough with the changing world.
• Recent research trends
• Micro-Cloud Computing
• "cloud = virtual resources" these days
• Micro-services -> Applications distributed over multiple small clouds
• Software Defined Multi-Cloud
• SDN allows us to manage a large number of routers. Extend this to clouds
• OpenADN Multi-Cloud Management
• Security in IoT

• DEFCON is a major security conference with a lot of hackers and security agency personnel. If you want a job in security, this is where you need to go
  
• Blockchains
• Make everything decentralized with no central point of control. 
• Apply that to networking applications (Decentralized DNS, CAs)

Questions:

Q: How do you know that the impact is positive/negative for a candidate.

A: As long as you make at least one or two positive, you will be remembered and will be a success

  

SIGCOMM 2017 descends on Los Angeles

Welcome to SIGCOMM 2017! For the fifth year in a row, Layer9.org will provide gavel-to-gavel coverage of the conference, powered by an all-star [and record-sized!] team of 31 scribes. Stay tuned here as we file reports on each talk and Q-and-A. The conference proceedings, including open-access copies of all of the papers, are at http://conferences.sigcomm.org/sigcomm/2017/program.html

If you would like to make a post of your own, please email signup@layer9.org if you do not already have an account.

Session 6 paper 3: Resilient Datacenter Load Balancing in the Wild (Hong Zhang, Junxue Zhang, Wei Bai, Kai Chen, Mosharaf Chowdhury)

Data center networks are usually multi-rooted, so load-balancing across the multiple paths are required. However, load-balancing are hard to deal with uncertainties of three types:
1. Traffic dynamics: e.g., congestion caused by bursts.
2. Topology asymmetry: e.g., link cut or heterogenous devices
3. Switch failures: including blackhole and silent random drops.

So a load balancing has two requirements:
1. It needs to sense uncertainties.
2. It needs to react to uncertainties.

Previous works all fail to sense switch failures, and have their own problems. Here are three examples showing the problem of previous solutions.

Example 1: problem of failure-ignorant. If random drop happens, congestion-aware LB does not sense it. Actually, it even divert more traffic to the failed switch, because there is very few traffic (mostly dropped by blackhole/random drops).

Example 2: Flowlet switching fails to react to uncertainty timely. DCTCP traffic is much less bursty then normal TCP, so it is hard to find a large enough gap to separate a flowlet.

Example 3: Problem of vigurous rerouting. It causes congestion mismatch: the TCP adjusts rate based on the current path's congestion condition, but it changes to another path, this rate may not be right.

The challenge of LB is, how to gracefully handle uncertainties. The author proposes Hermes, which has four properties: comprehensiveness, timeliness, transport friendly, and deployability. It has two modules: sensing module and rerouting module. Sensing module senses both congetion and failures, and rerouting module decides when an where to reroute traffic.

The sensing module uses ECN and RTT to sense congestion. It uses retransmission and timeout to infer failure: frequent timeout means blackhole, and frequent retransmission means random drops. It also uses active probing to improve visibility. The baseline of probing is to probe all paths. They reduce the overhead by only probing 2 random paths (refer to the power of two choices), plus 1 previous best path.

The rerouting module is cautious. The author gives an example of one flow, in which a reroute may first drop the rate to half, and then increase. So rerouting can be beneficial even with reordering, and we should reroute a flow immediately if it can reduce the FCT. Hermes has three heuristics: (1) only reroute when the new path is much better (because the sensing is inaccurate, sensed slightly better may not be actual better), (2) avoid rerouting with small remaining size, because the flow may not benefit much from the higher rate, but has to experience dropped rate, and (3) avoid rerouting flows with already high sending rate.

In the evaluation, the author shows that for some workload Conga is slightly better (17% lower average FCT), because switch-based solution has better visibility to the congestion. However, under asymmetric topo, Hermes is better than Conga and presto, because conga has very few flowlets, and presto is congestion oblivious. The author also shows under switch failure, Hermes is better by up to 32% FCT.

Q: How to handle micro-burst
A: Major goal is not handle micro-burst. Our goal is like conga, having feedback loop, to avoid global congestion. There is a tradeoff, if you want to avoid global congestion, you need longer feedback loop; Drill has much smaller feedback loop, so it does not avoid global congestion.

Q: Run simulation at 2G, what about 10G and 25G?
A: At baseline, we run 10G. The 2G is to create asymmatry.
Q: What happend at 25G?
A: Intereting problem. One thing might change is the transport protocol behavior and how LB interact with it. At 25G, maybe flowlets have diff pattern. Good to dig out.

Q: Distributed system, why independent choice lead to correct estimation?
A: We leverage the power of 2 choice. Each host randomly selects 2 path to probe, so we can avoid the herding problem.