Internet Service Guarantees

Or at least past attempts at service guarantees

Peter Dordal, Loyola University CS Department

Home & Small Office VoIP

http://intronetworks.cs.luc.edu/current/html/queuing.html#guaranteeing-voip-bandwidth

Make no mistake: if you try to fit 20 VoIP calls on a line with bandwidth sufficient only for 10, it will fail. Actually, if you try to fit 20 VoIP calls on a line with bandwidth sufficient only for 18, it will fail, and if the bandwidth is sufficient only for 19, then users will complain.

But suppose you have "more than enough" Internet capacity. Here's the problem: what if you have a "few" VoIP lines and you're also downloading bulk data, all on the same internet connection. How do you make sure the VoIP packets get priority? If you've got a 1Mbps link, one DS0 (ยต-law) line will use (not counting headers) 3% of the bandwidth. Counting headers but using reasonable compression, you might have five voice lines using well under 10% of the bandwidth. But if the router queue is long, you don't want VoIP traffic waiting at the back end.

For outbound traffic, the solution is simple. You get a little box that connects to (a) your ISP, (b) your internal data LAN, and (c) your phone lines. The box simply implements outbound priority queuing, sending VoIP packets at a higher priority. Such boxes are standard hardware items.

The harder direction is inbound. What if, at the router, a 1.0-second queue has built up, because someone is downloading a large file? Your inbound VoIP packets will likely sit at the back of this queue. If your ISP has a lot of sense, they will allow at their end of your link only a small queue, so this does not happen. Failing that, though, you can:

(a) negotiate with your ISP for them to do priority routing of VoIP packets at their end too. Their router can almost certainly easily be configured to act just like the little box at your end. Note that this option, though, means telling your ISP you want to use your Internet for voice; they may want to charge you extra.

(b) Try to throttle your inbound data by shaping (ie slowing) the stream of outboundACKs. Your little box, above, can estimate when the VoIP packets are slowing down, and, at that point, it might slow down outbound data ACKs to just a sufficient extent that the inbound VoIP stream returns to normal, or until the inbound data rate falls below (no-VoIP-data-rate − bandwidth-allocated-for-voice). It's dicey; I'm not sure how often this is done. Perhaps the biggest problem with slowing ACKs is that initially it does not work at all (because the upstream queue is still draining), and then all of a sudden it has a dramatic and probably-too-large effect.


Integrated Services Architecture (ISA) and RSVP

http://intronetworks.cs.luc.edu/current/html/realtime.html#integrated-services-rsvp

ISA is the overall architecture, and RSVP (ReSerVation Protocol) is a protocol for reserving bandwidth for voice calls or any other designated use.

The first point is that even without real-time traffic, we would still like to do some forms of traffic management. For example, we would probably like to prioritize telnet/ssh traffic, which are time-sensitive to at least some degree, to avoid queuing problems. Example of telnet + file download through a slow link. This is the same situation as the voice example above.

Simple case with all non-realtime traffic: we're only interested in controlling bandwidth.

Realtime traffic: may need throughput, delay, jitter, loss bounds

Notion of "flows": common user, endpoints, QoS needs

RSVP: Each flow can make a reservation with the routers. Note that this is a major retreat from the datagram-routing stateless-router model. Were virtual circuits the better routing model all along?

Problem with the Internet

The situation:

1. IP routers are stateless; that is, they use datagram routing. THE biggest argument in favor of ATM, by ATM proponents, was that virtual circuits allowed routers to know about connections and thus reserve capacity. How can a datagram router reserve anything?

IP routers are subject to being rebooted, and links are subject to failure. If routers have HARD reservation state, they MUST not lose it when rebooting. If a link fails, hosts may then not be able to tear down a connection (or if an intermediate router fails!). But routers with "hard" reservation state MUST receive teardown notification.

The RSVP solution: "soft state" that times out. Soft state can be refreshed if lost (though with some small probability of failure)

2. Not all routers may start providing RSVP services at once! (Actually, not any routers may start providing RSVP services ever.)

3. We need to support "all" known major needs for QoS. Voice is one, but the other biggie is multicast video/audio for teleconferencing. Unicast video is also supported, but multicasting is technically more complicated to integrate.
send

Overview of multicasting

http://intronetworks.cs.luc.edu/current/html/realtime.html#ip-multicast

Typical model: many more receivers than senders (though teleconferencing may involve lots of people sending, in turn)

Construction of multicast tree within the routers; this tree also is maintained through soft state. Multicast, like RSVP, has major "rollout" problems, although one way that has been addressed is through tunneling (= manual multicast-tree construction)

We need to avoid IPv4 fragmentation, so that receivers can identify packets by flow info

Features:

Basic Ethernet version of multicast: using multicast Ethernet addresses

IP: goal is to construct a "spanning tree" (draw picture). Compare to "multiple unicast"

A receiver R wishing to join the tree send JOIN messages to the sender S. However, the routers through which the JOIN message passes on the way from R to S may have nothing to do with the routers on the path from S to R. S then sends R's identity out towards R; routers forward it along the S-to-R path but take no action unless there is a need to create a new branch.

In some cases, sender doesn't even necessarily know who the receivers are! Receivers just contact routers.

Shortest-Path broadcast:
sender S sends, routers retransmit on all links if packet arrived directly from S; that is, arrived via shortest path to S.

Shortest-Path multicast:  routers with no members of multicast group G send this info to neighbors.
Kind of expen$ive.

IP addrs for multicast group: 224.x.y.z ... 239.x.y.z


RSVP

http://intronetworks.cs.luc.edu/current/html/realtime.html#rsvp

RSVP: reservations compatible with multicast: they are receiver-initiated. (Multicast senders may not even know who all the receivers are!)
Different receivers may have different needs; these can sometimes be accommodated by routers, without bothering the sender.

how RSVP routers recognize flows; IP6 flowID

RSVP Architecture service categories:
Within each category, a sender negotiates a "TSpec", somewhat like GCRA specs.

Guaranteed service:
Note that the MaxDelay, while bounded, may be relatively large.

Controlled-load: approximates best-effort service on lightly loaded networks. That is, queue bursts are "rare".
Controlled-load is good for adaptive applications. Voice & video can be made adaptive, though for voice the main adaptation is to increase the total delay. Voice bandwidth is only adaptive if you are using an adaptive codec (eg speex).

Routers involved in RSVP typically do weighted fair queuing on whole service categories, not per-flow!

typical reservation timeout: 30 sec

RSVP does include teardown messages (both directions), but RSVP won't fall apart if they are not sent. The reservation will go away after ~30 sec of no RESV messages (below).

RSVP is transparent to nonparticipating ISPs (though the service agreements may be impossible to guarantee).

Receivers send periodic (30 sec) RESV messages to refresh routers; routers forward these "up the path". Note that the receiver sends these to the router from which it is receiving the reserved flow. Each router figures out reverse path, and RESV packets are sent along this reverse path by the receiver.
Each RSVP sender host transmits RSVP "Path" messages downstream along the uni-/multicast routes provided by the routing protocol(s), following the paths of the data.  These Path messages store "path state" in each node along the way.  This path state includes at least the unicast IP address of the previous hop node, which is used to route the Resv messages hop-by-hop in the reverse direction.

Sometimes RESV messages can be merged. For example, if receiving nodes A and B each connect to R, which in turn connects to S, and A and B each send RESV messages towards S, then R may be able to consolidate these and send only a single RESV message to S.

Sender sends its TSpec in PATH mesg; note that some receivers may ask for a different TSpec

Unidirectional reservations and different paths each direction

       R1-----R2
     /          \
A--R0            R5---B
     \          /
      R3-----R4

Suppose A⟶B traffic travels A⟶R0⟶R1⟶R2⟶R5⟶B, and B⟶A traffic travels B⟶R5⟶R4⟶R3⟶R0⟶A, and A is the RSVP sender.
B sends RESV message R5⟶R2⟶R1⟶R0⟶A by virtue of saved state at these routers, which recognize RESV packets specially. R5 sees a RESV packet destined for A; it must know to send it to R2 (which it determines from its reservation state) rather than to R4.

Reservations are initiated by requests sent by receiver to sender (these follow the standard receiver-to-sender route); sender sends PATH packets back down to receiver; these lay groundwork for return-path computation. The PATH message also contains sender's traffic specification, or Tspec.

Problem: too many reservations. Routers have too much to do. And how do we decide who gets to reserve what?
 
Three models:
1. Charge $ for each reservation made
2. Charge a flat additional rate for the right to make reservations
3. Anyone can ask for a reservation, but the answer may be "no". Maybe there would be a cap on size


Traffic Specifications, or Tspecs

RFC2210: standard Tspec has

NO MaxDelay!!! But see notes in rfc 2212:

applications can use the Tspec values above to estimate their queuing delay, which when added to propagation delay gives an accurate total.

How we might do RSVP over the backbone: see notes in rfc2208. Basic idea: do something else there!! Maybe just trust to best-effort?

How VOIP may facilitate adoption of RSVP (well, probably not outside of corporate networks)

merging of multicast reservations

Reasons for lower QoS on some reserved paths:
MaxDelay QoS only: some receivers may accept larger buffer!!!


link-level cooperation with ATM links

One potential advantage over ATM: QoS can change dynamically (this is relevant for video). Also RSVP supports multicast.

Admission control: how exercised???

RSVP did not scale well; every router on the path had to maintain connection state for every connection through it. Thus began a search for something else.



Differentiated Services

http://intronetworks.cs.luc.edu/current/html/realtime.html#differentiated-services

Problem: Integrated Services may not scale well. Few ISPs have adopted it. Can we come up with a simpler mechanism?

Differentiated Services: basically just two service classes: high and low (now three levels)
A simple priority-queue mechanism to handle the service classes is enough.

Rules on which packets can be "premium": we set a maximum rate of such packets arriving from border router?

Goal: set some rules on admitting premium packets, and hope that their total numbers to any given destination is small enough that we can meet service targets (not exactly guarantees)

Packets are marked at ingress only. This simplifies things. Interior routers just have to grant priority status to the marked packets. Marking patterns are few in number ("hi" and "low" might be enough, though there are 12 mark patterns identified below); we are not marking on a per-flow basis like RSVP! Interior routers, in other words, just see large flows, and route them appropriately.

DS may start with a Service Level Agreement (SLA) with your ISP, that defines just what traffic (eg VOIP) will get what service. No need to change applications. DS bits will probably be set by the ISP. The DS field repurposes the old IP4 TOS field, widely ignored in the past.

Example: VOIP
The ISP (not the user!) would mark VOIP packets as they enter, subject to some ceiling. These are routed internally (within the ISP) with premium service. The ISP negotiates with its ISP for a total bulk delivery of premium packets.

One possible arrangement is that the leaf ISPs would use RSVP, but the Internet core would run DS. Packets would be DS-marked as they enter the core, based on their RSVP status.

DS bulk-traffic management generally works best with a "statistical" model: if you as an ISP are allocating 10% of your bandwidth to VoIP, you limit each ingress point to 10%. And you hope that the traffic is uniformly distributed to each egress point, so that at each egress your outbound VoIP share is also 10%. But note that it doesn't have to work out this way: you might have five equal-sized ingress points each accepting 10%, and all of it going to a single egress point that then has 50% VoIP traffic. Or a 100Mbps ingress point might have 10Mbps (10%) of VoIP traffic, but all of it goes to a 20Mbps egress link, amounting to 50%.


DS field: this is the first six bits of the second byte of the IP header, after 0x45 and before the 16-bit length field. The remaining two bits are the ECN bits.
The 6 bits are divided into 3 bits for class and 3 bits for drop_precedence. However, some bit patterns are reserved for legacy uses.
 
Two basic strategies: Expedited Forwarding (EF) and Assured Forwarding (AF). 
101 110    "EF", or "Expedited Forwarding": best service. This amounts to marking packets as EF packets, and then giving them the top priority.
 
Assured Forwarding: 3 bits of Class, 3 bits of Drop Precedence
Class:
100    class 4: best
011    class 3
010    class 2
001    class 1
 
Drop Precedence:
 
010:    don't drop
100:    medium
110    high

(See Stallings fig 8e:19.13 / 9e:20.8)

Main thing: The classes each get priority service, over best-effort.

What happens when your traffic must be carried by another ISP? Do they respect the first ISP's DS bits?

What happens if you send more than you've contracted for?

Routers SHOULD implement priority queues for service categories

Basic idea: get your traffic marked for the appropriate class. Then what?

000 000: current best-effort status
xxx 000: traditional IPv4 precedence

PHBs (Per-Hop Behaviors): implemented by all routers
Only "boundary" routers do traffic policing/shaping/classifying/re-marking to manage categories (re-marking is really part of shaping/policing)

EF: Expedited Forwarding

This is the best service, implemented as basically the highest-priority service. Packets should experience lowest queuing delay. The lower-priority traffic should not affect the overall EF performance, but too much EF traffic can degrade EF performance. As usual, EF packets will go into the topmost band of the priority queue at each router.

(We may give bulk traffic some guaranteed share, but if we have to be worrying about that then we've probably oversold EF.)

Functionality depends on ensuring that there is not too much EF traffic.

Basically, we control at the boundary the total volume of EF traffic (eg to a level that cannot saturate the slowest link), so that we have plenty of capacity for EF traffic.

EF provides a minimum-rate guarantee.
This can be tricky: if we accept input traffic from many sources, and have four traffic outlets R1, R2, R3, R4, then we should only accept enough EF traffic that any one Ri can handle it. But we might go for a more statistical model, if in practice 1/4 of the traffic goes to each Ri.

AF: Assured Forwarding

Simpler than EF, but no guarantee. Traffic totals can be higher. There is an easy way to send more traffic: it is just marked as having a lower drop precedence. This marking would initially be done by the sender, but the ingress router (at whichever level) is allowed to lower the precedence if it detects more AF traffic than it has agreed to allow. Such a router would use a token-bucket filter to determine which packets were conforming and which were not; the non-conforming packets would have their precedence level lowered.

In-out marking: we will refer to high and low precedence as "in" or "out"; packets will be marked by the ingress policer. (Actually, we have three precedence levels to use for marking.)

From RFC2597:

The drop precedence level of a packet could be assigned, for example, by using a leaky bucket traffic policer, which has as its parameters a rate and a size, which is the sum of two burst values: a committed burst size and an excess burst size.  A packet is assigned low drop precedence if the number of tokens in the bucket is greater than the excess burst size [ie bucket is full],  medium drop precedence if the number of tokens in the bucket is greater than zero, but at most the excess burst size, and high drop precedence if the bucket is empty.

RIO: RED with In/Out: RED = Random Early Detection

Packet "mangling" is used to mark DS bits; the routers have a sufficient number of priority bands for the drop precedences.
(It is not quite clear how to handle the different classes; they might get classful TBF service)

AF works nicely with RIO routers: RED with In and Out (or In, Middle, and Out): each traffic "level" is subject to a different drop threshold. This strategy does not involve reordering!