[e2e] packet size distribution of Internet traffic

[Cannara]

A point of information, having done work with folks using network-processor
chips (NPUs) for building switches, routers, balancers & security boxes, the
allocation of memory to packets is very much under the control of the device
programmer and is determined by a reasonable guess on, or thorough analysis
of, expected packet traffic.  Packets are typically sliced into a Header
portion and Payload.  The Header is kept for analysis, appropriate to the box
purpose, in fast RAM associated with the processor.  Usually, several
processors run in parallel and thus have parallel, independent fast RAM
dedicated to analyzing several packet headers at once.  

The payloads of packets are transferred to large, external RAM, which is
partitioned into linked lists of blocks, according to the programmer's design
needs.  These blocks can be as small as 128B, or as large as a maximum packet
(say 4kB for FDDI).  The tradeoff in performance and number of packets
storable is made by the programmer.

A similar, more limited tradeoff is made in the fast Header storage, based on
how much of the protocol content is needed by the box before completing a
packet's processing, and the line rate the box must keep up with.  A benefit
of processing Headers independent of Payloads is that output queues at most
need to have headers DMAed to them. 

The advent of NPUs has made many vendors reconsider (meaning lay off) their
ASIC efforts, so many current boxes, particularly in security processing, use
NPUs from Intel, IBM, Vitesse, etc.  This means that the choice to fragment or
not in RAM, typically only refers to payloads (and then not up to Jumbo
sizes).  Headers are nearly always stripped upon packet entry into the
processing unit, so at least that fragmentation occurs.  In any case, there's
extreme flexibility in how packets are internally fragmented, even among the
same vendor's boxes.

Alex

Nicolas Christin wrote:
> 
> On Tue, 15 Apr 2003, Nicolas Christin wrote:
> 
> > For ease of internal scheduling, routers generally internally deal
> > with fixed size cells
> 
> It has been brought to my attention that this statement was not entirely
> correct. It should read "some types of routers internally deal with
> fixed size cells". (See for instance "Reduced Complexity for Input
> Buffered Switches" by Kar, Stiliadis, and Lakshman, Hot Interconnects
> 2000, for a discussion on this.) But there are obviously some routers
> that do not fragment packets internally, and these routers appear to be
> currently dominant on the market share. My mistake.
> 
> Best regards,
> --
> Nicolas