"Fossies" - the Fresh Open Source Software Archive

Member "shorewall-docs-xml-5.2.8/traffic_shaping.xml" (24 Sep 2020, 99058 Bytes) of package /linux/misc/shorewall/shorewall-docs-xml-5.2.8.tar.bz2:


As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) XML source code syntax highlighting (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 <?xml version="1.0" encoding="UTF-8"?>
    2 <!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
    3 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
    4 <article>
    5   <!--$Id$-->
    6 
    7   <articleinfo>
    8     <title>Complex Traffic Shaping/Control</title>
    9 
   10     <authorgroup>
   11       <author>
   12         <firstname>Tom</firstname>
   13 
   14         <surname>Eastep</surname>
   15       </author>
   16 
   17       <author>
   18         <firstname>Arne</firstname>
   19 
   20         <surname>Bernin</surname>
   21       </author>
   22     </authorgroup>
   23 
   24     <pubdate><?dbtimestamp format="Y/m/d"?></pubdate>
   25 
   26     <copyright>
   27       <year>2001-2013</year>
   28 
   29       <year>2020</year>
   30 
   31       <holder>Thomas M. Eastep</holder>
   32     </copyright>
   33 
   34     <copyright>
   35       <year>2005</year>
   36 
   37       <holder>Arne Bernin &amp; Thomas M. Eastep</holder>
   38     </copyright>
   39 
   40     <legalnotice>
   41       <para>Permission is granted to copy, distribute and/or modify this
   42       document under the terms of the GNU Free Documentation License, Version
   43       1.2 or any later version published by the Free Software Foundation; with
   44       no Invariant Sections, with no Front-Cover, and with no Back-Cover
   45       Texts. A copy of the license is included in the section entitled
   46       <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
   47       License</ulink></quote>.</para>
   48     </legalnotice>
   49   </articleinfo>
   50 
   51   <important>
   52     <para>Traffic shaping is complex and the Shorewall community is not well
   53     equipped to answer traffic shaping questions. So if you are the type of
   54     person who needs "insert tab A into slot B" instructions for everything
   55     that you do, then please don't try to implement traffic shaping using
   56     Shorewall. You will just frustrate yourself and we won't be able to help
   57     you.</para>
   58   </important>
   59 
   60   <warning>
   61     <para>Said another way, reading just Shorewall documentation is not going
   62     to give you enough background to use this material.</para>
   63 
   64     <para>At a minimum, you will need to refer to at least the following
   65     additional information:</para>
   66 
   67     <itemizedlist>
   68       <listitem>
   69         <para><emphasis>The LARTC HOWTO</emphasis>: <ulink
   70         url="http://www.lartc.org">http://www.lartc.org</ulink></para>
   71       </listitem>
   72 
   73       <listitem>
   74         <para>T<emphasis>he HTB User's Guide</emphasis>: <ulink
   75         url="http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm">http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm</ulink></para>
   76       </listitem>
   77 
   78       <listitem>
   79         <para><emphasis>HFSC Scheduling with Linux</emphasis>: <ulink
   80         url="http://linux-ip.net/articles/hfsc.en/">http://linux-ip.net/articles/hfsc.en/</ulink></para>
   81       </listitem>
   82 
   83       <listitem>
   84         <para>Some of the documents listed at <ulink
   85         url="http://www.netfilter.org/documentation/index.html#documentation-howto">http://www.netfilter.org/documentation/index.html#documentation-howto</ulink>.
   86         The tutorial by Oskar Andreasson is particularly good.</para>
   87       </listitem>
   88 
   89       <listitem>
   90         <para>The output of <command>man iptables</command></para>
   91       </listitem>
   92     </itemizedlist>
   93   </warning>
   94 
   95   <section id="Intro">
   96     <title>Introduction</title>
   97 
   98     <para>Beginning with Shorewall 4.4.6, Shorewall includes two separate
   99     implementations of traffic shaping. This document describes the original
  100     implementation which is complex and difficult to configure. A much simpler
  101     version is described in <ulink role="bold"
  102     url="simple_traffic_shaping.html">Simple Traffic Shaping/Control</ulink>
  103     and is highly recommended unless you really need to delay certain traffic
  104     passing through your firewall.</para>
  105 
  106     <para>Shorewall has builtin support for traffic shaping and control. This
  107     support does not cover all options available (and especially all
  108     algorithms that can be used to queue traffic) in the Linux kernel but it
  109     should fit most needs. If you are using your own script for traffic
  110     control and you still want to use it in the future, you will find
  111     information on how to do this, <link linkend="owntcstart">later in this
  112     document</link>. But for this to work, you will also need to enable
  113     traffic shaping in the kernel and Shorewall as covered by the next
  114     sections.</para>
  115   </section>
  116 
  117   <section id="LinuxTC">
  118     <title>Linux traffic shaping and control</title>
  119 
  120     <para>This section gives a brief introduction of how controlling traffic
  121     with the Linux kernel works. Although this might be enough for configuring
  122     it in the Shorewall configuration files, we strongly recommend that you
  123     take a deeper look into the <ulink url="http://lartc.org/howto/">Linux
  124     Advanced Routing and Shaping HOWTO</ulink>. At the time of writing this,
  125     the current version is 1.0.0.</para>
  126 
  127     <para>Since kernel 2.2, Linux has extensive support for controlling
  128     traffic. You can define different algorithms that are used to queue the
  129     traffic before it leaves an interface. The standard one is called pfifo
  130     and is (as the name suggests) of the type First In First out. This means,
  131     that it does not shape anything, if you have a connection that eats up all
  132     your bandwidth, this queuing algorithm will not stop it from doing
  133     so.</para>
  134 
  135     <para>For Shorewall traffic shaping we use three algorithms: HTB
  136     (Hierarchical Token Bucket), HFSC (Hierarchical Fair Service Curves) and
  137     SFQ (Stochastic Fairness Queuing). SFQ is easy to explain: it just tries
  138     to track your connections (tcp or udp streams) and balances the traffic
  139     between them. This normally works well. HTB and HFSC allow you to define a
  140     set of classes, and you can put the traffic you want into these classes.
  141     You can define minimum and maximum bandwidth settings for those classes
  142     and order them hierarchically (the less prioritized classes only get
  143     bandwidth if the more important have what they need). Additionally, HFSC
  144     allows you to specify the maximum queuing delay that a packet may
  145     experience. Shorewall builtin traffic shaping allows you to define these
  146     classes (and their bandwidth limits), and it uses SFQ inside these classes
  147     to make sure, that different data streams are handled equally. If SFQ's
  148     default notion of a 'stream' doesn't work well for you, you can change it
  149     using the <emphasis role="bold">flow</emphasis> option described <link
  150     linkend="tcclasses">below</link>.</para>
  151 
  152     <para>You can shape incoming traffic through use of an
  153     <firstterm>Intermediate Functional Block</firstterm> (IFB) device. <link
  154     linkend="IFB">See below</link>. <emphasis role="bold">But beware: using an
  155     IFB can result in queues building up both at your ISPs router and at your
  156     own.</emphasis></para>
  157 
  158     <para>You shape and control outgoing traffic by assigning the traffic to
  159     <firstterm>classes</firstterm>. Each class is associated with exactly one
  160     network interface and has a number of attributes:</para>
  161 
  162     <orderedlist>
  163       <listitem>
  164         <para>PRIORITY - Used to give preference to one class over another
  165         when selecting a packet to send. The priority is a numeric value with
  166         1 being the highest priority, 2 being the next highest, and so
  167         on.</para>
  168       </listitem>
  169 
  170       <listitem>
  171         <para>RATE - The minimum bandwidth this class should get, when the
  172         traffic load rises. Classes with a higher priority (lower PRIORITY
  173         value) are served even if there are others that have a guaranteed
  174         bandwidth but have a lower priority (higher PRIORITY value).</para>
  175       </listitem>
  176 
  177       <listitem>
  178         <para>CEIL - The maximum bandwidth the class is allowed to use when
  179         the link is idle.</para>
  180       </listitem>
  181 
  182       <listitem>
  183         <para>MARK - Netfilter has a facility for
  184         <firstterm>marking</firstterm> packets. Packet marks have a numeric
  185         value which is limited in Shorewall to the values 1-255 (1-16383 if
  186         you set WIDE_TC_MARKS=Yes in <ulink
  187         url="manpages/shorewall.conf.html">shorewall.conf</ulink> (5) ). You
  188         assign packet marks to different types of traffic using entries in the
  189         <filename>/etc/shorewall/mangle</filename> file (Shorewall 4.6.0 or
  190         later) or <filename>/etc/shorewall/tcrules</filename> (Prior to
  191         Shorewall 4.6.0).</para>
  192 
  193         <note>
  194           <para>In Shorewall 4.4.26, WIDE_TC_MARKS was superseded by TC_BITS
  195           which specifies the width in bits of the traffic shaping mark field.
  196           The default is based on the setting of WIDE_TC_MARKS so as to
  197           provide upward compatibility. See the <ulink
  198           url="PacketMarking.html#Values">Packet Marking using
  199           /etc/shorewall/mangle</ulink> article.</para>
  200         </note>
  201       </listitem>
  202     </orderedlist>
  203 
  204     <para>One class for each interface must be designated as the
  205     <firstterm>default class</firstterm>. This is the class to which unmarked
  206     traffic (packets to which you have not assigned a mark value in
  207     <filename>/etc/shorewall/mangle</filename>) is assigned.</para>
  208 
  209     <para>Netfilter also supports a mark value on each connection. You can
  210     assign connection mark values in
  211     <filename>/etc/shorewall/mangle</filename>
  212     (<filename>/etc/shorewall/tcrules</filename>), you can copy the current
  213     packet's mark to the connection mark (SAVE), or you can copy the
  214     connection mark value to the current packet's mark (RESTORE). For more
  215     information, see<ulink url="PacketMarking.html"> this
  216     article</ulink>.</para>
  217   </section>
  218 
  219   <section id="Shorewall">
  220     <title>Enable TC support in Shorewall</title>
  221 
  222     <para>You need this support whether you use the builtin support or whether
  223     you provide your own tcstart script.</para>
  224 
  225     <para>To enable the builtin traffic shaping and control in Shorewall, you
  226     have to do the following:</para>
  227 
  228     <itemizedlist>
  229       <listitem>
  230         <para>Set <emphasis role="bold">TC_ENABLED</emphasis> to "<emphasis
  231         role="bold">Internal</emphasis>" in /etc/shorewall/shorewall.conf.
  232         Setting <emphasis role="bold">TC_ENABLED=Yes</emphasis> causes
  233         Shorewall to look for an external tcstart file (See <link
  234         linkend="tcstart">a later section</link> for details).</para>
  235       </listitem>
  236 
  237       <listitem>
  238         <para>Setting <emphasis role="bold">CLEAR_TC</emphasis> parameter in
  239         /etc/shorewall/shorewall.conf to <emphasis role="bold">Yes</emphasis>
  240         will clear the traffic shaping configuration during Shorewall
  241         [re]start and Shorewall stop. This is normally what you want when
  242         using the builtin support (and also if you use your own tcstart
  243         script)</para>
  244       </listitem>
  245 
  246       <listitem>
  247         <para>The other steps that follow depend on whether you use your own
  248         script or the builtin solution. They will be explained in the
  249         following sections.</para>
  250       </listitem>
  251     </itemizedlist>
  252   </section>
  253 
  254   <section id="Builtin">
  255     <title>Using builtin traffic shaping/control</title>
  256 
  257     <para>Shorewall's builtin traffic shaping feature provides a thin layer on
  258     top of the ingress qdesc, HTB and SFQ. That translation layer allows you
  259     to:</para>
  260 
  261     <itemizedlist>
  262       <listitem>
  263         <para>Define HTB and/or HFSC classes using Shorewall-style
  264         column-oriented configuration files.</para>
  265       </listitem>
  266 
  267       <listitem>
  268         <para>Integrate the reloading of your traffic shaping configuration
  269         with the reloading of your packet-filtering and marking
  270         configuration.</para>
  271       </listitem>
  272 
  273       <listitem>
  274         <para>Assign traffic to HTB or HFSC classes by TOS value.</para>
  275       </listitem>
  276 
  277       <listitem>
  278         <para>Assign outgoing TCP ACK packets to an HTB or HFSC class.</para>
  279       </listitem>
  280 
  281       <listitem>
  282         <para>Assign traffic to HTB and/or HFSC classes based on packet mark
  283         value or based on packet contents.</para>
  284       </listitem>
  285 
  286       <listitem>
  287         <para>Throttle incoming traffic</para>
  288       </listitem>
  289 
  290       <listitem>
  291         <para>Use an <emphasis>Intermediate functional block </emphasis>(IFB)
  292         to shape incoming traffic<emphasis> </emphasis></para>
  293       </listitem>
  294     </itemizedlist>
  295 
  296     <para>Those few features are really all that builtin traffic
  297     shaping/control provides; consequently, you need to understand HTB and/or
  298     HFSC and Linux traffic shaping as well as Netfilter packet marking in
  299     order to use the facility. Again, please see the links at top of this
  300     article.</para>
  301 
  302     <para>For defining bandwidths (for either devices or classes) please use
  303     kbit or kbps (for Kilobytes per second) and make sure there is <emphasis
  304     role="bold">NO</emphasis> space between the number and the unit (it is
  305     100kbit <emphasis role="bold">not</emphasis> 100 kbit). Using mbit, mbps
  306     or a raw number (which means bytes) could be used, but note that only
  307     integer numbers are supported (0.5 is <emphasis role="bold">not
  308     valid</emphasis>).</para>
  309 
  310     <para><emphasis role="bold">To properly configure the settings for your
  311     devices you need to find out the real up- and downstream rates you
  312     have</emphasis>. This is especially the case, if you are using a DSL
  313     connection or one of another type that do not have a guaranteed bandwidth.
  314     Don't trust the values your provider tells you for this; especially
  315     measuring the real download speed is important! There are several online
  316     tools that help you find out; search for "dsl speed test" on google (For
  317     Germany you can use <ulink
  318     url="http://www.speedcheck.arcor.de/cgi-bin/speedcheck.cgi">arcor speed
  319     check</ulink>). Be sure to choose a test site located near you.</para>
  320 
  321     <section id="tcdevices">
  322       <title>/etc/shorewall/tcdevices</title>
  323 
  324       <para>This file allows you to define the incoming and outgoing bandwidth
  325       for the devices you want traffic shaping to be enabled. That means, if
  326       you want to use traffic shaping for a device, you have to define it
  327       here. For additional information, see <ulink
  328       url="manpages/shorewall-tcdevices.html">shorewall-tcdevices</ulink>
  329       (5).</para>
  330 
  331       <para>Columns in the file are as follows:</para>
  332 
  333       <itemizedlist>
  334         <listitem>
  335           <para>INTERFACE - Name of interface. Each interface may be listed
  336           only once in this file. You may NOT specify the name of an alias
  337           (e.g., eth0:0) here; see <ulink url="FAQ.htm#faq18">FAQ #18</ulink>.
  338           You man NOT specify wildcards here, e.g. if you have multiple ppp
  339           interfaces, you need to put them all in here! Shorewall will
  340           determine if the device exists and will only configure the device if
  341           it does exist. If it doesn't exist or it is DOWN, the following
  342           warning is issued:</para>
  343 
  344           <para><emphasis role="bold">WARNING: Device &lt;device name&gt; is
  345           not in the UP state -- traffic-shaping configuration
  346           skipped</emphasis></para>
  347 
  348           <para>Shorewall assigns a sequential <firstterm>interface
  349           number</firstterm> to each interface (the first entry in
  350           <filename>/etc/shorewall/tcdevices</filename> is interface 1, the
  351           second is interface 2 and so on) You can also explicitly specify the
  352           interface number by prefixing the interface name with the number and
  353           a colon (":"). Example: 1:eth0.</para>
  354 
  355           <warning>
  356             <para>Device numbers are expressed in hexidecimal. So the device
  357             following 9 is A, not 10.</para>
  358           </warning>
  359         </listitem>
  360 
  361         <listitem>
  362           <para>IN-BANDWIDTH - The incoming Bandwidth of that interface.
  363           Please note that when you use this column, you are not traffic
  364           shaping incoming traffic, as the traffic is already received before
  365           you could do so. This Column allows you to define the maximum
  366           traffic allowed for this interface in total, if the rate is
  367           exceeded, the excess packets are dropped. You want this mainly if
  368           you have a DSL or Cable Connection to avoid queuing at your
  369           providers side. If you don't want any traffic to be dropped set this
  370           to a value faster than your interface maximum rate, or to 0
  371           (zero).</para>
  372 
  373           <para>To determine the optimum value for this setting, we recommend
  374           that you start by setting it significantly below your measured
  375           download bandwidth (20% or so). While downloading, measure the
  376           <emphasis>ping</emphasis> response time from the firewall to the
  377           upstream router as you gradually increase the setting.The optimal
  378           setting is at the point beyond which the <emphasis>ping</emphasis>
  379           time increases sharply as you increase the setting.</para>
  380 
  381           <note>
  382             <para>For fast lines, the actually download speed may be well
  383             below what you specify here. If you have this problem, then follow
  384             the bandwidth with a ":" and a <firstterm>burst</firstterm> size.
  385             The default burst is 10kb, but on my 50mbit line, I specify 200kb.
  386             (50mbit:200kb).</para>
  387           </note>
  388 
  389           <caution>
  390             <para>Incoming IPSec traffic traverses traffic shaping twice -
  391             firs as encrypted and encapsulated ESP packets and then en clair.
  392             As a result, incoming bandwidth can be significantly less than
  393             specified if IPSEC packets form a significant part of inoming
  394             traffic.</para>
  395           </caution>
  396         </listitem>
  397 
  398         <listitem>
  399           <para>OUT-BANDWIDTH - Specify the outgoing bandwidth of that
  400           interface. This is the maximum speed your connection can handle. It
  401           is also the speed you can refer as "full" if you define the tc
  402           classes. Outgoing traffic above this rate will be dropped.</para>
  403         </listitem>
  404 
  405         <listitem>
  406           <para>OPTIONS — A comma-separated list of options from the following
  407           list:</para>
  408 
  409           <variablelist>
  410             <varlistentry>
  411               <term><emphasis role="bold">classify</emphasis></term>
  412 
  413               <listitem>
  414                 <para>If specified, classification of traffic into the various
  415                 classes is done by CLASSIFY entries in
  416                 <filename>/etc/shorewall/mangle</filename>
  417                 (<filename>/etc/shorewall/tcrules</filename>) or by entries in
  418                 <filename>/etc/shorewall/tcfilters</filename>. No MARK value
  419                 will be associated with classes on this interface.</para>
  420               </listitem>
  421             </varlistentry>
  422 
  423             <varlistentry>
  424               <term><emphasis role="bold">hfsc</emphasis></term>
  425 
  426               <listitem>
  427                 <para>Shorewall normally uses the <firstterm>Hierarchical
  428                 Token Bucket</firstterm> (HTB) queuing discipline. When
  429                 <option>hfsc</option> is specified, the
  430                 <firstterm>Hierarchical Fair Service Curves</firstterm> (HFSC)
  431                 discipline is used instead.</para>
  432               </listitem>
  433             </varlistentry>
  434 
  435             <varlistentry>
  436               <term><emphasis role="bold">linklayer</emphasis></term>
  437 
  438               <listitem>
  439                 <para>Added in Shorewall 4.5.6. Type of link (ethernet, atm,
  440                 adsl). When specified, causes scheduler packet size
  441                 manipulation as described in tc-stab (8). When this option is
  442                 given, the following options may also be given after
  443                 it:</para>
  444 
  445                 <variablelist>
  446                   <varlistentry>
  447                     <term><emphasis
  448                     role="bold">mtu</emphasis>=<replaceable>mtu</replaceable></term>
  449 
  450                     <listitem>
  451                       <para>The device MTU; default 2048 (will be rounded up
  452                       to a power of two)</para>
  453                     </listitem>
  454                   </varlistentry>
  455 
  456                   <varlistentry>
  457                     <term><emphasis
  458                     role="bold">mpu</emphasis>=<replaceable>mpubytes</replaceable></term>
  459 
  460                     <listitem>
  461                       <para>Minimum packet size used in calculations. Smaller
  462                       packets will be rounded up to this size</para>
  463                     </listitem>
  464                   </varlistentry>
  465 
  466                   <varlistentry>
  467                     <term><emphasis
  468                     role="bold">tsize</emphasis>=<replaceable>tablesize</replaceable></term>
  469 
  470                     <listitem>
  471                       <para>Size table entries; default is 512</para>
  472                     </listitem>
  473                   </varlistentry>
  474 
  475                   <varlistentry>
  476                     <term><emphasis
  477                     role="bold">overhead</emphasis>=<replaceable>overheadbytes</replaceable></term>
  478 
  479                     <listitem>
  480                       <para>Number of overhead bytes per packet</para>
  481                     </listitem>
  482                   </varlistentry>
  483                 </variablelist>
  484               </listitem>
  485             </varlistentry>
  486 
  487             <varlistentry>
  488               <term><emphasis role="bold">connmark</emphasis></term>
  489 
  490               <listitem>
  491                 <para>Added in Shorewall 5.2.7. May be specified on IFB
  492                 devices to enable use of firewall marks to select the
  493                 appropriate traffic shaping class.</para>
  494               </listitem>
  495             </varlistentry>
  496           </variablelist>
  497         </listitem>
  498 
  499         <listitem>
  500           <para>REDIRECTED INTERFACES — Entries are appropriate in this column
  501           only if the device in the INTERFACE column names a <link
  502           linkend="IFB">Intermediate Functional Block (IFB)</link>. It lists
  503           the physical interfaces that will have their input shaped using
  504           classes defined on the IFB. Neither the IFB nor any of the
  505           interfaces listed in this column may have an IN-BANDWIDTH specified.
  506           You may specify zero (0) or a dash ("-:) in the IN-BANDWIDTH
  507           column.</para>
  508 
  509           <para>IFB devices automatically get the <emphasis
  510           role="bold">classify</emphasis> option unless the <emphasis
  511           role="bold">connmark</emphasis> option is specified.</para>
  512         </listitem>
  513       </itemizedlist>
  514 
  515       <example id="Example0">
  516         <title/>
  517 
  518         <para>Suppose you are using PPP over Ethernet (DSL) and ppp0 is the
  519         interface for this. The device has an outgoing bandwidth of 500kbit
  520         and an incoming bandwidth of 6000kbit</para>
  521 
  522         <programlisting>#INTERFACE    IN-BANDWITH      OUT-BANDWIDTH
  523 ppp0           6000kbit         500kbit</programlisting>
  524       </example>
  525     </section>
  526 
  527     <section id="tcclasses">
  528       <title>/etc/shorewall/tcclasses</title>
  529 
  530       <para>This file allows you to define the actual classes that are used to
  531       split the outgoing traffic. For additional information, see <ulink
  532       url="manpages/shorewall-tcclasses.html">shorewall-tcclasses</ulink>
  533       (5).</para>
  534 
  535       <itemizedlist>
  536         <listitem>
  537           <para>INTERFACE - Name of interface. Users may also specify the
  538           interface number. Must match the name (or number) of an interface
  539           with an entry in <filename>/etc/shorewall/tcdevices</filename>. If
  540           the interface has the <emphasis role="bold">classify</emphasis>
  541           option in <filename>/etc/shorewall/tcdevices</filename>, then the
  542           interface name or number must be followed by a colon and a
  543           <firstterm>class number</firstterm>. Examples: eth0:1, 4:9. Class
  544           numbers must be unique for a given interface. Normally, all classes
  545           defined here are sub-classes of a root class that is implicitly
  546           defined from the entry in <ulink
  547           url="shorewall-tcdevices.html">shorewall-tcdevices</ulink>(5). You
  548           can establish a class hierarchy by specifying a
  549           <emphasis>parent</emphasis> class (e.g.,
  550           <emphasis>interface</emphasis>:<emphasis>parent-class</emphasis>:<emphasis>class</emphasis>)
  551           -- the number of a class that you have previously defined. The
  552           sub-class may borrow unused bandwidth from its parent.</para>
  553 
  554           <warning>
  555             <para>Class numbers are expressed in hexidecimal. So the class
  556             following class 9 is A, not 10.</para>
  557           </warning>
  558         </listitem>
  559 
  560         <listitem>
  561           <para>MARK - The mark value which is an integer in the range 1-255
  562           (1-16383 if you set WIDE_TC_MARKS=Yes or set TC_BITS=14 in <ulink
  563           url="manpages/shorewall.conf.html">shorewall.conf</ulink> (5) ). You
  564           define these marks in the mangle or tcrules file, marking the
  565           traffic you want to go into the queuing classes defined in here. You
  566           can use the same marks for different Interfaces. You must specify
  567           "-' in this column if the device specified in the INTERFACE column
  568           has the <emphasis role="bold">classify</emphasis> option in
  569           <filename>/etc/shorewall/tcdevices</filename>.</para>
  570 
  571           <note>
  572             <para>In Shorewall 4.5.0, WIDE_TC_MARKS was superseded by TC_BITS
  573             which specifies the width in bits of the traffic shaping mark
  574             field. The default is based on the setting of WIDE_TC_MARKS so as
  575             to provide upward compatibility.</para>
  576           </note>
  577         </listitem>
  578 
  579         <listitem>
  580           <para>RATE - The minimum bandwidth this class should get, when the
  581           traffic load rises. Please note that first the classes which equal
  582           or a lesser priority value are served even if there are others that
  583           have a guaranteed bandwidth but a lower priority. <emphasis
  584           role="bold">If the sum of the RATEs for all classes assigned to an
  585           INTERFACE exceed that interfaces's OUT-BANDWIDTH, then the
  586           OUT-BANDWIDTH limit will not be honored.</emphasis></para>
  587 
  588           <para>When using HFSC, this column may contain 1, 2 or 3 pieces of
  589           information separated by colons (":"). In addition to the minimum
  590           bandwidth, leaf classes may specify realtime criteria: DMAX (maximum
  591           delay in milliseconds) and optionally UMAX (the largest packet
  592           expected in the class). See <link linkend="HFSC">below</link> for
  593           details.</para>
  594         </listitem>
  595 
  596         <listitem>
  597           <para>CEIL - The maximum bandwidth this class is allowed to use when
  598           the link is idle. Useful if you have traffic which can get full
  599           speed when more important services (e.g. interactive like ssh) are
  600           not used. You can use the value "full" in here for setting the
  601           maximum bandwidth to the defined output bandwidth of that
  602           interface.</para>
  603         </listitem>
  604 
  605         <listitem>
  606           <para>PRIORITY - you have to define a priority for the class.
  607           packets in a class with a higher priority (=lesser value) are
  608           handled before less prioritized ones. You can just define the mark
  609           value here also, if you are increasing the mark values with lesser
  610           priority.</para>
  611         </listitem>
  612 
  613         <listitem>
  614           <para>OPTIONS - A comma-separated list of options including the
  615           following:</para>
  616 
  617           <itemizedlist>
  618             <listitem>
  619               <para>default - this is the default class for that interface
  620               where all traffic should go, that is not classified
  621               otherwise.</para>
  622 
  623               <note>
  624                 <para>defining default for exactly <emphasis
  625                 role="bold">one</emphasis> class per interface is
  626                 mandatory!</para>
  627               </note>
  628             </listitem>
  629 
  630             <listitem>
  631               <para>tos-&lt;tosname&gt; - this lets you define a filter for
  632               the given &lt;tosname&gt; which lets you define a value of the
  633               Type Of Service bits in the ip package which causes the package
  634               to go in this class. Please note, that this filter overrides all
  635               mark settings, so if you define a tos filter for a class all
  636               traffic having that mark will go in it regardless of the mark on
  637               the package. You can use the following for this option:
  638               tos-minimize-delay (16) tos-maximize-throughput (8)
  639               tos-maximize-reliability (4) tos-minimize-cost (2)
  640               tos-normal-service (0)</para>
  641 
  642               <note>
  643                 <para>Each of this options is only valid for <emphasis
  644                 role="bold">one</emphasis> class per interface.</para>
  645               </note>
  646             </listitem>
  647 
  648             <listitem>
  649               <para>tcp-ack - if defined causes an tc filter to be created
  650               that puts all tcp ack packets on that interface that have an
  651               size of &lt;=64 Bytes to go in this class. This is useful for
  652               speeding up downloads. Please note that the size of the ack
  653               packets is limited to 64 bytes as some applications (p2p for
  654               example) use to make every package an ack package which would
  655               cause them all into here. We want only packets WITHOUT payload
  656               to match, so the size limit. Bigger packets just take their
  657               normal way into the classes.</para>
  658 
  659               <note>
  660                 <para>This option is only valid for <emphasis
  661                 role="bold">class</emphasis> per interface.</para>
  662               </note>
  663             </listitem>
  664 
  665             <listitem>
  666               <para>occurs=<emphasis>number</emphasis> - Typically used with
  667               an IPMARK entry in mangle or tcrules. Causes the rule to be
  668               replicated for a total of <emphasis>number</emphasis> rules.
  669               Each rule has a successively class number and mark value.</para>
  670 
  671               <para>When 'occurs' is used:</para>
  672 
  673               <itemizedlist>
  674                 <listitem>
  675                   <para>The associated device may not have the 'classify'
  676                   option.</para>
  677                 </listitem>
  678 
  679                 <listitem>
  680                   <para>The class may not be the default class.</para>
  681                 </listitem>
  682 
  683                 <listitem>
  684                   <para>The class may not have any 'tos=' options (including
  685                   'tcp-ack').</para>
  686                 </listitem>
  687 
  688                 <listitem>
  689                   <para>The class should not specify a MARK value. If one is
  690                   specified, it will be ignored with a warning message.</para>
  691                 </listitem>
  692               </itemizedlist>
  693 
  694               <para>The 'RATE' and 'CEIL' parameters apply to each instance of
  695               the class. So the total RATE represented by an entry with
  696               'occurs' will be the listed RATE multiplied by
  697               <emphasis>number</emphasis>. For additional information, see
  698               <ulink url="manpages/shorewall-mangle.html">mangle</ulink> (5)
  699               or <ulink url="manpages/shorewall-tcrules.html">tcrules</ulink>
  700               (5).</para>
  701             </listitem>
  702 
  703             <listitem>
  704               <para>flow=<emphasis>keys</emphasis> - Shorewall attaches an SFQ
  705               queuing discipline to each leaf HTB and HFSC class. SFQ ensures
  706               that each <firstterm>flow</firstterm> gets equal access to the
  707               interface. The default definition of a flow corresponds roughly
  708               to a Netfilter connection. So if one internal system is running
  709               BitTorrent, for example, it can have lots of 'flows' and can
  710               thus take up a larger share of the bandwidth than a system
  711               having only a single active connection. The
  712               <option>flow</option> classifier (module cls_flow) works around
  713               this by letting you define what a 'flow' is. The clasifier must
  714               be used carefully or it can block off all traffic on an
  715               interface! The flow option can be specified for an HTB or HFSC
  716               leaf class (one that has no sub-classes). We recommend that you
  717               use the following:</para>
  718 
  719               <simplelist>
  720                 <member>Shaping internet-bound traffic: <emphasis
  721                 role="bold">flow=nfct-src</emphasis></member>
  722 
  723                 <member>Shaping traffic bound for your local net: <emphasis
  724                 role="bold">flow=dst</emphasis></member>
  725               </simplelist>
  726 
  727               <para>These will cause a 'flow' to consists of the traffic
  728               to/from each internal system.</para>
  729 
  730               <para>When more than one key is give, they must be enclosed in
  731               parenthesis and separated by commas.</para>
  732 
  733               <para>To see a list of the possible flow keys, run this
  734               command:</para>
  735 
  736               <blockquote>
  737                 <para><command>tc filter add flow help</command></para>
  738               </blockquote>
  739 
  740               <para>Those that begin with "nfct-" are Netfilter connection
  741               tracking fields. As shown above, we recommend flow=nfct-src;
  742               that means that we want to use the source IP address
  743               <emphasis>before SNAT</emphasis> as the key.</para>
  744 
  745               <note>
  746                 <para>Shorewall cannot determine ahead of time if the flow
  747                 classifier is available in your kernel (especially if it was
  748                 built into the kernel as opposed to being loaded as a module).
  749                 Consequently, you should check ahead of time to ensure that
  750                 both your kernel and 'tc' utility support the feature.</para>
  751 
  752                 <para>You can test the 'tc' utility by typing (as
  753                 root):</para>
  754 
  755                 <blockquote>
  756                   <para><command>tc filter add flow help</command></para>
  757                 </blockquote>
  758 
  759                 <para>If flow is supported, you will see:</para>
  760 
  761                 <programlisting>   Usage: ... flow ...
  762 
  763       [mapping mode]: map key KEY [ OPS ] ...
  764       [hashing mode]: hash keys KEY-LIST ...
  765 
  766    ...</programlisting>
  767 
  768                 <para>If 'flow' is not supported, you will see:</para>
  769 
  770                 <programlisting>   Unknown filter "flow", hence option "help" is unparsable</programlisting>
  771 
  772                 <para>If your kernel supports module autoloading, just type
  773                 (as root):</para>
  774 
  775                 <blockquote>
  776                   <para><command>modprobe cls_flow</command></para>
  777                 </blockquote>
  778 
  779                 <para>If 'flow' is supported, no output is produced;
  780                 otherwise, you will see:</para>
  781 
  782                 <programlisting>   FATAL: Module cls_flow not found.</programlisting>
  783 
  784                 <para>If your kernel is not modularized or does not support
  785                 module autoloading, look at your kernel configuration (either
  786                 <filename>/proc/config.gz</filename> or the
  787                 <filename>.config</filename> file in <filename
  788                 class="directory">/lib/modules/&lt;kernel-version&gt;/build/</filename></para>
  789 
  790                 <para>If 'flow' is supported, you will see: NET_CLS_FLOW=m or
  791                 NET_CLS_FLOW=y.</para>
  792 
  793                 <para>For modularized kernels, Shorewall will attempt to load
  794                 <filename>/lib/modules/&lt;kernel-version&gt;/net/sched/cls_flow.ko</filename>
  795                 by default.</para>
  796               </note>
  797             </listitem>
  798 
  799             <listitem>
  800               <para>pfifo - When specified for a leaf class, the pfifo queing
  801               discipline is applied to the class rather than the sfq queuing
  802               discipline.</para>
  803             </listitem>
  804 
  805             <listitem>
  806               <para>limit=<emphasis>number</emphasis> - Added in Shorewall
  807               4.4.3. When specified for a leaf class, specifies the maximum
  808               number of packets that may be queued within the class. The
  809               <emphasis>number</emphasis> must be &gt; 2 and less than 128. If
  810               not specified, the value 127 is assumed</para>
  811             </listitem>
  812 
  813             <listitem>
  814               <para>red=(<replaceable>redoption</replaceable>,...) - Added in
  815               Shorewall 4.5.6. When specified on a leaf class, causes the
  816               class to use the red queuing discipline rather than SFQ. See
  817               tc-red (8) for additional information.</para>
  818 
  819               <para>See <ulink
  820               url="manpages/shorewall-tcdevices.html">shorewall-tcdevices</ulink>
  821               (5) for a description of the allowable
  822               <replaceable>redoptions</replaceable>.</para>
  823             </listitem>
  824 
  825             <listitem>
  826               <para>fq_codel[=(<replaceable>codeloption</replaceable>,...)] -
  827               Added in Shorewall 4.5.12. When specified on a leaf class,
  828               causes the class to use the FQ CODEL (<firstterm>Fair-queuing
  829               Controlled-delay</firstterm>) queuing discipline rather than
  830               SFQ. See tc-fq_codel (8) for additional information.</para>
  831 
  832               <para>See <ulink
  833               url="manpages/shorewall-tcclasses.html">shorewall-tcclasses</ulink>
  834               (5) for a description of the allowable
  835               <replaceable>codloptions</replaceable>.</para>
  836             </listitem>
  837           </itemizedlist>
  838         </listitem>
  839       </itemizedlist>
  840     </section>
  841 
  842     <section id="tcrules">
  843       <title>/etc/shorewall/mangle and /etc/shorewall/rules</title>
  844 
  845       <important>
  846         <para>Unlike rules in the <ulink
  847         url="manpages/shorewall-rules.html">shorewall-rules</ulink>(5) file,
  848         evaluation of rules in this file will continue after a match. So the
  849         final mark for each packet will be the one assigned by the LAST tcrule
  850         that matches.</para>
  851 
  852         <para>Also unlike rules in the <ulink
  853         url="manpages/shorewall-rules.html">shorewall-rules</ulink>(5) file,
  854         the mangle (tcrules) file is not stateful. So every packet that goes
  855         into, out of or through your firewall is subject to entries in the
  856         mangle (tcrules) file.</para>
  857 
  858         <para>Because mangle (tcrules) entries are not stateful, it is
  859         necessary to understand basic IP socket operation. Here is an edited
  860         excerpt from a post on the Shorewall Users list:<blockquote>
  861             <para>For the purposes of this discussion, the world is separated
  862             into clients and servers. Servers provide services to
  863             clients.</para>
  864 
  865             <para>When a server starts, it creates a socket and
  866             <emphasis>binds</emphasis> the socket to an
  867             <emphasis>address</emphasis>. For AF_INET (IPv4) and AF_INET6
  868             (IPv6) sockets, that address is an ordered triple consisting of an
  869             IPv4 or IPv6 address, a protocol, and possibly a port number. Port
  870             numbers are only used when the protocol is TCP, UDP, SCTP or DCCP.
  871             The protocol and port number used by a server are typically
  872             well-known so that clients will be able to connect to it or send
  873             datagrams to it. So SSH servers bind to TCP port 22, SMTP servers
  874             bind to TCP port 25, etc. We will call this port the SERVER
  875             PORT.</para>
  876 
  877             <para>When a client want to use the service provided by a server,
  878             it also creates a socket and, like the server's socket, the
  879             client's socket must be bound to an address. But in the case of
  880             the client, the socket is usually given an automatic address
  881             binding. For AF_INET and AF_INET6 sockets. the IP address is the
  882             IP address of the client system (loose generalization) and the
  883             port number is selected from a <firstterm>local port
  884             range</firstterm>. On Linux systems, the local port range can be
  885             seen by <command>cat
  886             /proc/sys/net/ipv4/ip_local_port_range</command>. So it is not
  887             possible in advance to determine what port the client will be
  888             using. Whatever it is, we'll call it the CLIENT PORT.</para>
  889 
  890             <para>Now: <blockquote>
  891                 <para>Packets sent from the client to the server will
  892                 have:<blockquote>
  893                     <para>SOURCE PORT = CLIENT PORT</para>
  894 
  895                     <para>DEST PORT = SERVER PORT</para>
  896                   </blockquote></para>
  897 
  898                 <para>Packets sent from the server to the client will have:
  899                 <blockquote>
  900                     <para>SOURCE PORT = SERVER PORT</para>
  901 
  902                     <para>DEST PORT = CLIENT PORT</para>
  903                   </blockquote></para>
  904               </blockquote></para>
  905 
  906             <para>Since the SERVER PORT is generally the only port known ahead
  907             of time, we must categorize traffic from the server to the client
  908             using the SOURCE PORT.</para>
  909           </blockquote></para>
  910       </important>
  911 
  912       <para>The fwmark classifier provides a convenient way to classify
  913       packets for traffic shaping. The
  914       <filename>/etc/shorewall/mangle</filename>
  915       (<filename>/etc/shorewall/tcrules</filename>) file is used for
  916       specifying these marks in a tabular fashion. For an in-depth look at the
  917       packet marking facility in Netfilter/Shorewall, please see <ulink
  918       url="PacketMarking.html">this article</ulink>.</para>
  919 
  920       <para><emphasis role="bold">For marking forwarded traffic, you must
  921       either set MARK_IN_FORWARD_CHAIN=Yes shorewall.conf or by using the :F
  922       qualifier (see below).</emphasis></para>
  923 
  924       <para>See shorewall-mangle(5) and shorewall-tcrules(5) for a description
  925       of the entries in these files. Note that the mangle file superseded the
  926       tcrules file in Shorewall 4.6.0.</para>
  927 
  928       <para>The following examples are for the mangle file.</para>
  929 
  930       <example id="Example1">
  931         <title/>
  932 
  933         <para>All packets arriving on eth1 should be marked with 1. All
  934         packets arriving on eth2 and eth3 should be marked with 2. All packets
  935         originating on the firewall itself should be marked with 3.</para>
  936 
  937         <programlisting>#ACTION       SOURCE    DEST           PROTO     DPORT
  938 MARK(1)       eth1      0.0.0.0/0      all
  939 MARK(2)       eth2      0.0.0.0/0      all
  940 MARK(2)       eth3      0.0.0.0/0      all
  941 MARK(3)       $FW       0.0.0.0/0      all</programlisting>
  942       </example>
  943 
  944       <example id="Example2">
  945         <title/>
  946 
  947         <para>All GRE (protocol 47) packets destined for 155.186.235.151
  948         should be marked with 12.</para>
  949 
  950         <programlisting>#ACTION       SOURCE    DEST            PROTO      DPORT
  951 MARK(12):T    0.0.0.0/0 155.182.235.151 47</programlisting>
  952       </example>
  953 
  954       <example id="Example3">
  955         <title/>
  956 
  957         <para>All SSH request packets originating in 192.168.1.0/24 and
  958         destined for 155.186.235.151 should be marked with 22.</para>
  959 
  960         <programlisting>#ACTION       SOURCE         DEST            PROTO      DPORT
  961 MARK(22):T    192.168.1.0/24 155.182.235.151 tcp        22</programlisting>
  962       </example>
  963 
  964       <example id="Example4">
  965         <title/>
  966 
  967         <para>All SSH packets packets going out of the first device in in
  968         /etc/shorewall/tcdevices should be assigned to the class with mark
  969         value 10.</para>
  970 
  971         <programlisting>#ACTION           SOURCE         DEST            PROTO      DPORT           SPORT
  972 CLASSIFY(1:110)   0.0.0.0/0      0.0.0.0/0       tcp        22
  973 CLASSIFY(1:110)   0.0.0.0/0      0.0.0.0/0       tcp        -               22</programlisting>
  974       </example>
  975 
  976       <example id="Example5">
  977         <title/>
  978 
  979         <para>Mark all ICMP echo traffic with packet mark 1. Mark all peer to
  980         peer traffic with packet mark 4.</para>
  981 
  982         <para>This is a little more complex than otherwise expected. Since the
  983         ipp2p module is unable to determine all packets in a connection are
  984         P2P packets, we mark the entire connection as P2P if any of the
  985         packets are determined to match. We assume packet/connection mark 0 to
  986         means unclassified. Traffic originating on the firewall is not covered
  987         by this example.</para>
  988 
  989         <programlisting>#ACTION        SOURCE         DEST            PROTO      DPORT         SPORT    USER      TEST
  990 MARK(1)        0.0.0.0/0      0.0.0.0/0       icmp       echo-request
  991 MARK(1)        0.0.0.0/0      0.0.0.0/0       icmp       echo-reply
  992 
  993 RESTORE        0.0.0.0/0      0.0.0.0/0       all        -             -        -         0
  994 CONTINUE       0.0.0.0/0      0.0.0.0/0       all        -             -        -         !0
  995 MARK(4)        0.0.0.0/0      0.0.0.0/0       ipp2p:all
  996 SAVE           0.0.0.0/0      0.0.0.0/0       all        -             -        -         !0</programlisting>
  997 
  998         <para>The last four rules can be translated as:</para>
  999 
 1000         <blockquote>
 1001           <para>"If a packet hasn't been classified (packet mark is 0), copy
 1002           the connection mark to the packet mark. If the packet mark is set,
 1003           we're done. If the packet is P2P, set the packet mark to 4. If the
 1004           packet mark has been set, save it to the connection mark."</para>
 1005         </blockquote>
 1006       </example>
 1007 
 1008       <example>
 1009         <title/>
 1010 
 1011         <para>Mark all forwarded VOIP connections with connection mark 1 and
 1012         ensure that all VOIP packets also receive that mark (assumes that
 1013         nf_conntrack_sip is loaded).</para>
 1014 
 1015         <programlisting>#ACTION  SOURCE         DEST            PROTO      DPORT         SPORT    USER      TEST      CONNBYTES      TOS      HELPER
 1016 RESTORE  0.0.0.0/0      0.0.0.0/0       all        -             -        -         0
 1017 CONTINUE 0.0.0.0/0      0.0.0.0/0       all        -             -        -         !0
 1018 1        0.0.0.0/0      0.0.0.0/0       all        -             -        -         -         -              -        sip
 1019 SAVE     0.0.0.0/0      0.0.0.0/0       all        -             -        -         !0</programlisting>
 1020       </example>
 1021     </section>
 1022 
 1023     <section id="ppp">
 1024       <title>ppp devices</title>
 1025 
 1026       <para>If you use ppp/pppoe/pppoa) to connect to your Internet provider
 1027       and you use traffic shaping you need to restart shorewall traffic
 1028       shaping. The reason for this is, that if the ppp connection gets
 1029       restarted (and it usually does this at least daily), all
 1030       <quote>tc</quote> filters/qdiscs related to that interface are
 1031       deleted.</para>
 1032 
 1033       <para>The easiest way to achieve this, is just to restart shorewall once
 1034       the link is up. To achieve this add a small executable script
 1035       to<quote>/etc/ppp/ip-up.d</quote>.</para>
 1036 
 1037       <programlisting>#! /bin/sh
 1038 
 1039 /sbin/shorewall refresh</programlisting>
 1040     </section>
 1041 
 1042     <section>
 1043       <title>Sharing a TC configuration between Shorewall and
 1044       Shorewall6</title>
 1045 
 1046       <para>Beginning with Shorewall 4.4.15, the traffic-shaping configuration
 1047       in the tcdevices, tcclasses and tcfilters files can be shared between
 1048       Shorewall and Shorewall6. Only one of the products can control the
 1049       configuration but the other can configure CLASSIFY rules in its own
 1050       mangle or tcrules file that refer to the shared classes.</para>
 1051 
 1052       <para>To defined the configuration in Shorewall and shared it with
 1053       Shorewall6:</para>
 1054 
 1055       <itemizedlist>
 1056         <listitem>
 1057           <para>Set TC_ENABLED=Internal in <ulink
 1058           url="manpages/shorewall.conf.html">shorewall.conf</ulink>
 1059           (5).</para>
 1060         </listitem>
 1061 
 1062         <listitem>
 1063           <para>Set TC_ENABLED=Shared in <ulink
 1064           url="manpages/shorewall.conf.html">shorewall6.conf</ulink>
 1065           (5).</para>
 1066         </listitem>
 1067 
 1068         <listitem>
 1069           <para>Create symbolic links from /etc/shorewall6 to
 1070           /etc/shorewall/tcdevices and /etc/shorewall/tcclasses:</para>
 1071 
 1072           <programlisting>ln -s ../shorewall/tcdevices /etc/shorewall6/tcdevices
 1073 ln -s ../shorewall/tcclasses /etc/shorewall6/tcclasses</programlisting>
 1074         </listitem>
 1075 
 1076         <listitem>
 1077           <para>If you need to define IPv6 tcfilter entries, do so in
 1078           /etc/shorewall/tcfilters. That file now allows entries that apply to
 1079           IPv6.</para>
 1080         </listitem>
 1081       </itemizedlist>
 1082 
 1083       <para>Shorewall6 compilations to have access to the tcdevices and
 1084       tcclasses files although it will create no output. That access allows
 1085       CLASSIFY rules in /etc/shorewall6/mangle to be validated against the TC
 1086       configuration.</para>
 1087 
 1088       <para>In this configuration, it is Shorewall that controls TC
 1089       configuration (except for IPv6 mangle). You can reverse the settings in
 1090       the files if you want to control the configuration using
 1091       Shorewall6.</para>
 1092     </section>
 1093 
 1094     <section id="perIP">
 1095       <title>Per-IP Traffic Shaping</title>
 1096 
 1097       <para>Some network administrators feel that they have to divy up their
 1098       available bandwidth by IP address rather than by prioritizing the
 1099       traffic based on the type of traffic. This gets really awkward when
 1100       there are a large number of local IP addresses.</para>
 1101 
 1102       <para>This section describes the Shorewall facility for making this
 1103       configuration less tedious (and a lot more efficient). Note that it
 1104       requires that you <ulink url="Dynamic.html#xtables-addons">install
 1105       xtables-addons</ulink>. So before you try this facility, we suggest that
 1106       first you add the following OPTION to each external interface described
 1107       in /etc/shorewall/tcdevices:</para>
 1108 
 1109       <programlisting>flow=nfct-src</programlisting>
 1110 
 1111       <para>If you shape traffic on your internal interface(s), then add this
 1112       to their entries:</para>
 1113 
 1114       <programlisting>flow=dst</programlisting>
 1115 
 1116       <para>You may find that this simple change is all that is needed to
 1117       control bandwidth hogs like Bit Torrent. If it doesn't, then proceed as
 1118       described in this section.</para>
 1119 
 1120       <para>The facility has two components:</para>
 1121 
 1122       <orderedlist>
 1123         <listitem>
 1124           <para>An IPMARK MARKing command in
 1125           <filename>/etc/shorewall/mangle</filename>
 1126           (<filename>/etc/shorewall/tcrules</filename>).</para>
 1127         </listitem>
 1128 
 1129         <listitem>
 1130           <para>An <emphasis role="bold">occurs</emphasis> OPTION in
 1131           /etc/shorewall/tcclasses.</para>
 1132         </listitem>
 1133       </orderedlist>
 1134 
 1135       <para>The facility is currently only available with IPv4.</para>
 1136 
 1137       <para>In a sense, the IPMARK target is more like an IPCLASSIFY target in
 1138       that the mark value is later interpreted as a class ID. A packet mark is
 1139       32 bits wide; so is a class ID. The <emphasis>major</emphasis> class
 1140       occupies the high-order 16 bits and the <emphasis>minor</emphasis> class
 1141       occupies the low-order 16 bits. So the class ID 1:4ff (remember that
 1142       class IDs are always in hex) is equivalent to a mark value of 0x104ff.
 1143       Remember that Shorewall uses the interface number as the
 1144       <emphasis>major</emphasis> number where the first interface in tcdevices
 1145       has <emphasis>major</emphasis> number 1, the second has
 1146       <emphasis>major</emphasis> number 2, and so on.</para>
 1147 
 1148       <para>The IPMARK target assigns a mark to each matching packet based on
 1149       the either the source or destination IP address. By default, it assigns
 1150       a mark value equal to the low-order 8 bits of the source address.</para>
 1151 
 1152       <para>The syntax is as follows:</para>
 1153 
 1154       <blockquote>
 1155         <para><emphasis role="bold">IPMARK</emphasis>[<emphasis
 1156         role="bold">(</emphasis>[{<emphasis
 1157         role="bold">src</emphasis>|<emphasis
 1158         role="bold">dst</emphasis>}][<emphasis
 1159         role="bold">,</emphasis>[<emphasis>mask1</emphasis>][,[<emphasis>mask2</emphasis>][<emphasis
 1160         role="bold">,</emphasis>[<emphasis>shift</emphasis>]]]]<emphasis
 1161         role="bold">)</emphasis>]</para>
 1162       </blockquote>
 1163 
 1164       <para>Default values are:</para>
 1165 
 1166       <simplelist>
 1167         <member>src</member>
 1168 
 1169         <member>mask1 = 0xFF</member>
 1170 
 1171         <member>mask2 = 0x00</member>
 1172 
 1173         <member>shift = 0</member>
 1174       </simplelist>
 1175 
 1176       <para><emphasis role="bold">src</emphasis> and <emphasis
 1177       role="bold">dst</emphasis> specify whether the mark is to be based on
 1178       the source or destination address respectively. The selected address is
 1179       first shifted right by <emphasis>shift</emphasis>, then LANDed with
 1180       <emphasis>mask1</emphasis> and then LORed with
 1181       <emphasis>mask2</emphasis>. The <emphasis>shift</emphasis> argument is
 1182       intended to be used primarily with IPv6 addresses.</para>
 1183 
 1184       <para>Example:</para>
 1185 
 1186       <programlisting>IPMARK(src,0xff,0x10100)
 1187 
 1188 Source IP address is 192.168.4.3 = 0xc0a80403
 1189 
 1190         0xc0a80403 &gt;&gt; 0         = 0xc0a80403
 1191         0xc0a80403 LAND 0xFF    = 0x03
 1192         0x03       LOR  0x10100 = 0x10103
 1193 
 1194         So the mark value is 0x10103 which corresponds to class id 1:103.</programlisting>
 1195 
 1196       <para>It is important to realize that, while class IDs are composed of a
 1197       <emphasis>major</emphasis> and a <emphasis>minor</emphasis> value, the
 1198       set of <emphasis>minor</emphasis> values must be unique. You must keep
 1199       this in mind when deciding how to map IP addresses to class IDs. For
 1200       example, suppose that your internal network is 192.168.1.0/29 (host IP
 1201       addresses 192.168.1.1 - 192.168.1.6). Your first notion might be to use
 1202       IPMARK(src,0xFF,0x10000) so as to produce class IDs 1:1 through 1:6. But
 1203       1:1 is the class ID of the base HTB class on interface 1. So you might
 1204       chose instead to use IPMARK(src,0xFF,0x10100) as shown in the example
 1205       above so as to avoid minor class 1.</para>
 1206 
 1207       <para>The <emphasis role="bold">occurs</emphasis> option in
 1208       <filename>/etc/shorewall/tcclasses</filename> causes the class
 1209       definition to be replicated many times.</para>
 1210 
 1211       <para>The synax is:</para>
 1212 
 1213       <blockquote>
 1214         <para><emphasis
 1215         role="bold">occurs=</emphasis><emphasis>number</emphasis></para>
 1216       </blockquote>
 1217 
 1218       <para>When <emphasis role="bold">occurs</emphasis> is used:</para>
 1219 
 1220       <orderedlist numeration="loweralpha">
 1221         <listitem>
 1222           <para>The associated device may not have the <emphasis
 1223           role="bold">classify</emphasis> option.</para>
 1224         </listitem>
 1225 
 1226         <listitem>
 1227           <para>The class may not be the default class.</para>
 1228         </listitem>
 1229 
 1230         <listitem>
 1231           <para>The class may not have any <emphasis
 1232           role="bold">tos=</emphasis> options (including <emphasis
 1233           role="bold">tcp-ack</emphasis>).</para>
 1234         </listitem>
 1235       </orderedlist>
 1236 
 1237       <para>The class should not specify a MARK value. Any MARK value given is
 1238       ignored with a warning. The RATE and CEIL parameters apply to each
 1239       instance of the class. So the total RATE represented by an entry with
 1240       <emphasis role="bold">occurs</emphasis> will be the listed RATE
 1241       multiplied by <emphasis>number</emphasis>.</para>
 1242 
 1243       <para>Example:</para>
 1244 
 1245       <para><filename>/etc/shorewall/tcdevices</filename>:</para>
 1246 
 1247       <programlisting>#INTERFACE IN_BANDWIDTH OUT_BANDWIDTH
 1248 eth0       100mbit      100mbit</programlisting>
 1249 
 1250       <para><filename>/etc/shorewall/tcclasses</filename>:</para>
 1251 
 1252       <programlisting>#DEVICE   MARK RATE     CEIL PRIORITY    OPTIONS
 1253 eth0:101     - 1kbit 230kbit        4    occurs=6</programlisting>
 1254 
 1255       <para>The above defines 6 classes with class IDs 0x101-0x106. Each class
 1256       has a guaranteed rate of 1kbit/second and a ceiling of 230kbit.</para>
 1257 
 1258       <para><filename>/etc/shoreall/mangle</filename> or
 1259       <filename>/etc/shoreall/tcrules</filename>:</para>
 1260 
 1261       <programlisting>#ACTION                          SOURCE             DEST
 1262 IPMARK(src,0xff,0x10100):F       192.168.1.0/29     eth0</programlisting>
 1263 
 1264       <para>This facility also alters the way in which Shorewall generates a
 1265       class number when none is given. Prior to the implementation of this
 1266       facility, the class number was constructed by concatinating the MARK
 1267       value with the either '1' or '10'. '10' was used when there were more
 1268       than 10 devices defined in
 1269       <filename>/etc/shorewall/tcdevices</filename>.</para>
 1270 
 1271       <para>With this facility, a new method is added; class numbers are
 1272       assigned sequentially beginning with 2. The WIDE_TC_MARKS option in
 1273       <filename>shorewall.conf</filename> selects which construction to use.
 1274       WIDE_TC_MARKS=No (the default) produces pre-Shorewall 4.4 behavior.
 1275       WIDE_TC_MARKS=Yes (TC_BITS &gt;= 14 in Shorewall 4.4.26 and later)
 1276       produces the new behavior.</para>
 1277     </section>
 1278 
 1279     <section id="Real">
 1280       <title>Real life examples</title>
 1281 
 1282       <section>
 1283         <title>A Shorewall User's Experience</title>
 1284 
 1285         <para>Chuck Kollars has provided <ulink
 1286         url="http://www.ckollars.org/shaping.html">an excellent
 1287         writeup</ulink> about his traffic shaping experiences.</para>
 1288       </section>
 1289 
 1290       <section id="Wondershaper">
 1291         <title>Configuration to replace Wondershaper</title>
 1292 
 1293         <para>You are able to fully replace the wondershaper script by using
 1294         the buitin traffic control.. In this example it is assumed that your
 1295         interface for your Internet connection is ppp0 (for DSL), if you use
 1296         another connection type, you have to change it. You also need to
 1297         change the settings in the tcdevices.wondershaper file to reflect your
 1298         line speed. The relevant lines of the config files follow here. Please
 1299         note that this is just a 1:1 replacement doing exactly what
 1300         wondershaper should do. You are free to change it...</para>
 1301 
 1302         <section id="realtcd">
 1303           <title>tcdevices file</title>
 1304 
 1305           <programlisting>#INTERFACE      IN_BANDWITH     OUT_BANDWIDTH
 1306 ppp0            5000kbit        500kbit</programlisting>
 1307         </section>
 1308 
 1309         <section id="realtcc">
 1310           <title>tcclasses file</title>
 1311 
 1312           <programlisting>#INTERFACE      MARK    RATE            CEIL        PRIORITY    OPTIONS
 1313 ppp0            1       5*full/10       full            1       tcp-ack,tos-minimize-delay
 1314 ppp0            2       3*full/10       9*full/10       2       default
 1315 ppp0            3       2*full/10       8*full/10       2</programlisting>
 1316         </section>
 1317 
 1318         <section id="realtcr">
 1319           <title>mangle file</title>
 1320 
 1321           <programlisting>#ACTION         SOURCE          DEST            PROTO   DPORT    SPORT   USER
 1322 MARK(1):F       0.0.0.0/0       0.0.0.0/0       icmp    echo-request
 1323 MARK(1):F       0.0.0.0/0       0.0.0.0/0       icmp    echo-reply
 1324 # mark traffic which should have a lower priority with a 3:
 1325 # mldonkey
 1326 MARK(3):F       0.0.0.0/0       0.0.0.0/0       udp     -        4666</programlisting>
 1327 
 1328           <para>Wondershaper allows you to define a set of hosts and/or ports
 1329           you want to classify as low priority. To achieve this , you have to
 1330           add these hosts to tcrules and set the mark to 3 (true if you use
 1331           the example configuration files).</para>
 1332         </section>
 1333 
 1334         <section id="lowpro">
 1335           <title>Setting hosts to low priority</title>
 1336 
 1337           <para>lets assume the following settings from your old wondershaper
 1338           script (don't assume these example values are really useful, they
 1339           are only used for demonstrating ;-):</para>
 1340 
 1341           <programlisting>
 1342 # low priority OUTGOING traffic - you can leave this blank if you want
 1343 # low priority source netmasks
 1344 NOPRIOHOSTSRC="192.168.1.128/25 192.168.3.28"
 1345 
 1346 # low priority destination netmasks
 1347 NOPRIOHOSTDST=60.0.0.0/24
 1348 
 1349 # low priority source ports
 1350 NOPRIOPORTSRC="6662 6663"
 1351 
 1352 # low priority destination ports
 1353 NOPRIOPORTDST="6662 6663"  </programlisting>
 1354 
 1355           <para>This would result in the following additional settings to the
 1356           mangle file:</para>
 1357 
 1358           <programlisting>#ACTION                SOURCE          DEST           PROTO   DPORT     SPORT   USER
 1359 MARK(3)               192.168.1.128/25 0.0.0.0/0      all
 1360 MARK(3)               192.168.3.28     0.0.0.0/0      all
 1361 MARK(3)               0.0.0.0/0        60.0.0.0/24    all
 1362 MARK(3)               0.0.0.0/0        0.0.0.0/0      udp     6662,6663
 1363 MARK(3)               0.0.0.0/0        0.0.0.0/0      udp     -         6662,6663
 1364 MARK(3)               0.0.0.0/0        0.0.0.0/0      tcp     6662,6663
 1365 MARK(3)               0.0.0.0/0        0.0.0.0/0      tcp     -         6662,6663</programlisting>
 1366         </section>
 1367       </section>
 1368 
 1369       <section id="simiple">
 1370         <title>A simple setup</title>
 1371 
 1372         <para>This is a simple setup for people sharing an Internet connection
 1373         and using different computers for this. It just basically shapes
 1374         between 2 hosts which have the ip addresses 192.168.2.23 and
 1375         192.168.2.42</para>
 1376 
 1377         <section id="simpletcd">
 1378           <title>tcdevices file</title>
 1379 
 1380           <programlisting>#INTERFACE      IN_BANDWITH     OUT_BANDWIDTH
 1381 ppp0            6000kbit        700kbit</programlisting>
 1382 
 1383           <para>We have 6mbit down and 700kbit upstream.</para>
 1384         </section>
 1385 
 1386         <section id="simpletcc">
 1387           <title>tcclasses file</title>
 1388 
 1389           <programlisting>#INTERFACE      MARK    RATE            CEIL            PRIORITY    OPTIONS
 1390 ppp0            1       10kbit          50kbit          1           tcp-ack,tos-minimize-delay
 1391 ppp0            2       300kbit         full            2
 1392 ppp0            3       300kbit         full            2
 1393 ppp0            4       90kbit          200kbit         3           default</programlisting>
 1394 
 1395           <para>We add a class for tcp ack packets with highest priority, so
 1396           that downloads are fast. The following 2 classes share most of the
 1397           bandwidth between the 2 hosts, if the connection is idle, they may
 1398           use full speed. As the hosts should be treated equally they have the
 1399           same priority. The last class is for the remaining traffic.</para>
 1400         </section>
 1401 
 1402         <section id="simpletcr">
 1403           <title>mangle file</title>
 1404 
 1405           <programlisting>#ACTION               SOURCE          DEST            PROTO   DPORT        SPORT     USER
 1406 MARK(1):F             0.0.0.0/0       0.0.0.0/0       icmp    echo-request
 1407 MARK(1):F             0.0.0.0/0       0.0.0.0/0       icmp    echo-reply
 1408 MARK(2):F             192.168.2.23    0.0.0.0/0       all
 1409 MARK(3):F             192.168.2.42    0.0.0.0/0       all</programlisting>
 1410 
 1411           <para>Corresponding tcrules file:</para>
 1412 
 1413           <programlisting>#ACTION         SOURCE          DEST            PROTO   DPORT        SPORT     USER
 1414 1:F             0.0.0.0/0       0.0.0.0/0       icmp    echo-request
 1415 1:F             0.0.0.0/0       0.0.0.0/0       icmp    echo-reply
 1416 2:F             192.168.2.23    0.0.0.0/0       all
 1417 3:F             192.168.2.42    0.0.0.0/0       all</programlisting>
 1418 
 1419           <para>We mark icmp ping and replies so they will go into the fast
 1420           interactive class and set a mark for each host.</para>
 1421         </section>
 1422       </section>
 1423     </section>
 1424   </section>
 1425 
 1426   <section id="Xen">
 1427     <title>A Warning to Xen Users</title>
 1428 
 1429     <para>If you are running traffic shaping in your dom0 and traffic shaping
 1430     doesn't seem to be limiting outgoing traffic properly, it may be due to
 1431     "checksum offloading" in your domU(s). Check the output of "shorewall show
 1432     tc". Here's an excerpt from the output of that command:</para>
 1433 
 1434     <programlisting>class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1500 rate 76000bit <emphasis
 1435         role="bold">ceil 230000bit</emphasis> burst 1537b/8 mpu 0b overhead 0b cburst 1614b/8 mpu 0b overhead 0b level 0 
 1436  Sent 559018700 bytes 75324 pkt (dropped 0, overlimits 0 requeues 0) 
 1437  <emphasis role="bold">rate 299288bit</emphasis> 3pps backlog 0b 0p requeues 0 
 1438  lended: 53963 borrowed: 21361 <emphasis role="bold">giants: 90174</emphasis>
 1439  tokens: -26688 ctokens: -14783</programlisting>
 1440 
 1441     <para>There are two obvious problems in the above output:</para>
 1442 
 1443     <orderedlist>
 1444       <listitem>
 1445         <para>The rate (299288) is considerably larger than the ceiling
 1446         (230000).</para>
 1447       </listitem>
 1448 
 1449       <listitem>
 1450         <para>There are a large number (90174) of giants reported.</para>
 1451       </listitem>
 1452     </orderedlist>
 1453 
 1454     <para>This problem will be corrected by disabling "checksum offloading" in
 1455     your domU(s) using the <command>ethtool</command> utility. See the <ulink
 1456     url="XenMyWay-Routed.html">one of the Xen articles</ulink> for
 1457     instructions.</para>
 1458   </section>
 1459 
 1460   <section id="HFSC">
 1461     <title>An HFSC Example</title>
 1462 
 1463     <para>As mentioned at the top of this article, there is an excellent
 1464     introduction to HFSC at <ulink
 1465     url="http://linux-ip.net/articles/hfsc.en/">http://linux-ip.net/articles/hfsc.en/</ulink>.
 1466     At the end of that article are 'tc' commands that implement the
 1467     configuration in the article. Those tc commands correspond to the
 1468     following Shorewall traffic shaping configuration.</para>
 1469 
 1470     <para><filename>/etc/shorewall/tcdevices</filename>:</para>
 1471 
 1472     <programlisting>#INTERFACE    IN_BANDWITH      OUT_BANDWIDTH          OPTIONS
 1473 eth0          -                1000kbit               hfsc</programlisting>
 1474 
 1475     <para><filename>/etc/shorewall/tcclasses</filename>:</para>
 1476 
 1477     <programlisting>#INTERFACE           MARK      RATE               CEIL     PRIORITY      OPTIONS
 1478 1:10                 1         500kbit            full     1
 1479 1:20                 2         500kbit            full     1
 1480 1:10:11              3         400kbit:53ms:1500b full     2
 1481 1:10:12              4         100kbit:30ms:1500b full     2</programlisting>
 1482 
 1483     <para>The following sub-section offers some notes about the
 1484     article.</para>
 1485 
 1486     <section id="MajicNumbers">
 1487       <title>Where Did all of those Magic Numbers come from?</title>
 1488 
 1489       <para>As you read the article, numbers seem to be introduced out of thin
 1490       air. I'll try to shed some light on those.</para>
 1491 
 1492       <para>There is very clear development of these numbers:</para>
 1493 
 1494       <itemizedlist>
 1495         <listitem>
 1496           <para>12ms to transfer a 1500b packet at 1000kbits/second.</para>
 1497         </listitem>
 1498 
 1499         <listitem>
 1500           <para>100kbits per second with 1500b packets, requires 8 packets per
 1501           second.</para>
 1502         </listitem>
 1503 
 1504         <listitem>
 1505           <para>A packet from class 1:12 must be sent every 120ms.</para>
 1506         </listitem>
 1507 
 1508         <listitem>
 1509           <para>Total transmit delay can be no more than 132ms (120 +
 1510           12).</para>
 1511         </listitem>
 1512       </itemizedlist>
 1513 
 1514       <para>We then learn that the queuing latency can be reduced to 30ms if
 1515       we use a two-part service curve whose first part is 400kbits/second.
 1516       Where did those come from?</para>
 1517 
 1518       <itemizedlist>
 1519         <listitem>
 1520           <para>The latency is calculated from the rate. If it takes 12ms to
 1521           transmit a 1500 byte packet at 1000kbits/second, it takes 30ms to
 1522           transmit a 1500b at 400kbits/second.</para>
 1523         </listitem>
 1524 
 1525         <listitem>
 1526           <para>For the slope of the first part of the service curve, in
 1527           theory we can pick any number between 100 (the rate of class 1:12)
 1528           and 500 (the rate of the parent class) with higher numbers providing
 1529           lower latency.</para>
 1530         </listitem>
 1531       </itemizedlist>
 1532 
 1533       <para>The final curious number is the latency for class 1:11 - 52.5ms.
 1534       It is a consequence of everything that has gone before.</para>
 1535 
 1536       <para>To acheive 400kbits/second with 1500-byte packets, 33.33 packets
 1537       per second are required. So a packet from class 1:11 must be sent every
 1538       30 ms. As the article says, "...the maximum transmission delay of this
 1539       class increases from 30ms to a total of 52.5 ms.". So we are looking for
 1540       an additional 22.5 ms.</para>
 1541 
 1542       <para>Assume that both class 1:11 and 1:12 transmit for 30 ms at
 1543       400kbits/second. That is a total of 800kbits/second for 30ms. So Class
 1544       1:11 is punished for the excess. How long is the punishment? The two
 1545       classes sent 24,000 bits in 30ms; they are only allowed 0.030 * 500,000
 1546       = 15,000. So they are 9,000 bits over their quota. The amount of time
 1547       required to transmit 9,000 bits at 400,000 bits/second is
 1548       22.5ms!.</para>
 1549     </section>
 1550   </section>
 1551 
 1552   <section id="IFB">
 1553     <title>Intermediate Functional Block (IFB) Devices</title>
 1554 
 1555     <para>The principles behind an IFB is fairly simple:</para>
 1556 
 1557     <itemizedlist>
 1558       <listitem>
 1559         <para>It looks like a network interface although it is never given an
 1560         IPv4 configuration.</para>
 1561       </listitem>
 1562 
 1563       <listitem>
 1564         <para>Because it is a network interface, queuing disciplines can be
 1565         associated with an IFB.</para>
 1566       </listitem>
 1567     </itemizedlist>
 1568 
 1569     <para>The magic of an IFB comes in the fact that a filter may be defined
 1570     on a real network interface such that each packet that arrives on that
 1571     interface is queued for the IFB! In that way, the IFB provides a means for
 1572     shaping input traffic.</para>
 1573 
 1574     <para>To use an IFB, you must have IFB support in your kernel
 1575     (configuration option CONFIG_IFB). Assuming that you have a modular
 1576     kernel, the name of the IFB module is 'ifb' and may be loaded using the
 1577     command <command>modprobe ifb</command> (if you have modprobe installed)
 1578     or <command>insmod /path/to/module/ifb</command>.</para>
 1579 
 1580     <para>By default, two IFB devices (ifb0 and ifb1) are created. You can
 1581     control that using the numifbs option (e.g., <command>modprobe ifb
 1582     numifbs=1</command>).</para>
 1583 
 1584     <para>To create a single IFB when Shorewall starts, place the following
 1585     two commands in <filename>/etc/shorewall/init</filename>:</para>
 1586 
 1587     <programlisting><command>modprobe ifb numifbs=1
 1588 ip link set ifb0 up</command></programlisting>
 1589 
 1590     <para>Entries in <filename>/etc/shorewall/mangle</filename> or
 1591     <filename>/etc/shorewall/tcrules</filename> have no effect on shaping
 1592     traffic through an IFB unless the IFB is defined in shorewall-tcclasses(5)
 1593     with the <emphasis role="bold">connmark</emphasis> option. To allow
 1594     classification of such traffic, the /etc/shorewall/tcfilters file has been
 1595     added. Entries in that file create <ulink
 1596     url="http://b42.cz/notes/u32_classifier/">u32 classification
 1597     rules</ulink>.</para>
 1598 
 1599     <section id="tcfilters">
 1600       <title>/etc/shorewall/tcfilters</title>
 1601 
 1602       <para>While this file was created to allow shaping of traffic through an
 1603       IFB, the file may be used for general traffic classification as well.
 1604       The file is similar to <ulink
 1605       url="shorewall-mangle.html">shorewall-mangle</ulink>(5) with the
 1606       following key exceptions:</para>
 1607 
 1608       <itemizedlist>
 1609         <listitem>
 1610           <para>The first match determines the classification, whereas in the
 1611           mangle file, the last match determines the classification.</para>
 1612         </listitem>
 1613 
 1614         <listitem>
 1615           <para>ipsets are not supported</para>
 1616         </listitem>
 1617 
 1618         <listitem>
 1619           <para>DNS Names are not supported</para>
 1620         </listitem>
 1621 
 1622         <listitem>
 1623           <para>Address ranges and lists are not supported</para>
 1624         </listitem>
 1625 
 1626         <listitem>
 1627           <para>Exclusion is not supported.</para>
 1628         </listitem>
 1629 
 1630         <listitem>
 1631           <para>filters are applied to packets as they <emphasis>appear on the
 1632           wire</emphasis>. So incoming packets will not have DNAT applied yet
 1633           (the destination IP address will be the external address) and
 1634           outgoing packets will have had SNAT applied.</para>
 1635         </listitem>
 1636       </itemizedlist>
 1637 
 1638       <para>The last point warrants elaboration. When looking at traffic being
 1639       shaped by an IFB, there are two cases to consider:</para>
 1640 
 1641       <orderedlist>
 1642         <listitem>
 1643           <para>Requests — packets being sent from remote clients to local
 1644           servers. These packets may undergo subsequent DNAT, either as a
 1645           result of entries in <filename>/etc/shorewall/nat</filename> or as a
 1646           result of DNAT or REDIRECT rules.</para>
 1647 
 1648           <para>Example: <filename>/etc/shorewall/rules</filename>:</para>
 1649 
 1650           <programlisting>#ACTION       SOURCE           DEST            PROTO    DPORT           SPORT          ORIGDEST
 1651 DNAT          net              dmz:192.168.4.5 tcp      80              -              206.124.146.177</programlisting>
 1652 
 1653           <para>Requests redirected by this rule will have destination IP
 1654           address 206.124.146.177 and destination port 80.</para>
 1655         </listitem>
 1656 
 1657         <listitem>
 1658           <para>Responses — packets being sent from remote servers to local
 1659           clients. These packets may undergo subsequent DNAT as a result of
 1660           entries in <filename>/etc/shorewall/nat</filename> or in
 1661           <filename>/etc/shorewall/masq</filename>. The packet's destination
 1662           IP address will be the external address specified in the
 1663           entry.</para>
 1664 
 1665           <para>Example:
 1666           <filename>/etc/shorewall/masq</filename>:<programlisting>#INTERFACE        SOURCE           ADDRESS
 1667 eth0              192.168.1.0/24   206.124.146.179</programlisting></para>
 1668 
 1669           <para>When running Shorewall 5.0.14 or later, the equivalent
 1670           <filename>/etc/shorewall/snat</filename> would be:</para>
 1671 
 1672           <programlisting>#ACTION                SOURCE         DEST       ...
 1673 SNAT(206.124.146.179)  192.168.1.0/24 eth0</programlisting>
 1674 
 1675           <para>HTTP response packets corresponding to requests that fall
 1676           under that rule will have destination IP address 206.124.146.179 and
 1677           <emphasis role="bold">source</emphasis> port 80.</para>
 1678         </listitem>
 1679       </orderedlist>
 1680 
 1681       <para>Beginning with Shorewall 4.4.15, both IPv4 and IPv6 rules can be
 1682       defined in this file. See <ulink
 1683       url="manpages/shorewall-tcfilters.html">shorewall-tcfilters</ulink> (5)
 1684       for details.</para>
 1685 
 1686       <para>Columns in the file are as follow. As in all Shorewall
 1687       configuration files, a hyphen ("-") may be used to indicate that no
 1688       value is supplied in the column.</para>
 1689 
 1690       <variablelist>
 1691         <varlistentry>
 1692           <term>CLASS</term>
 1693 
 1694           <listitem>
 1695             <para>The interface name or number followed by a colon (":") and
 1696             the class number.</para>
 1697           </listitem>
 1698         </varlistentry>
 1699 
 1700         <varlistentry>
 1701           <term>SOURCE</term>
 1702 
 1703           <listitem>
 1704             <para>SOURCE IP address (host or network). DNS names are not
 1705             allowed.</para>
 1706           </listitem>
 1707         </varlistentry>
 1708 
 1709         <varlistentry>
 1710           <term>DEST</term>
 1711 
 1712           <listitem>
 1713             <para>DESTINATION IP address (host or network). DNS names are not
 1714             allowed.</para>
 1715           </listitem>
 1716         </varlistentry>
 1717 
 1718         <varlistentry>
 1719           <term>PROTO</term>
 1720 
 1721           <listitem>
 1722             <para>Protocol name or number.</para>
 1723           </listitem>
 1724         </varlistentry>
 1725 
 1726         <varlistentry>
 1727           <term>DPORT</term>
 1728 
 1729           <listitem>
 1730             <para>Comma-separated list of destination port names or numbers.
 1731             May only be specified if the protocol is TCP, UDP, SCTP or ICMP.
 1732             Port ranges are supported except for ICMP.</para>
 1733           </listitem>
 1734         </varlistentry>
 1735 
 1736         <varlistentry>
 1737           <term>SPORT</term>
 1738 
 1739           <listitem>
 1740             <para>Comma-separated list of source port names or numbers. May
 1741             only be specified if the protocol is TCP, UDP or SCTP. Port ranges
 1742             are supported.</para>
 1743           </listitem>
 1744         </varlistentry>
 1745 
 1746         <varlistentry>
 1747           <term>TOS</term>
 1748 
 1749           <listitem>
 1750             <para>Specifies the value of the TOS field. The value can be any
 1751             of the following:</para>
 1752 
 1753             <itemizedlist>
 1754               <listitem>
 1755                 <para><option>tos-minimize-delay</option></para>
 1756               </listitem>
 1757 
 1758               <listitem>
 1759                 <para><option>tos-maximuze-throughput</option></para>
 1760               </listitem>
 1761 
 1762               <listitem>
 1763                 <para><option>tos-maximize-reliability</option></para>
 1764               </listitem>
 1765 
 1766               <listitem>
 1767                 <para><option>tos-minimize-cost</option></para>
 1768               </listitem>
 1769 
 1770               <listitem>
 1771                 <para><option>tos-normal-service</option></para>
 1772               </listitem>
 1773 
 1774               <listitem>
 1775                 <para><replaceable>hex-number</replaceable></para>
 1776               </listitem>
 1777 
 1778               <listitem>
 1779                 <para><replaceable>hex-number</replaceable>/<replaceable>hex-number</replaceable></para>
 1780               </listitem>
 1781             </itemizedlist>
 1782 
 1783             <para>The <replaceable>hex-number</replaceable>s must be exactly
 1784             two digits (e.g., 0x04).</para>
 1785           </listitem>
 1786         </varlistentry>
 1787 
 1788         <varlistentry>
 1789           <term>LENGTH</term>
 1790 
 1791           <listitem>
 1792             <para>Must be a power of 2 between 32 and 8192 inclusive. Packets
 1793             with a total length that is strictly less than the specified value
 1794             will match the rule.</para>
 1795           </listitem>
 1796         </varlistentry>
 1797       </variablelist>
 1798 
 1799       <para>Example:</para>
 1800 
 1801       <para>I've used this configuration on my own firewall. The IFB portion
 1802       is more for test purposes rather than to serve any well-reasoned QOS
 1803       strategy.</para>
 1804 
 1805       <para><filename>/etc/shorewall/init</filename>:<programlisting>qt modprobe ifb numifbs=1
 1806 qt ip link set dev ifb0 up</programlisting></para>
 1807 
 1808       <para><filename>/etc/shorewall/interfaces</filename>:</para>
 1809 
 1810       <programlisting>#ZONE          INTERFACE         BROADCAST
 1811 -              ifb0</programlisting>
 1812 
 1813       <para><filename>/etc/shorewall/tcdevices</filename>:</para>
 1814 
 1815       <para><programlisting>
 1816 #INTERFACE      IN_BANDWITH     OUT_BANDWIDTH   OPTIONS         REDIRECT
 1817 1:eth0          -               384kbit         classify
 1818 2:ifb0          -               1300kbit        -               eth0</programlisting>
 1819       <filename>/etc/shorewall/tcclasses</filename>:<programlisting>#INTERFACE      MARK    RATE            CEIL            PRIORITY        OPTIONS
 1820 1:110           -       5*full/10       full            1               tcp-ack,tos-minimize-delay
 1821 1:120           -       2*full/10       6*full/10       2               default
 1822 1:130           -       2*full/10       6*full/10       3
 1823 2:110           -       5*full/10       full            1               tcp-ack,tos-minimize-delay
 1824 2:120           -       2*full/10       6*full/10       2               default
 1825 2:130           -       2*full/10       6*full/10       3</programlisting><filename>/etc/shorewall/tcfilters</filename>:<programlisting>#INTERFACE:     SOURCE          DEST            PROTO   DPORT   SPORT
 1826 #
 1827 #                                  OUTGOING TRAFFIC
 1828 #
 1829 1:130           206.124.146.178 -               tcp     -       49441,49442    #BITTORRENT on wookie
 1830 1:110           206.124.146.178                                                #wookie
 1831 1:110           206.124.146.179                                                #SNAT of internal systems
 1832 1:110           206.124.146.180                                                #Work Laptop
 1833 1:110           -               -               icmp    echo-request,echo-reply
 1834 1:110           -               -               icmp    echo-reply
 1835 1:130           206.124.146.177 -               tcp     -       873,25         #Bulk Traffic
 1836 #
 1837 #                                   INCOMING TRAFFIC
 1838 #
 1839 2:110           -               206.124.146.178                          #Wookie
 1840 2:110           -               206.124.146.179                          #SNAT Responses
 1841 2:110           -               206.124.146.180                          #Work Laptop
 1842 2:130           -               206.124.146.177 tcp     25               #Incoming Email.</programlisting></para>
 1843 
 1844       <para>You can examine the installed filters with the <command>shorewall
 1845       show filters</command> command. What follows shows the output for
 1846       <filename class="devicefile">eth0</filename> with the filters shown
 1847       above. <emphasis role="bold">Bold font</emphasis> are comments
 1848       explaining the rules.<programlisting>gateway:~ # shorewall-lite show filters
 1849 Shorewall Lite 4.1.6 Classifiers at gateway - Fri Mar 21 08:06:47 PDT 2008
 1850 
 1851 Device eth1:
 1852 
 1853 Device eth2:
 1854 
 1855 Device eth0:
 1856 filter parent 1: protocol ip pref 10 u32 
 1857 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 3:</emphasis> ht divisor 1 <emphasis
 1858             role="bold">  &lt;========= Start of table 3. parses TCP header</emphasis>
 1859 
 1860 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 3</emphasis>::800 order 2048 key ht 3 bkt 0 <emphasis
 1861             role="bold">flowid 1:130</emphasis>  (rule hit 102 success 0)
 1862   match 03690000/ffff0000 at nexthdr+0 (success 0 )        <emphasis
 1863             role="bold">   &lt;========= SOURCE PORT 873 goes to class 1:130</emphasis>
 1864 
 1865 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 2:</emphasis> ht divisor 1 <emphasis
 1866             role="bold">  &lt;========= Start of table 2. parses ICMP header</emphasis>
 1867 
 1868 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 2</emphasis>::800 order 2048 key ht 2 bkt 0 <emphasis
 1869             role="bold">flowid 1:110</emphasis>  (rule hit 0 success 0)
 1870   match 08000000/ff000000 at nexthdr+0 (success 0 )        <emphasis
 1871             role="bold">   &lt;========= ICMP Type 8 goes to class 1:110</emphasis>
 1872 
 1873 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 2</emphasis>::801 order 2049 key ht 2 bkt 0 <emphasis
 1874             role="bold">flowid 1:110</emphasis>  (rule hit 0 success 0)
 1875   match 00000000/ff000000 at nexthdr+0 (success 0 )         <emphasis
 1876             role="bold">  &lt;========= ICMP Type 0 goes to class 1:110</emphasis>
 1877 
 1878 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 1:</emphasis> ht divisor 1 <emphasis
 1879             role="bold">  &lt;========= Start of table 1. parses TCP header</emphasis>
 1880 
 1881 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 1:</emphasis>:800 order 2048 key ht 1 bkt 0 <emphasis
 1882             role="bold">flowid 1:130</emphasis>  (rule hit 0 success 0)
 1883   match c1210000/ffff0000 at nexthdr+0 (success 0 )         <emphasis
 1884             role="bold">  &lt;========= SOURCE PORT 49441 goes to class 1:130</emphasis>
 1885 
 1886 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 1</emphasis>::801 order 2049 key ht 1 bkt 0 <emphasis
 1887             role="bold">flowid 1:130</emphasis>  (rule hit 0 success 0)
 1888   match c1220000/ffff0000 at nexthdr+0 (success 0 )         <emphasis
 1889             role="bold">  &lt;========= SOURCE PORT 49442 goes to class 1:130</emphasis>
 1890 
 1891 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis> ht divisor 1 <emphasis
 1892             role="bold">&lt;========= Start of Table 800. Packets start here!</emphasis>
 1893 
 1894    <emphasis role="bold">=============== The following 2 rules are generated by the class definition in /etc/shorewall/classes ==================</emphasis>
 1895 
 1896 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:800 order 2048 key ht 800 bkt 0 <emphasis
 1897             role="bold">flowid 1:110</emphasis>  (rule hit 2204 success 138)
 1898   match 00060000/00ff0000 at 8 (success 396 )                 <emphasis
 1899             role="bold">&lt;========= TCP   </emphasis> 
 1900   match 05000000/0f00ffc0 at 0 (success 250 )                 <emphasis
 1901             role="bold">&lt;========= Header length 20 and Packet Length &lt; 64</emphasis> 
 1902   match 00100000/00ff0000 at 32 (success 138 )                <emphasis
 1903             role="bold">&lt;========= ACK</emphasis>
 1904 
 1905 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:801 order 2049 key ht 800 bkt 0 <emphasis
 1906             role="bold">flowid 1:110</emphasis>  (rule hit 2066 success 0)
 1907   match 00100000/00100000 at 0 (success 0 )                  <emphasis
 1908             role="bold">&lt;========= Minimize-delay</emphasis><emphasis
 1909             role="bold"> goes to class 1:110</emphasis>
 1910 
 1911    <emphasis role="bold">                     =============== Jump to Table 1 if the matches are met ==================</emphasis>
 1912  
 1913 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:802 order 2050 key ht 800 bkt 0 <emphasis
 1914             role="bold">link 1:</emphasis>  (rule hit 2066 success 0)
 1915   match ce7c92b2/ffffffff at 12 (success 1039 )              <emphasis
 1916             role="bold">&lt;========= SOURCE 206.124.146.178 </emphasis>         
 1917   match 00060000/00ff0000 at 8 (success 0 )                  <emphasis
 1918             role="bold">&lt;========= PROTO TCP</emphasis>
 1919     offset 0f00&gt;&gt;6 at 0  eat 
 1920 
 1921 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:803 order 2051 key ht 800 bkt 0 <emphasis
 1922             role="bold">flowid 1:110</emphasis>  (rule hit 2066 success 1039)
 1923   match ce7c92b2/ffffffff at 12 (success 1039 )               <emphasis
 1924             role="bold">&lt;========= SOURCE 206.124.146.178 goes to class 1:110</emphasis>
 1925 
 1926 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:804 order 2052 key ht 800 bkt 0 <emphasis
 1927             role="bold">flowid 1:110</emphasis>  (rule hit 1027 success 132)
 1928   match ce7c92b3/ffffffff at 12 (success 132 )               <emphasis
 1929             role="bold"> &lt;========= SOURCE 206.124.146.179 goes to class 1:110</emphasis>
 1930 
 1931 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:805 order 2053 key ht 800 bkt 0 <emphasis
 1932             role="bold">flowid 1:110</emphasis>  (rule hit 895 success 603)
 1933   match ce7c92b4/ffffffff at 12 (success 603 )                <emphasis
 1934             role="bold">&lt;========= SOURCE 206.124.146.180 goes to class 1:110</emphasis>
 1935 
 1936    <emphasis role="bold">                     =============== Jump to Table 2 if the matches are met ==================</emphasis>
 1937 
 1938 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:806 order 2054 key ht 800 bkt 0 <emphasis
 1939             role="bold">link 2:</emphasis>  (rule hit 292 success 0)
 1940   match 00010000/00ff0000 at 8 (success 0 )                   <emphasis
 1941             role="bold">&lt;========= PROTO ICMP</emphasis> 
 1942     offset 0f00&gt;&gt;6 at 0  eat
 1943 
 1944    <emphasis role="bold">                     =============== Jump to Table 3 if the matches are met ==================</emphasis>
 1945  
 1946 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:807 order 2055 key ht 800 bkt 0 <emphasis
 1947             role="bold">link 3:</emphasis>  (rule hit 292 success 0)
 1948   match ce7c92b1/ffffffff at 12 (success 265 )                <emphasis
 1949             role="bold">&lt;========= SOURCE 206.124.146.177</emphasis>
 1950   match 00060000/00ff0000 at 8 (success 102 )                 <emphasis
 1951             role="bold">&lt;========= PROTO TCP</emphasis>
 1952     offset 0f00&gt;&gt;6 at 0  eat </programlisting></para>
 1953     </section>
 1954 
 1955     <section>
 1956       <title>IFBs and SNAT/MASQUERADE</title>
 1957 
 1958       <para>IFB traffic shaping takes place immediately after the traffic is
 1959       received by the incoming interface and before it has been passed to any
 1960       Netfilter hook. This has two consequences:</para>
 1961 
 1962       <itemizedlist>
 1963         <listitem>
 1964           <para>There is no opportunity to mark the packets before they are
 1965           processed by the IFBs traffic shaping rules.</para>
 1966         </listitem>
 1967 
 1968         <listitem>
 1969           <para>The DEST IP address is still the IP address of the external
 1970           interface on which the traffic arrived.</para>
 1971         </listitem>
 1972       </itemizedlist>
 1973 
 1974       <para>As a result, in the tcdevices file description above, a <emphasis
 1975       role="bold">connmark</emphasis> option was added to that file in
 1976       Shorewall 5.2.7. The <emphasis role="bold">connmark</emphasis> option
 1977       allows firewall marks to be used to segregate traffic by DEST IP.</para>
 1978 
 1979       <para>Example (based closely on one supplied by Rodrigo Araujo, who also
 1980       wrote much of the code supporting the <emphasis
 1981       role="bold">connmark</emphasis> option):</para>
 1982 
 1983       <para><emphasis
 1984       role="bold">/etc/shorewall/shorewall.conf:</emphasis></para>
 1985 
 1986       <programlisting>...
 1987 TC_ENABLED=Internal
 1988 ...</programlisting>
 1989 
 1990       <para><emphasis role="bold">/etc/shorewall/interfaces:</emphasis></para>
 1991 
 1992       <programlisting>##############################################################################
 1993 ?FORMAT 2
 1994 ###############################################################################
 1995 #ZONE   INTERFACE   OPTIONS
 1996 net     NET_IF          dhcp,tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,physical=eth0
 1997 loc     LOC_IF          tcpflags,nosmurfs,routefilter,logmartians,physical=eth1</programlisting>
 1998 
 1999       <para><emphasis role="bold">/etc/shorewall/snat:</emphasis></para>
 2000 
 2001       <programlisting>?FORMAT 2
 2002 #ACTION       SOURCE      DEST    PROTO    PORT        IPSEC    MARK   
 2003 USER    SWITCH    ORIGDEST    PROBABILITY
 2004 MASQUERADE    -           NET_IF</programlisting>
 2005 
 2006       <para><emphasis role="bold">/etc/shorewall/tcdevices:</emphasis></para>
 2007 
 2008       <programlisting>#INTERFACE    IN_BANDWITH OUT_BANDWIDTH   OPTIONS     REDIRECT
 2009 ## net upload
 2010 10:NET_IF   -               1000mbit         htb
 2011 ## net download
 2012 11:ifb0         -               1000mbit         htb,connmark   NET_IF</programlisting>
 2013 
 2014       <para><emphasis role="bold">/etc/shorewall/tcclasses:</emphasis></para>
 2015 
 2016       <programlisting>#INTERFACE    MARK    RATE        CEIL        PRIO    OPTIONS
 2017 10:5000           111       500kbit            full       10     tcp-ack,tos-minimize-delay
 2018 11:5000           110       500kbit            full       10     tcp-ack,tos-minimize-delay
 2019 
 2020 10:1000           100       full-50500         full       20      default
 2021 11:1000           101       full-100500        full       20      default
 2022 
 2023 10:50             10        50mbit             50mbit     101     flow=nfct-src
 2024 11:100            11        100mbit            100mbit    101     flow=dst</programlisting>
 2025 
 2026       <para><emphasis role="bold">/etc/shorewall/tcfilters:</emphasis></para>
 2027 
 2028       <programlisting>#CLASS        SOURCE      DEST        PROTO   DPORT   SPORT   TOS LENGTH
 2029 ## limit LAN upload - works
 2030 10:50          10.100.100.0/24
 2031 ## limit LAN download - DOESN'T WORK BECAUSE OF MASQUERADE ON eth0 !!!! (snat file)
 2032 #11:100        -                  10.100.100.0/24</programlisting>
 2033 
 2034       <para><emphasis role="bold">/etc/shorewall/mangle:</emphasis></para>
 2035 
 2036       <programlisting>#ACTION       SOURCE      DEST        PROTO   DPORT   SPORT   USER    TEST    LENGTH  TOS CONNBYTES   HELPER  PROBABILITY DSCP    SWITCH
 2037 ## this only works with the aforementioned conntrack option
 2038 ## and LAN users' download traffic will get the 11:100 class (defined in tcclasses) applied
 2039 CONNMARK(11):F       10.100.100.0/24    - { TEST=0x0/0xff }</programlisting>
 2040     </section>
 2041   </section>
 2042 
 2043   <section id="show">
 2044     <title>Understanding the output of 'shorewall show tc'</title>
 2045 
 2046     <para>The <command>shorewall show tc</command> (<command>shorewall-lite
 2047     show tc</command>) command displays information about the current state of
 2048     traffic shaping. For each device, it executes the following
 2049     commands:</para>
 2050 
 2051     <programlisting> echo Device $device:
 2052  tc -s -d qdisc show dev $device
 2053  echo
 2054  tc -s -d class show dev $device
 2055  echo </programlisting>
 2056 
 2057     <para>So, the traffic-shaping output is generated entirely by the
 2058     <command>tc</command> utility.</para>
 2059 
 2060     <para>Here's the output of for eth0. The configuration is the one shown in
 2061     the preceding section (the output was obtained almost 24 hours later than
 2062     the <command>shorewall show filters</command> output shown above).</para>
 2063 
 2064     <programlisting>Device eth0:
 2065 
 2066 <emphasis role="bold">       ============== The primary queuing discipline is HTB (Hierarchical Token Bucket) ==================== </emphasis>
 2067 
 2068 qdisc htb 1: r2q 10 default 120 direct_packets_stat 9 ver 3.17
 2069  Sent 2133336743 bytes 4484781 pkt (dropped 198, overlimits 4911403 requeues 21) <emphasis
 2070         role="bold">&lt;=========== Note the overlimits and dropped counts</emphasis>
 2071  rate 0bit 0pps backlog 0b 8p requeues 21
 2072 
 2073 <emphasis role="bold">============== The ingress filter. If you specify IN-BANDWIDTH, you can see the 'dropped' count here. =========</emphasis>
 2074 
 2075                        <emphasis role="bold">In this case, the packets are being sent to the IFB for shaping</emphasis>
 2076  
 2077 qdisc ingress ffff: ---------------- 
 2078  Sent 4069015112 bytes 4997252 pkt (dropped 0, overlimits 0 requeues 0) 
 2079  rate 0bit 0pps backlog 0b 0p requeues 0
 2080 
 2081  <emphasis role="bold">============ Each of the leaf HTB classes has an SFQ qdisc to ensure that each flow gets its turn ============</emphasis>
 2082  
 2083 qdisc sfq 110: parent 1:110 limit 128p quantum 1514b flows 128/1024 perturb 10sec 
 2084  Sent 613372519 bytes 2870225 pkt (dropped 0, overlimits 0 requeues 6) 
 2085  rate 0bit 0pps backlog 0b 0p requeues 6 
 2086 qdisc sfq 120: parent 1:120 limit 128p quantum 1514b flows 128/1024 perturb 10sec 
 2087  Sent 18434920 bytes 60961 pkt (dropped 0, overlimits 0 requeues 0) 
 2088  rate 0bit 0pps backlog 0b 0p requeues 0 
 2089 qdisc sfq 130: parent 1:130 limit 128p quantum 1514b flows 128/1024 perturb 10sec 
 2090  Sent 1501528722 bytes 1553586 pkt (dropped 198, overlimits 0 requeues 15) 
 2091  rate 0bit 0pps backlog 11706b 8p requeues 15 
 2092 
 2093                            <emphasis role="bold">============= Class 1:110 -- the high-priority class ===========
 2094 
 2095 
 2096                                    Note the rate and ceiling calculated from 'full'</emphasis>
 2097 
 2098 class htb <emphasis role="bold">1:110</emphasis> parent 1:1 leaf 110: prio 1 quantum 4800 <emphasis
 2099         role="bold">rate 192000bit ceil 384000bit</emphasis> burst 1695b/8 mpu 0b overhead 0b cburst 1791b/8 mpu 0b overhead 0b level 0 
 2100  Sent 613372519 bytes 2870225 pkt (dropped 0, overlimits 0 requeues 0) 
 2101  <emphasis role="bold">rate 195672bit 28pps backlog 0b 0p</emphasis> requeues 0 <emphasis
 2102         role="bold">&lt;=========== Note the current rate of traffic. There is no queuing (no packet backlog)</emphasis> 
 2103  lended: 2758458 borrowed: 111773 giants:
 2104  tokens: 46263 ctokens: 35157
 2105 
 2106                                       <emphasis role="bold">============== The root class ============</emphasis>
 2107 
 2108 class htb <emphasis role="bold">1:1 root</emphasis> rate 384000bit ceil 384000bit burst 1791b/8 mpu 0b overhead 0b cburst 1791b/8 mpu 0b overhead 0b level 7 
 2109  <emphasis role="bold">Sent 2133276316 bytes 4484785 pkt</emphasis> (dropped 0, overlimits 0 requeues 0) <emphasis
 2110         role="bold">&lt;==== Total output traffic since last 'restart'</emphasis>
 2111  rate 363240bit 45pps backlog 0b 0p requeues 0 
 2112  lended: 1081936 borrowed: 0 giants: 0
 2113  tokens: -52226 ctokens: -52226
 2114 
 2115                       <emphasis role="bold">============= Bulk Class (outgoing rsync, email and bittorrent) ============</emphasis>
 2116 
 2117 class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1900 rate 76000bit ceil 230000bit burst 1637b/8 mpu 0b overhead 0b cburst 1714b/8 mpu 0b overhead 0b level 0 
 2118  Sent 1501528722 bytes 1553586 pkt (dropped 198, overlimits 0 requeues 0) 
 2119  <emphasis role="bold">rate 162528bit 14pps backlog 0b 8p</emphasis> requeues 0 <emphasis
 2120         role="bold">&lt;======== Queuing is occurring (8 packet backlog). The rate is still below the ceiling.</emphasis>
 2121  lended: 587134 borrowed: 966459 giants: 0               <emphasis role="bold">During peak activity, the rate tops out at around 231000 (just above ceiling).</emphasis>
 2122  tokens: -30919 ctokens: -97657
 2123 
 2124                        <emphasis role="bold">================= Default class (mostly serving web pages) ===============</emphasis>
 2125 
 2126 class htb 1:120 parent 1:1 leaf 120: prio 2 quantum 1900 rate 76000bit ceil 230000bit burst 1637b/8 mpu 0b overhead 0b cburst 1714b/8 mpu 0b overhead 0b level 0 
 2127  Sent 18434920 bytes 60961 pkt (dropped 0, overlimits 0 requeues 0) 
 2128  rate 2240bit 2pps backlog 0b 0p requeues 0 
 2129  lended: 57257 borrowed: 3704 giants: 0
 2130  tokens: 156045 ctokens: 54178
 2131 
 2132 </programlisting>
 2133   </section>
 2134 
 2135   <section id="External">
 2136     <title id="tcstart">Using your own tc script</title>
 2137 
 2138     <section id="owntcstart">
 2139       <title>Replacing builtin tcstart file</title>
 2140 
 2141       <para>If you prefer your own tcstart file, just install it in
 2142       /etc/shorewall/tcstart.</para>
 2143 
 2144       <para>In your tcstart script, when you want to run the <quote>tc</quote>
 2145       utility, use the run_tc function supplied by Shorewall if you want tc
 2146       errors to stop the firewall.</para>
 2147 
 2148       <orderedlist>
 2149         <listitem>
 2150           <para>Set TC_ENABLED=Yes and CLEAR_TC=Yes</para>
 2151         </listitem>
 2152 
 2153         <listitem>
 2154           <para>Supply an /etc/shorewall/tcstart script to configure your
 2155           traffic shaping rules.</para>
 2156         </listitem>
 2157 
 2158         <listitem>
 2159           <para>Optionally supply an /etc/shorewall/tcclear script to stop
 2160           traffic shaping. That is usually unnecessary.</para>
 2161         </listitem>
 2162 
 2163         <listitem>
 2164           <para>If your tcstart script uses the <quote>fwmark</quote>
 2165           classifier, you can mark packets using entries in
 2166           /etc/shorewall/mangle or /etc/shorewall/tcrules.</para>
 2167         </listitem>
 2168       </orderedlist>
 2169     </section>
 2170 
 2171     <section id="Start">
 2172       <title>Traffic control outside Shorewall</title>
 2173 
 2174       <para>To start traffic shaping when you bring up your network
 2175       interfaces, you will have to arrange for your traffic shaping
 2176       configuration script to be run at that time. How you do that is
 2177       distribution dependent and will not be covered here. You then
 2178       should:</para>
 2179 
 2180       <orderedlist>
 2181         <listitem>
 2182           <para>Set TC_ENABLED=No and CLEAR_TC=No</para>
 2183         </listitem>
 2184 
 2185         <listitem>
 2186           <para>If your script uses the <quote>fwmark</quote> classifier, you
 2187           can mark packets using entries in /etc/shorewall/mangle or
 2188           /etc/shorewall/tcrules.</para>
 2189         </listitem>
 2190       </orderedlist>
 2191     </section>
 2192   </section>
 2193 
 2194   <section id="Testing">
 2195     <title>Testing Tools</title>
 2196 
 2197     <para>At least one Shorewall user has found this tool helpful: <ulink
 2198     url="http://e2epi.internet2.edu/network-performance-toolkit.html">http://e2epi.internet2.edu/network-performance-toolkit.html</ulink></para>
 2199   </section>
 2200 </article>