"Fossies" - the Fresh Open Source Software Archive 
As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) XML source code syntax highlighting (style:
standard) with prefixed line numbers.
Alternatively you can here
view or
download the uninterpreted source code file.
1 <?xml version="1.0" encoding="UTF-8"?>
2 <!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
3 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
4 <article>
5 <!--$Id$-->
6
7 <articleinfo>
8 <title>Complex Traffic Shaping/Control</title>
9
10 <authorgroup>
11 <author>
12 <firstname>Tom</firstname>
13
14 <surname>Eastep</surname>
15 </author>
16
17 <author>
18 <firstname>Arne</firstname>
19
20 <surname>Bernin</surname>
21 </author>
22 </authorgroup>
23
24 <pubdate><?dbtimestamp format="Y/m/d"?></pubdate>
25
26 <copyright>
27 <year>2001-2013</year>
28
29 <year>2020</year>
30
31 <holder>Thomas M. Eastep</holder>
32 </copyright>
33
34 <copyright>
35 <year>2005</year>
36
37 <holder>Arne Bernin & Thomas M. Eastep</holder>
38 </copyright>
39
40 <legalnotice>
41 <para>Permission is granted to copy, distribute and/or modify this
42 document under the terms of the GNU Free Documentation License, Version
43 1.2 or any later version published by the Free Software Foundation; with
44 no Invariant Sections, with no Front-Cover, and with no Back-Cover
45 Texts. A copy of the license is included in the section entitled
46 <quote><ulink url="GnuCopyright.htm">GNU Free Documentation
47 License</ulink></quote>.</para>
48 </legalnotice>
49 </articleinfo>
50
51 <important>
52 <para>Traffic shaping is complex and the Shorewall community is not well
53 equipped to answer traffic shaping questions. So if you are the type of
54 person who needs "insert tab A into slot B" instructions for everything
55 that you do, then please don't try to implement traffic shaping using
56 Shorewall. You will just frustrate yourself and we won't be able to help
57 you.</para>
58 </important>
59
60 <warning>
61 <para>Said another way, reading just Shorewall documentation is not going
62 to give you enough background to use this material.</para>
63
64 <para>At a minimum, you will need to refer to at least the following
65 additional information:</para>
66
67 <itemizedlist>
68 <listitem>
69 <para><emphasis>The LARTC HOWTO</emphasis>: <ulink
70 url="http://www.lartc.org">http://www.lartc.org</ulink></para>
71 </listitem>
72
73 <listitem>
74 <para>T<emphasis>he HTB User's Guide</emphasis>: <ulink
75 url="http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm">http://luxik.cdi.cz/~devik/qos/htb/manual/userg.htm</ulink></para>
76 </listitem>
77
78 <listitem>
79 <para><emphasis>HFSC Scheduling with Linux</emphasis>: <ulink
80 url="http://linux-ip.net/articles/hfsc.en/">http://linux-ip.net/articles/hfsc.en/</ulink></para>
81 </listitem>
82
83 <listitem>
84 <para>Some of the documents listed at <ulink
85 url="http://www.netfilter.org/documentation/index.html#documentation-howto">http://www.netfilter.org/documentation/index.html#documentation-howto</ulink>.
86 The tutorial by Oskar Andreasson is particularly good.</para>
87 </listitem>
88
89 <listitem>
90 <para>The output of <command>man iptables</command></para>
91 </listitem>
92 </itemizedlist>
93 </warning>
94
95 <section id="Intro">
96 <title>Introduction</title>
97
98 <para>Beginning with Shorewall 4.4.6, Shorewall includes two separate
99 implementations of traffic shaping. This document describes the original
100 implementation which is complex and difficult to configure. A much simpler
101 version is described in <ulink role="bold"
102 url="simple_traffic_shaping.html">Simple Traffic Shaping/Control</ulink>
103 and is highly recommended unless you really need to delay certain traffic
104 passing through your firewall.</para>
105
106 <para>Shorewall has builtin support for traffic shaping and control. This
107 support does not cover all options available (and especially all
108 algorithms that can be used to queue traffic) in the Linux kernel but it
109 should fit most needs. If you are using your own script for traffic
110 control and you still want to use it in the future, you will find
111 information on how to do this, <link linkend="owntcstart">later in this
112 document</link>. But for this to work, you will also need to enable
113 traffic shaping in the kernel and Shorewall as covered by the next
114 sections.</para>
115 </section>
116
117 <section id="LinuxTC">
118 <title>Linux traffic shaping and control</title>
119
120 <para>This section gives a brief introduction of how controlling traffic
121 with the Linux kernel works. Although this might be enough for configuring
122 it in the Shorewall configuration files, we strongly recommend that you
123 take a deeper look into the <ulink url="http://lartc.org/howto/">Linux
124 Advanced Routing and Shaping HOWTO</ulink>. At the time of writing this,
125 the current version is 1.0.0.</para>
126
127 <para>Since kernel 2.2, Linux has extensive support for controlling
128 traffic. You can define different algorithms that are used to queue the
129 traffic before it leaves an interface. The standard one is called pfifo
130 and is (as the name suggests) of the type First In First out. This means,
131 that it does not shape anything, if you have a connection that eats up all
132 your bandwidth, this queuing algorithm will not stop it from doing
133 so.</para>
134
135 <para>For Shorewall traffic shaping we use three algorithms: HTB
136 (Hierarchical Token Bucket), HFSC (Hierarchical Fair Service Curves) and
137 SFQ (Stochastic Fairness Queuing). SFQ is easy to explain: it just tries
138 to track your connections (tcp or udp streams) and balances the traffic
139 between them. This normally works well. HTB and HFSC allow you to define a
140 set of classes, and you can put the traffic you want into these classes.
141 You can define minimum and maximum bandwidth settings for those classes
142 and order them hierarchically (the less prioritized classes only get
143 bandwidth if the more important have what they need). Additionally, HFSC
144 allows you to specify the maximum queuing delay that a packet may
145 experience. Shorewall builtin traffic shaping allows you to define these
146 classes (and their bandwidth limits), and it uses SFQ inside these classes
147 to make sure, that different data streams are handled equally. If SFQ's
148 default notion of a 'stream' doesn't work well for you, you can change it
149 using the <emphasis role="bold">flow</emphasis> option described <link
150 linkend="tcclasses">below</link>.</para>
151
152 <para>You can shape incoming traffic through use of an
153 <firstterm>Intermediate Functional Block</firstterm> (IFB) device. <link
154 linkend="IFB">See below</link>. <emphasis role="bold">But beware: using an
155 IFB can result in queues building up both at your ISPs router and at your
156 own.</emphasis></para>
157
158 <para>You shape and control outgoing traffic by assigning the traffic to
159 <firstterm>classes</firstterm>. Each class is associated with exactly one
160 network interface and has a number of attributes:</para>
161
162 <orderedlist>
163 <listitem>
164 <para>PRIORITY - Used to give preference to one class over another
165 when selecting a packet to send. The priority is a numeric value with
166 1 being the highest priority, 2 being the next highest, and so
167 on.</para>
168 </listitem>
169
170 <listitem>
171 <para>RATE - The minimum bandwidth this class should get, when the
172 traffic load rises. Classes with a higher priority (lower PRIORITY
173 value) are served even if there are others that have a guaranteed
174 bandwidth but have a lower priority (higher PRIORITY value).</para>
175 </listitem>
176
177 <listitem>
178 <para>CEIL - The maximum bandwidth the class is allowed to use when
179 the link is idle.</para>
180 </listitem>
181
182 <listitem>
183 <para>MARK - Netfilter has a facility for
184 <firstterm>marking</firstterm> packets. Packet marks have a numeric
185 value which is limited in Shorewall to the values 1-255 (1-16383 if
186 you set WIDE_TC_MARKS=Yes in <ulink
187 url="manpages/shorewall.conf.html">shorewall.conf</ulink> (5) ). You
188 assign packet marks to different types of traffic using entries in the
189 <filename>/etc/shorewall/mangle</filename> file (Shorewall 4.6.0 or
190 later) or <filename>/etc/shorewall/tcrules</filename> (Prior to
191 Shorewall 4.6.0).</para>
192
193 <note>
194 <para>In Shorewall 4.4.26, WIDE_TC_MARKS was superseded by TC_BITS
195 which specifies the width in bits of the traffic shaping mark field.
196 The default is based on the setting of WIDE_TC_MARKS so as to
197 provide upward compatibility. See the <ulink
198 url="PacketMarking.html#Values">Packet Marking using
199 /etc/shorewall/mangle</ulink> article.</para>
200 </note>
201 </listitem>
202 </orderedlist>
203
204 <para>One class for each interface must be designated as the
205 <firstterm>default class</firstterm>. This is the class to which unmarked
206 traffic (packets to which you have not assigned a mark value in
207 <filename>/etc/shorewall/mangle</filename>) is assigned.</para>
208
209 <para>Netfilter also supports a mark value on each connection. You can
210 assign connection mark values in
211 <filename>/etc/shorewall/mangle</filename>
212 (<filename>/etc/shorewall/tcrules</filename>), you can copy the current
213 packet's mark to the connection mark (SAVE), or you can copy the
214 connection mark value to the current packet's mark (RESTORE). For more
215 information, see<ulink url="PacketMarking.html"> this
216 article</ulink>.</para>
217 </section>
218
219 <section id="Shorewall">
220 <title>Enable TC support in Shorewall</title>
221
222 <para>You need this support whether you use the builtin support or whether
223 you provide your own tcstart script.</para>
224
225 <para>To enable the builtin traffic shaping and control in Shorewall, you
226 have to do the following:</para>
227
228 <itemizedlist>
229 <listitem>
230 <para>Set <emphasis role="bold">TC_ENABLED</emphasis> to "<emphasis
231 role="bold">Internal</emphasis>" in /etc/shorewall/shorewall.conf.
232 Setting <emphasis role="bold">TC_ENABLED=Yes</emphasis> causes
233 Shorewall to look for an external tcstart file (See <link
234 linkend="tcstart">a later section</link> for details).</para>
235 </listitem>
236
237 <listitem>
238 <para>Setting <emphasis role="bold">CLEAR_TC</emphasis> parameter in
239 /etc/shorewall/shorewall.conf to <emphasis role="bold">Yes</emphasis>
240 will clear the traffic shaping configuration during Shorewall
241 [re]start and Shorewall stop. This is normally what you want when
242 using the builtin support (and also if you use your own tcstart
243 script)</para>
244 </listitem>
245
246 <listitem>
247 <para>The other steps that follow depend on whether you use your own
248 script or the builtin solution. They will be explained in the
249 following sections.</para>
250 </listitem>
251 </itemizedlist>
252 </section>
253
254 <section id="Builtin">
255 <title>Using builtin traffic shaping/control</title>
256
257 <para>Shorewall's builtin traffic shaping feature provides a thin layer on
258 top of the ingress qdesc, HTB and SFQ. That translation layer allows you
259 to:</para>
260
261 <itemizedlist>
262 <listitem>
263 <para>Define HTB and/or HFSC classes using Shorewall-style
264 column-oriented configuration files.</para>
265 </listitem>
266
267 <listitem>
268 <para>Integrate the reloading of your traffic shaping configuration
269 with the reloading of your packet-filtering and marking
270 configuration.</para>
271 </listitem>
272
273 <listitem>
274 <para>Assign traffic to HTB or HFSC classes by TOS value.</para>
275 </listitem>
276
277 <listitem>
278 <para>Assign outgoing TCP ACK packets to an HTB or HFSC class.</para>
279 </listitem>
280
281 <listitem>
282 <para>Assign traffic to HTB and/or HFSC classes based on packet mark
283 value or based on packet contents.</para>
284 </listitem>
285
286 <listitem>
287 <para>Throttle incoming traffic</para>
288 </listitem>
289
290 <listitem>
291 <para>Use an <emphasis>Intermediate functional block </emphasis>(IFB)
292 to shape incoming traffic<emphasis> </emphasis></para>
293 </listitem>
294 </itemizedlist>
295
296 <para>Those few features are really all that builtin traffic
297 shaping/control provides; consequently, you need to understand HTB and/or
298 HFSC and Linux traffic shaping as well as Netfilter packet marking in
299 order to use the facility. Again, please see the links at top of this
300 article.</para>
301
302 <para>For defining bandwidths (for either devices or classes) please use
303 kbit or kbps (for Kilobytes per second) and make sure there is <emphasis
304 role="bold">NO</emphasis> space between the number and the unit (it is
305 100kbit <emphasis role="bold">not</emphasis> 100 kbit). Using mbit, mbps
306 or a raw number (which means bytes) could be used, but note that only
307 integer numbers are supported (0.5 is <emphasis role="bold">not
308 valid</emphasis>).</para>
309
310 <para><emphasis role="bold">To properly configure the settings for your
311 devices you need to find out the real up- and downstream rates you
312 have</emphasis>. This is especially the case, if you are using a DSL
313 connection or one of another type that do not have a guaranteed bandwidth.
314 Don't trust the values your provider tells you for this; especially
315 measuring the real download speed is important! There are several online
316 tools that help you find out; search for "dsl speed test" on google (For
317 Germany you can use <ulink
318 url="http://www.speedcheck.arcor.de/cgi-bin/speedcheck.cgi">arcor speed
319 check</ulink>). Be sure to choose a test site located near you.</para>
320
321 <section id="tcdevices">
322 <title>/etc/shorewall/tcdevices</title>
323
324 <para>This file allows you to define the incoming and outgoing bandwidth
325 for the devices you want traffic shaping to be enabled. That means, if
326 you want to use traffic shaping for a device, you have to define it
327 here. For additional information, see <ulink
328 url="manpages/shorewall-tcdevices.html">shorewall-tcdevices</ulink>
329 (5).</para>
330
331 <para>Columns in the file are as follows:</para>
332
333 <itemizedlist>
334 <listitem>
335 <para>INTERFACE - Name of interface. Each interface may be listed
336 only once in this file. You may NOT specify the name of an alias
337 (e.g., eth0:0) here; see <ulink url="FAQ.htm#faq18">FAQ #18</ulink>.
338 You man NOT specify wildcards here, e.g. if you have multiple ppp
339 interfaces, you need to put them all in here! Shorewall will
340 determine if the device exists and will only configure the device if
341 it does exist. If it doesn't exist or it is DOWN, the following
342 warning is issued:</para>
343
344 <para><emphasis role="bold">WARNING: Device <device name> is
345 not in the UP state -- traffic-shaping configuration
346 skipped</emphasis></para>
347
348 <para>Shorewall assigns a sequential <firstterm>interface
349 number</firstterm> to each interface (the first entry in
350 <filename>/etc/shorewall/tcdevices</filename> is interface 1, the
351 second is interface 2 and so on) You can also explicitly specify the
352 interface number by prefixing the interface name with the number and
353 a colon (":"). Example: 1:eth0.</para>
354
355 <warning>
356 <para>Device numbers are expressed in hexidecimal. So the device
357 following 9 is A, not 10.</para>
358 </warning>
359 </listitem>
360
361 <listitem>
362 <para>IN-BANDWIDTH - The incoming Bandwidth of that interface.
363 Please note that when you use this column, you are not traffic
364 shaping incoming traffic, as the traffic is already received before
365 you could do so. This Column allows you to define the maximum
366 traffic allowed for this interface in total, if the rate is
367 exceeded, the excess packets are dropped. You want this mainly if
368 you have a DSL or Cable Connection to avoid queuing at your
369 providers side. If you don't want any traffic to be dropped set this
370 to a value faster than your interface maximum rate, or to 0
371 (zero).</para>
372
373 <para>To determine the optimum value for this setting, we recommend
374 that you start by setting it significantly below your measured
375 download bandwidth (20% or so). While downloading, measure the
376 <emphasis>ping</emphasis> response time from the firewall to the
377 upstream router as you gradually increase the setting.The optimal
378 setting is at the point beyond which the <emphasis>ping</emphasis>
379 time increases sharply as you increase the setting.</para>
380
381 <note>
382 <para>For fast lines, the actually download speed may be well
383 below what you specify here. If you have this problem, then follow
384 the bandwidth with a ":" and a <firstterm>burst</firstterm> size.
385 The default burst is 10kb, but on my 50mbit line, I specify 200kb.
386 (50mbit:200kb).</para>
387 </note>
388
389 <caution>
390 <para>Incoming IPSec traffic traverses traffic shaping twice -
391 firs as encrypted and encapsulated ESP packets and then en clair.
392 As a result, incoming bandwidth can be significantly less than
393 specified if IPSEC packets form a significant part of inoming
394 traffic.</para>
395 </caution>
396 </listitem>
397
398 <listitem>
399 <para>OUT-BANDWIDTH - Specify the outgoing bandwidth of that
400 interface. This is the maximum speed your connection can handle. It
401 is also the speed you can refer as "full" if you define the tc
402 classes. Outgoing traffic above this rate will be dropped.</para>
403 </listitem>
404
405 <listitem>
406 <para>OPTIONS — A comma-separated list of options from the following
407 list:</para>
408
409 <variablelist>
410 <varlistentry>
411 <term><emphasis role="bold">classify</emphasis></term>
412
413 <listitem>
414 <para>If specified, classification of traffic into the various
415 classes is done by CLASSIFY entries in
416 <filename>/etc/shorewall/mangle</filename>
417 (<filename>/etc/shorewall/tcrules</filename>) or by entries in
418 <filename>/etc/shorewall/tcfilters</filename>. No MARK value
419 will be associated with classes on this interface.</para>
420 </listitem>
421 </varlistentry>
422
423 <varlistentry>
424 <term><emphasis role="bold">hfsc</emphasis></term>
425
426 <listitem>
427 <para>Shorewall normally uses the <firstterm>Hierarchical
428 Token Bucket</firstterm> (HTB) queuing discipline. When
429 <option>hfsc</option> is specified, the
430 <firstterm>Hierarchical Fair Service Curves</firstterm> (HFSC)
431 discipline is used instead.</para>
432 </listitem>
433 </varlistentry>
434
435 <varlistentry>
436 <term><emphasis role="bold">linklayer</emphasis></term>
437
438 <listitem>
439 <para>Added in Shorewall 4.5.6. Type of link (ethernet, atm,
440 adsl). When specified, causes scheduler packet size
441 manipulation as described in tc-stab (8). When this option is
442 given, the following options may also be given after
443 it:</para>
444
445 <variablelist>
446 <varlistentry>
447 <term><emphasis
448 role="bold">mtu</emphasis>=<replaceable>mtu</replaceable></term>
449
450 <listitem>
451 <para>The device MTU; default 2048 (will be rounded up
452 to a power of two)</para>
453 </listitem>
454 </varlistentry>
455
456 <varlistentry>
457 <term><emphasis
458 role="bold">mpu</emphasis>=<replaceable>mpubytes</replaceable></term>
459
460 <listitem>
461 <para>Minimum packet size used in calculations. Smaller
462 packets will be rounded up to this size</para>
463 </listitem>
464 </varlistentry>
465
466 <varlistentry>
467 <term><emphasis
468 role="bold">tsize</emphasis>=<replaceable>tablesize</replaceable></term>
469
470 <listitem>
471 <para>Size table entries; default is 512</para>
472 </listitem>
473 </varlistentry>
474
475 <varlistentry>
476 <term><emphasis
477 role="bold">overhead</emphasis>=<replaceable>overheadbytes</replaceable></term>
478
479 <listitem>
480 <para>Number of overhead bytes per packet</para>
481 </listitem>
482 </varlistentry>
483 </variablelist>
484 </listitem>
485 </varlistentry>
486
487 <varlistentry>
488 <term><emphasis role="bold">connmark</emphasis></term>
489
490 <listitem>
491 <para>Added in Shorewall 5.2.7. May be specified on IFB
492 devices to enable use of firewall marks to select the
493 appropriate traffic shaping class.</para>
494 </listitem>
495 </varlistentry>
496 </variablelist>
497 </listitem>
498
499 <listitem>
500 <para>REDIRECTED INTERFACES — Entries are appropriate in this column
501 only if the device in the INTERFACE column names a <link
502 linkend="IFB">Intermediate Functional Block (IFB)</link>. It lists
503 the physical interfaces that will have their input shaped using
504 classes defined on the IFB. Neither the IFB nor any of the
505 interfaces listed in this column may have an IN-BANDWIDTH specified.
506 You may specify zero (0) or a dash ("-:) in the IN-BANDWIDTH
507 column.</para>
508
509 <para>IFB devices automatically get the <emphasis
510 role="bold">classify</emphasis> option unless the <emphasis
511 role="bold">connmark</emphasis> option is specified.</para>
512 </listitem>
513 </itemizedlist>
514
515 <example id="Example0">
516 <title/>
517
518 <para>Suppose you are using PPP over Ethernet (DSL) and ppp0 is the
519 interface for this. The device has an outgoing bandwidth of 500kbit
520 and an incoming bandwidth of 6000kbit</para>
521
522 <programlisting>#INTERFACE IN-BANDWITH OUT-BANDWIDTH
523 ppp0 6000kbit 500kbit</programlisting>
524 </example>
525 </section>
526
527 <section id="tcclasses">
528 <title>/etc/shorewall/tcclasses</title>
529
530 <para>This file allows you to define the actual classes that are used to
531 split the outgoing traffic. For additional information, see <ulink
532 url="manpages/shorewall-tcclasses.html">shorewall-tcclasses</ulink>
533 (5).</para>
534
535 <itemizedlist>
536 <listitem>
537 <para>INTERFACE - Name of interface. Users may also specify the
538 interface number. Must match the name (or number) of an interface
539 with an entry in <filename>/etc/shorewall/tcdevices</filename>. If
540 the interface has the <emphasis role="bold">classify</emphasis>
541 option in <filename>/etc/shorewall/tcdevices</filename>, then the
542 interface name or number must be followed by a colon and a
543 <firstterm>class number</firstterm>. Examples: eth0:1, 4:9. Class
544 numbers must be unique for a given interface. Normally, all classes
545 defined here are sub-classes of a root class that is implicitly
546 defined from the entry in <ulink
547 url="shorewall-tcdevices.html">shorewall-tcdevices</ulink>(5). You
548 can establish a class hierarchy by specifying a
549 <emphasis>parent</emphasis> class (e.g.,
550 <emphasis>interface</emphasis>:<emphasis>parent-class</emphasis>:<emphasis>class</emphasis>)
551 -- the number of a class that you have previously defined. The
552 sub-class may borrow unused bandwidth from its parent.</para>
553
554 <warning>
555 <para>Class numbers are expressed in hexidecimal. So the class
556 following class 9 is A, not 10.</para>
557 </warning>
558 </listitem>
559
560 <listitem>
561 <para>MARK - The mark value which is an integer in the range 1-255
562 (1-16383 if you set WIDE_TC_MARKS=Yes or set TC_BITS=14 in <ulink
563 url="manpages/shorewall.conf.html">shorewall.conf</ulink> (5) ). You
564 define these marks in the mangle or tcrules file, marking the
565 traffic you want to go into the queuing classes defined in here. You
566 can use the same marks for different Interfaces. You must specify
567 "-' in this column if the device specified in the INTERFACE column
568 has the <emphasis role="bold">classify</emphasis> option in
569 <filename>/etc/shorewall/tcdevices</filename>.</para>
570
571 <note>
572 <para>In Shorewall 4.5.0, WIDE_TC_MARKS was superseded by TC_BITS
573 which specifies the width in bits of the traffic shaping mark
574 field. The default is based on the setting of WIDE_TC_MARKS so as
575 to provide upward compatibility.</para>
576 </note>
577 </listitem>
578
579 <listitem>
580 <para>RATE - The minimum bandwidth this class should get, when the
581 traffic load rises. Please note that first the classes which equal
582 or a lesser priority value are served even if there are others that
583 have a guaranteed bandwidth but a lower priority. <emphasis
584 role="bold">If the sum of the RATEs for all classes assigned to an
585 INTERFACE exceed that interfaces's OUT-BANDWIDTH, then the
586 OUT-BANDWIDTH limit will not be honored.</emphasis></para>
587
588 <para>When using HFSC, this column may contain 1, 2 or 3 pieces of
589 information separated by colons (":"). In addition to the minimum
590 bandwidth, leaf classes may specify realtime criteria: DMAX (maximum
591 delay in milliseconds) and optionally UMAX (the largest packet
592 expected in the class). See <link linkend="HFSC">below</link> for
593 details.</para>
594 </listitem>
595
596 <listitem>
597 <para>CEIL - The maximum bandwidth this class is allowed to use when
598 the link is idle. Useful if you have traffic which can get full
599 speed when more important services (e.g. interactive like ssh) are
600 not used. You can use the value "full" in here for setting the
601 maximum bandwidth to the defined output bandwidth of that
602 interface.</para>
603 </listitem>
604
605 <listitem>
606 <para>PRIORITY - you have to define a priority for the class.
607 packets in a class with a higher priority (=lesser value) are
608 handled before less prioritized ones. You can just define the mark
609 value here also, if you are increasing the mark values with lesser
610 priority.</para>
611 </listitem>
612
613 <listitem>
614 <para>OPTIONS - A comma-separated list of options including the
615 following:</para>
616
617 <itemizedlist>
618 <listitem>
619 <para>default - this is the default class for that interface
620 where all traffic should go, that is not classified
621 otherwise.</para>
622
623 <note>
624 <para>defining default for exactly <emphasis
625 role="bold">one</emphasis> class per interface is
626 mandatory!</para>
627 </note>
628 </listitem>
629
630 <listitem>
631 <para>tos-<tosname> - this lets you define a filter for
632 the given <tosname> which lets you define a value of the
633 Type Of Service bits in the ip package which causes the package
634 to go in this class. Please note, that this filter overrides all
635 mark settings, so if you define a tos filter for a class all
636 traffic having that mark will go in it regardless of the mark on
637 the package. You can use the following for this option:
638 tos-minimize-delay (16) tos-maximize-throughput (8)
639 tos-maximize-reliability (4) tos-minimize-cost (2)
640 tos-normal-service (0)</para>
641
642 <note>
643 <para>Each of this options is only valid for <emphasis
644 role="bold">one</emphasis> class per interface.</para>
645 </note>
646 </listitem>
647
648 <listitem>
649 <para>tcp-ack - if defined causes an tc filter to be created
650 that puts all tcp ack packets on that interface that have an
651 size of <=64 Bytes to go in this class. This is useful for
652 speeding up downloads. Please note that the size of the ack
653 packets is limited to 64 bytes as some applications (p2p for
654 example) use to make every package an ack package which would
655 cause them all into here. We want only packets WITHOUT payload
656 to match, so the size limit. Bigger packets just take their
657 normal way into the classes.</para>
658
659 <note>
660 <para>This option is only valid for <emphasis
661 role="bold">class</emphasis> per interface.</para>
662 </note>
663 </listitem>
664
665 <listitem>
666 <para>occurs=<emphasis>number</emphasis> - Typically used with
667 an IPMARK entry in mangle or tcrules. Causes the rule to be
668 replicated for a total of <emphasis>number</emphasis> rules.
669 Each rule has a successively class number and mark value.</para>
670
671 <para>When 'occurs' is used:</para>
672
673 <itemizedlist>
674 <listitem>
675 <para>The associated device may not have the 'classify'
676 option.</para>
677 </listitem>
678
679 <listitem>
680 <para>The class may not be the default class.</para>
681 </listitem>
682
683 <listitem>
684 <para>The class may not have any 'tos=' options (including
685 'tcp-ack').</para>
686 </listitem>
687
688 <listitem>
689 <para>The class should not specify a MARK value. If one is
690 specified, it will be ignored with a warning message.</para>
691 </listitem>
692 </itemizedlist>
693
694 <para>The 'RATE' and 'CEIL' parameters apply to each instance of
695 the class. So the total RATE represented by an entry with
696 'occurs' will be the listed RATE multiplied by
697 <emphasis>number</emphasis>. For additional information, see
698 <ulink url="manpages/shorewall-mangle.html">mangle</ulink> (5)
699 or <ulink url="manpages/shorewall-tcrules.html">tcrules</ulink>
700 (5).</para>
701 </listitem>
702
703 <listitem>
704 <para>flow=<emphasis>keys</emphasis> - Shorewall attaches an SFQ
705 queuing discipline to each leaf HTB and HFSC class. SFQ ensures
706 that each <firstterm>flow</firstterm> gets equal access to the
707 interface. The default definition of a flow corresponds roughly
708 to a Netfilter connection. So if one internal system is running
709 BitTorrent, for example, it can have lots of 'flows' and can
710 thus take up a larger share of the bandwidth than a system
711 having only a single active connection. The
712 <option>flow</option> classifier (module cls_flow) works around
713 this by letting you define what a 'flow' is. The clasifier must
714 be used carefully or it can block off all traffic on an
715 interface! The flow option can be specified for an HTB or HFSC
716 leaf class (one that has no sub-classes). We recommend that you
717 use the following:</para>
718
719 <simplelist>
720 <member>Shaping internet-bound traffic: <emphasis
721 role="bold">flow=nfct-src</emphasis></member>
722
723 <member>Shaping traffic bound for your local net: <emphasis
724 role="bold">flow=dst</emphasis></member>
725 </simplelist>
726
727 <para>These will cause a 'flow' to consists of the traffic
728 to/from each internal system.</para>
729
730 <para>When more than one key is give, they must be enclosed in
731 parenthesis and separated by commas.</para>
732
733 <para>To see a list of the possible flow keys, run this
734 command:</para>
735
736 <blockquote>
737 <para><command>tc filter add flow help</command></para>
738 </blockquote>
739
740 <para>Those that begin with "nfct-" are Netfilter connection
741 tracking fields. As shown above, we recommend flow=nfct-src;
742 that means that we want to use the source IP address
743 <emphasis>before SNAT</emphasis> as the key.</para>
744
745 <note>
746 <para>Shorewall cannot determine ahead of time if the flow
747 classifier is available in your kernel (especially if it was
748 built into the kernel as opposed to being loaded as a module).
749 Consequently, you should check ahead of time to ensure that
750 both your kernel and 'tc' utility support the feature.</para>
751
752 <para>You can test the 'tc' utility by typing (as
753 root):</para>
754
755 <blockquote>
756 <para><command>tc filter add flow help</command></para>
757 </blockquote>
758
759 <para>If flow is supported, you will see:</para>
760
761 <programlisting> Usage: ... flow ...
762
763 [mapping mode]: map key KEY [ OPS ] ...
764 [hashing mode]: hash keys KEY-LIST ...
765
766 ...</programlisting>
767
768 <para>If 'flow' is not supported, you will see:</para>
769
770 <programlisting> Unknown filter "flow", hence option "help" is unparsable</programlisting>
771
772 <para>If your kernel supports module autoloading, just type
773 (as root):</para>
774
775 <blockquote>
776 <para><command>modprobe cls_flow</command></para>
777 </blockquote>
778
779 <para>If 'flow' is supported, no output is produced;
780 otherwise, you will see:</para>
781
782 <programlisting> FATAL: Module cls_flow not found.</programlisting>
783
784 <para>If your kernel is not modularized or does not support
785 module autoloading, look at your kernel configuration (either
786 <filename>/proc/config.gz</filename> or the
787 <filename>.config</filename> file in <filename
788 class="directory">/lib/modules/<kernel-version>/build/</filename></para>
789
790 <para>If 'flow' is supported, you will see: NET_CLS_FLOW=m or
791 NET_CLS_FLOW=y.</para>
792
793 <para>For modularized kernels, Shorewall will attempt to load
794 <filename>/lib/modules/<kernel-version>/net/sched/cls_flow.ko</filename>
795 by default.</para>
796 </note>
797 </listitem>
798
799 <listitem>
800 <para>pfifo - When specified for a leaf class, the pfifo queing
801 discipline is applied to the class rather than the sfq queuing
802 discipline.</para>
803 </listitem>
804
805 <listitem>
806 <para>limit=<emphasis>number</emphasis> - Added in Shorewall
807 4.4.3. When specified for a leaf class, specifies the maximum
808 number of packets that may be queued within the class. The
809 <emphasis>number</emphasis> must be > 2 and less than 128. If
810 not specified, the value 127 is assumed</para>
811 </listitem>
812
813 <listitem>
814 <para>red=(<replaceable>redoption</replaceable>,...) - Added in
815 Shorewall 4.5.6. When specified on a leaf class, causes the
816 class to use the red queuing discipline rather than SFQ. See
817 tc-red (8) for additional information.</para>
818
819 <para>See <ulink
820 url="manpages/shorewall-tcdevices.html">shorewall-tcdevices</ulink>
821 (5) for a description of the allowable
822 <replaceable>redoptions</replaceable>.</para>
823 </listitem>
824
825 <listitem>
826 <para>fq_codel[=(<replaceable>codeloption</replaceable>,...)] -
827 Added in Shorewall 4.5.12. When specified on a leaf class,
828 causes the class to use the FQ CODEL (<firstterm>Fair-queuing
829 Controlled-delay</firstterm>) queuing discipline rather than
830 SFQ. See tc-fq_codel (8) for additional information.</para>
831
832 <para>See <ulink
833 url="manpages/shorewall-tcclasses.html">shorewall-tcclasses</ulink>
834 (5) for a description of the allowable
835 <replaceable>codloptions</replaceable>.</para>
836 </listitem>
837 </itemizedlist>
838 </listitem>
839 </itemizedlist>
840 </section>
841
842 <section id="tcrules">
843 <title>/etc/shorewall/mangle and /etc/shorewall/rules</title>
844
845 <important>
846 <para>Unlike rules in the <ulink
847 url="manpages/shorewall-rules.html">shorewall-rules</ulink>(5) file,
848 evaluation of rules in this file will continue after a match. So the
849 final mark for each packet will be the one assigned by the LAST tcrule
850 that matches.</para>
851
852 <para>Also unlike rules in the <ulink
853 url="manpages/shorewall-rules.html">shorewall-rules</ulink>(5) file,
854 the mangle (tcrules) file is not stateful. So every packet that goes
855 into, out of or through your firewall is subject to entries in the
856 mangle (tcrules) file.</para>
857
858 <para>Because mangle (tcrules) entries are not stateful, it is
859 necessary to understand basic IP socket operation. Here is an edited
860 excerpt from a post on the Shorewall Users list:<blockquote>
861 <para>For the purposes of this discussion, the world is separated
862 into clients and servers. Servers provide services to
863 clients.</para>
864
865 <para>When a server starts, it creates a socket and
866 <emphasis>binds</emphasis> the socket to an
867 <emphasis>address</emphasis>. For AF_INET (IPv4) and AF_INET6
868 (IPv6) sockets, that address is an ordered triple consisting of an
869 IPv4 or IPv6 address, a protocol, and possibly a port number. Port
870 numbers are only used when the protocol is TCP, UDP, SCTP or DCCP.
871 The protocol and port number used by a server are typically
872 well-known so that clients will be able to connect to it or send
873 datagrams to it. So SSH servers bind to TCP port 22, SMTP servers
874 bind to TCP port 25, etc. We will call this port the SERVER
875 PORT.</para>
876
877 <para>When a client want to use the service provided by a server,
878 it also creates a socket and, like the server's socket, the
879 client's socket must be bound to an address. But in the case of
880 the client, the socket is usually given an automatic address
881 binding. For AF_INET and AF_INET6 sockets. the IP address is the
882 IP address of the client system (loose generalization) and the
883 port number is selected from a <firstterm>local port
884 range</firstterm>. On Linux systems, the local port range can be
885 seen by <command>cat
886 /proc/sys/net/ipv4/ip_local_port_range</command>. So it is not
887 possible in advance to determine what port the client will be
888 using. Whatever it is, we'll call it the CLIENT PORT.</para>
889
890 <para>Now: <blockquote>
891 <para>Packets sent from the client to the server will
892 have:<blockquote>
893 <para>SOURCE PORT = CLIENT PORT</para>
894
895 <para>DEST PORT = SERVER PORT</para>
896 </blockquote></para>
897
898 <para>Packets sent from the server to the client will have:
899 <blockquote>
900 <para>SOURCE PORT = SERVER PORT</para>
901
902 <para>DEST PORT = CLIENT PORT</para>
903 </blockquote></para>
904 </blockquote></para>
905
906 <para>Since the SERVER PORT is generally the only port known ahead
907 of time, we must categorize traffic from the server to the client
908 using the SOURCE PORT.</para>
909 </blockquote></para>
910 </important>
911
912 <para>The fwmark classifier provides a convenient way to classify
913 packets for traffic shaping. The
914 <filename>/etc/shorewall/mangle</filename>
915 (<filename>/etc/shorewall/tcrules</filename>) file is used for
916 specifying these marks in a tabular fashion. For an in-depth look at the
917 packet marking facility in Netfilter/Shorewall, please see <ulink
918 url="PacketMarking.html">this article</ulink>.</para>
919
920 <para><emphasis role="bold">For marking forwarded traffic, you must
921 either set MARK_IN_FORWARD_CHAIN=Yes shorewall.conf or by using the :F
922 qualifier (see below).</emphasis></para>
923
924 <para>See shorewall-mangle(5) and shorewall-tcrules(5) for a description
925 of the entries in these files. Note that the mangle file superseded the
926 tcrules file in Shorewall 4.6.0.</para>
927
928 <para>The following examples are for the mangle file.</para>
929
930 <example id="Example1">
931 <title/>
932
933 <para>All packets arriving on eth1 should be marked with 1. All
934 packets arriving on eth2 and eth3 should be marked with 2. All packets
935 originating on the firewall itself should be marked with 3.</para>
936
937 <programlisting>#ACTION SOURCE DEST PROTO DPORT
938 MARK(1) eth1 0.0.0.0/0 all
939 MARK(2) eth2 0.0.0.0/0 all
940 MARK(2) eth3 0.0.0.0/0 all
941 MARK(3) $FW 0.0.0.0/0 all</programlisting>
942 </example>
943
944 <example id="Example2">
945 <title/>
946
947 <para>All GRE (protocol 47) packets destined for 155.186.235.151
948 should be marked with 12.</para>
949
950 <programlisting>#ACTION SOURCE DEST PROTO DPORT
951 MARK(12):T 0.0.0.0/0 155.182.235.151 47</programlisting>
952 </example>
953
954 <example id="Example3">
955 <title/>
956
957 <para>All SSH request packets originating in 192.168.1.0/24 and
958 destined for 155.186.235.151 should be marked with 22.</para>
959
960 <programlisting>#ACTION SOURCE DEST PROTO DPORT
961 MARK(22):T 192.168.1.0/24 155.182.235.151 tcp 22</programlisting>
962 </example>
963
964 <example id="Example4">
965 <title/>
966
967 <para>All SSH packets packets going out of the first device in in
968 /etc/shorewall/tcdevices should be assigned to the class with mark
969 value 10.</para>
970
971 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT
972 CLASSIFY(1:110) 0.0.0.0/0 0.0.0.0/0 tcp 22
973 CLASSIFY(1:110) 0.0.0.0/0 0.0.0.0/0 tcp - 22</programlisting>
974 </example>
975
976 <example id="Example5">
977 <title/>
978
979 <para>Mark all ICMP echo traffic with packet mark 1. Mark all peer to
980 peer traffic with packet mark 4.</para>
981
982 <para>This is a little more complex than otherwise expected. Since the
983 ipp2p module is unable to determine all packets in a connection are
984 P2P packets, we mark the entire connection as P2P if any of the
985 packets are determined to match. We assume packet/connection mark 0 to
986 means unclassified. Traffic originating on the firewall is not covered
987 by this example.</para>
988
989 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT USER TEST
990 MARK(1) 0.0.0.0/0 0.0.0.0/0 icmp echo-request
991 MARK(1) 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
992
993 RESTORE 0.0.0.0/0 0.0.0.0/0 all - - - 0
994 CONTINUE 0.0.0.0/0 0.0.0.0/0 all - - - !0
995 MARK(4) 0.0.0.0/0 0.0.0.0/0 ipp2p:all
996 SAVE 0.0.0.0/0 0.0.0.0/0 all - - - !0</programlisting>
997
998 <para>The last four rules can be translated as:</para>
999
1000 <blockquote>
1001 <para>"If a packet hasn't been classified (packet mark is 0), copy
1002 the connection mark to the packet mark. If the packet mark is set,
1003 we're done. If the packet is P2P, set the packet mark to 4. If the
1004 packet mark has been set, save it to the connection mark."</para>
1005 </blockquote>
1006 </example>
1007
1008 <example>
1009 <title/>
1010
1011 <para>Mark all forwarded VOIP connections with connection mark 1 and
1012 ensure that all VOIP packets also receive that mark (assumes that
1013 nf_conntrack_sip is loaded).</para>
1014
1015 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT USER TEST CONNBYTES TOS HELPER
1016 RESTORE 0.0.0.0/0 0.0.0.0/0 all - - - 0
1017 CONTINUE 0.0.0.0/0 0.0.0.0/0 all - - - !0
1018 1 0.0.0.0/0 0.0.0.0/0 all - - - - - - sip
1019 SAVE 0.0.0.0/0 0.0.0.0/0 all - - - !0</programlisting>
1020 </example>
1021 </section>
1022
1023 <section id="ppp">
1024 <title>ppp devices</title>
1025
1026 <para>If you use ppp/pppoe/pppoa) to connect to your Internet provider
1027 and you use traffic shaping you need to restart shorewall traffic
1028 shaping. The reason for this is, that if the ppp connection gets
1029 restarted (and it usually does this at least daily), all
1030 <quote>tc</quote> filters/qdiscs related to that interface are
1031 deleted.</para>
1032
1033 <para>The easiest way to achieve this, is just to restart shorewall once
1034 the link is up. To achieve this add a small executable script
1035 to<quote>/etc/ppp/ip-up.d</quote>.</para>
1036
1037 <programlisting>#! /bin/sh
1038
1039 /sbin/shorewall refresh</programlisting>
1040 </section>
1041
1042 <section>
1043 <title>Sharing a TC configuration between Shorewall and
1044 Shorewall6</title>
1045
1046 <para>Beginning with Shorewall 4.4.15, the traffic-shaping configuration
1047 in the tcdevices, tcclasses and tcfilters files can be shared between
1048 Shorewall and Shorewall6. Only one of the products can control the
1049 configuration but the other can configure CLASSIFY rules in its own
1050 mangle or tcrules file that refer to the shared classes.</para>
1051
1052 <para>To defined the configuration in Shorewall and shared it with
1053 Shorewall6:</para>
1054
1055 <itemizedlist>
1056 <listitem>
1057 <para>Set TC_ENABLED=Internal in <ulink
1058 url="manpages/shorewall.conf.html">shorewall.conf</ulink>
1059 (5).</para>
1060 </listitem>
1061
1062 <listitem>
1063 <para>Set TC_ENABLED=Shared in <ulink
1064 url="manpages/shorewall.conf.html">shorewall6.conf</ulink>
1065 (5).</para>
1066 </listitem>
1067
1068 <listitem>
1069 <para>Create symbolic links from /etc/shorewall6 to
1070 /etc/shorewall/tcdevices and /etc/shorewall/tcclasses:</para>
1071
1072 <programlisting>ln -s ../shorewall/tcdevices /etc/shorewall6/tcdevices
1073 ln -s ../shorewall/tcclasses /etc/shorewall6/tcclasses</programlisting>
1074 </listitem>
1075
1076 <listitem>
1077 <para>If you need to define IPv6 tcfilter entries, do so in
1078 /etc/shorewall/tcfilters. That file now allows entries that apply to
1079 IPv6.</para>
1080 </listitem>
1081 </itemizedlist>
1082
1083 <para>Shorewall6 compilations to have access to the tcdevices and
1084 tcclasses files although it will create no output. That access allows
1085 CLASSIFY rules in /etc/shorewall6/mangle to be validated against the TC
1086 configuration.</para>
1087
1088 <para>In this configuration, it is Shorewall that controls TC
1089 configuration (except for IPv6 mangle). You can reverse the settings in
1090 the files if you want to control the configuration using
1091 Shorewall6.</para>
1092 </section>
1093
1094 <section id="perIP">
1095 <title>Per-IP Traffic Shaping</title>
1096
1097 <para>Some network administrators feel that they have to divy up their
1098 available bandwidth by IP address rather than by prioritizing the
1099 traffic based on the type of traffic. This gets really awkward when
1100 there are a large number of local IP addresses.</para>
1101
1102 <para>This section describes the Shorewall facility for making this
1103 configuration less tedious (and a lot more efficient). Note that it
1104 requires that you <ulink url="Dynamic.html#xtables-addons">install
1105 xtables-addons</ulink>. So before you try this facility, we suggest that
1106 first you add the following OPTION to each external interface described
1107 in /etc/shorewall/tcdevices:</para>
1108
1109 <programlisting>flow=nfct-src</programlisting>
1110
1111 <para>If you shape traffic on your internal interface(s), then add this
1112 to their entries:</para>
1113
1114 <programlisting>flow=dst</programlisting>
1115
1116 <para>You may find that this simple change is all that is needed to
1117 control bandwidth hogs like Bit Torrent. If it doesn't, then proceed as
1118 described in this section.</para>
1119
1120 <para>The facility has two components:</para>
1121
1122 <orderedlist>
1123 <listitem>
1124 <para>An IPMARK MARKing command in
1125 <filename>/etc/shorewall/mangle</filename>
1126 (<filename>/etc/shorewall/tcrules</filename>).</para>
1127 </listitem>
1128
1129 <listitem>
1130 <para>An <emphasis role="bold">occurs</emphasis> OPTION in
1131 /etc/shorewall/tcclasses.</para>
1132 </listitem>
1133 </orderedlist>
1134
1135 <para>The facility is currently only available with IPv4.</para>
1136
1137 <para>In a sense, the IPMARK target is more like an IPCLASSIFY target in
1138 that the mark value is later interpreted as a class ID. A packet mark is
1139 32 bits wide; so is a class ID. The <emphasis>major</emphasis> class
1140 occupies the high-order 16 bits and the <emphasis>minor</emphasis> class
1141 occupies the low-order 16 bits. So the class ID 1:4ff (remember that
1142 class IDs are always in hex) is equivalent to a mark value of 0x104ff.
1143 Remember that Shorewall uses the interface number as the
1144 <emphasis>major</emphasis> number where the first interface in tcdevices
1145 has <emphasis>major</emphasis> number 1, the second has
1146 <emphasis>major</emphasis> number 2, and so on.</para>
1147
1148 <para>The IPMARK target assigns a mark to each matching packet based on
1149 the either the source or destination IP address. By default, it assigns
1150 a mark value equal to the low-order 8 bits of the source address.</para>
1151
1152 <para>The syntax is as follows:</para>
1153
1154 <blockquote>
1155 <para><emphasis role="bold">IPMARK</emphasis>[<emphasis
1156 role="bold">(</emphasis>[{<emphasis
1157 role="bold">src</emphasis>|<emphasis
1158 role="bold">dst</emphasis>}][<emphasis
1159 role="bold">,</emphasis>[<emphasis>mask1</emphasis>][,[<emphasis>mask2</emphasis>][<emphasis
1160 role="bold">,</emphasis>[<emphasis>shift</emphasis>]]]]<emphasis
1161 role="bold">)</emphasis>]</para>
1162 </blockquote>
1163
1164 <para>Default values are:</para>
1165
1166 <simplelist>
1167 <member>src</member>
1168
1169 <member>mask1 = 0xFF</member>
1170
1171 <member>mask2 = 0x00</member>
1172
1173 <member>shift = 0</member>
1174 </simplelist>
1175
1176 <para><emphasis role="bold">src</emphasis> and <emphasis
1177 role="bold">dst</emphasis> specify whether the mark is to be based on
1178 the source or destination address respectively. The selected address is
1179 first shifted right by <emphasis>shift</emphasis>, then LANDed with
1180 <emphasis>mask1</emphasis> and then LORed with
1181 <emphasis>mask2</emphasis>. The <emphasis>shift</emphasis> argument is
1182 intended to be used primarily with IPv6 addresses.</para>
1183
1184 <para>Example:</para>
1185
1186 <programlisting>IPMARK(src,0xff,0x10100)
1187
1188 Source IP address is 192.168.4.3 = 0xc0a80403
1189
1190 0xc0a80403 >> 0 = 0xc0a80403
1191 0xc0a80403 LAND 0xFF = 0x03
1192 0x03 LOR 0x10100 = 0x10103
1193
1194 So the mark value is 0x10103 which corresponds to class id 1:103.</programlisting>
1195
1196 <para>It is important to realize that, while class IDs are composed of a
1197 <emphasis>major</emphasis> and a <emphasis>minor</emphasis> value, the
1198 set of <emphasis>minor</emphasis> values must be unique. You must keep
1199 this in mind when deciding how to map IP addresses to class IDs. For
1200 example, suppose that your internal network is 192.168.1.0/29 (host IP
1201 addresses 192.168.1.1 - 192.168.1.6). Your first notion might be to use
1202 IPMARK(src,0xFF,0x10000) so as to produce class IDs 1:1 through 1:6. But
1203 1:1 is the class ID of the base HTB class on interface 1. So you might
1204 chose instead to use IPMARK(src,0xFF,0x10100) as shown in the example
1205 above so as to avoid minor class 1.</para>
1206
1207 <para>The <emphasis role="bold">occurs</emphasis> option in
1208 <filename>/etc/shorewall/tcclasses</filename> causes the class
1209 definition to be replicated many times.</para>
1210
1211 <para>The synax is:</para>
1212
1213 <blockquote>
1214 <para><emphasis
1215 role="bold">occurs=</emphasis><emphasis>number</emphasis></para>
1216 </blockquote>
1217
1218 <para>When <emphasis role="bold">occurs</emphasis> is used:</para>
1219
1220 <orderedlist numeration="loweralpha">
1221 <listitem>
1222 <para>The associated device may not have the <emphasis
1223 role="bold">classify</emphasis> option.</para>
1224 </listitem>
1225
1226 <listitem>
1227 <para>The class may not be the default class.</para>
1228 </listitem>
1229
1230 <listitem>
1231 <para>The class may not have any <emphasis
1232 role="bold">tos=</emphasis> options (including <emphasis
1233 role="bold">tcp-ack</emphasis>).</para>
1234 </listitem>
1235 </orderedlist>
1236
1237 <para>The class should not specify a MARK value. Any MARK value given is
1238 ignored with a warning. The RATE and CEIL parameters apply to each
1239 instance of the class. So the total RATE represented by an entry with
1240 <emphasis role="bold">occurs</emphasis> will be the listed RATE
1241 multiplied by <emphasis>number</emphasis>.</para>
1242
1243 <para>Example:</para>
1244
1245 <para><filename>/etc/shorewall/tcdevices</filename>:</para>
1246
1247 <programlisting>#INTERFACE IN_BANDWIDTH OUT_BANDWIDTH
1248 eth0 100mbit 100mbit</programlisting>
1249
1250 <para><filename>/etc/shorewall/tcclasses</filename>:</para>
1251
1252 <programlisting>#DEVICE MARK RATE CEIL PRIORITY OPTIONS
1253 eth0:101 - 1kbit 230kbit 4 occurs=6</programlisting>
1254
1255 <para>The above defines 6 classes with class IDs 0x101-0x106. Each class
1256 has a guaranteed rate of 1kbit/second and a ceiling of 230kbit.</para>
1257
1258 <para><filename>/etc/shoreall/mangle</filename> or
1259 <filename>/etc/shoreall/tcrules</filename>:</para>
1260
1261 <programlisting>#ACTION SOURCE DEST
1262 IPMARK(src,0xff,0x10100):F 192.168.1.0/29 eth0</programlisting>
1263
1264 <para>This facility also alters the way in which Shorewall generates a
1265 class number when none is given. Prior to the implementation of this
1266 facility, the class number was constructed by concatinating the MARK
1267 value with the either '1' or '10'. '10' was used when there were more
1268 than 10 devices defined in
1269 <filename>/etc/shorewall/tcdevices</filename>.</para>
1270
1271 <para>With this facility, a new method is added; class numbers are
1272 assigned sequentially beginning with 2. The WIDE_TC_MARKS option in
1273 <filename>shorewall.conf</filename> selects which construction to use.
1274 WIDE_TC_MARKS=No (the default) produces pre-Shorewall 4.4 behavior.
1275 WIDE_TC_MARKS=Yes (TC_BITS >= 14 in Shorewall 4.4.26 and later)
1276 produces the new behavior.</para>
1277 </section>
1278
1279 <section id="Real">
1280 <title>Real life examples</title>
1281
1282 <section>
1283 <title>A Shorewall User's Experience</title>
1284
1285 <para>Chuck Kollars has provided <ulink
1286 url="http://www.ckollars.org/shaping.html">an excellent
1287 writeup</ulink> about his traffic shaping experiences.</para>
1288 </section>
1289
1290 <section id="Wondershaper">
1291 <title>Configuration to replace Wondershaper</title>
1292
1293 <para>You are able to fully replace the wondershaper script by using
1294 the buitin traffic control.. In this example it is assumed that your
1295 interface for your Internet connection is ppp0 (for DSL), if you use
1296 another connection type, you have to change it. You also need to
1297 change the settings in the tcdevices.wondershaper file to reflect your
1298 line speed. The relevant lines of the config files follow here. Please
1299 note that this is just a 1:1 replacement doing exactly what
1300 wondershaper should do. You are free to change it...</para>
1301
1302 <section id="realtcd">
1303 <title>tcdevices file</title>
1304
1305 <programlisting>#INTERFACE IN_BANDWITH OUT_BANDWIDTH
1306 ppp0 5000kbit 500kbit</programlisting>
1307 </section>
1308
1309 <section id="realtcc">
1310 <title>tcclasses file</title>
1311
1312 <programlisting>#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
1313 ppp0 1 5*full/10 full 1 tcp-ack,tos-minimize-delay
1314 ppp0 2 3*full/10 9*full/10 2 default
1315 ppp0 3 2*full/10 8*full/10 2</programlisting>
1316 </section>
1317
1318 <section id="realtcr">
1319 <title>mangle file</title>
1320
1321 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT USER
1322 MARK(1):F 0.0.0.0/0 0.0.0.0/0 icmp echo-request
1323 MARK(1):F 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
1324 # mark traffic which should have a lower priority with a 3:
1325 # mldonkey
1326 MARK(3):F 0.0.0.0/0 0.0.0.0/0 udp - 4666</programlisting>
1327
1328 <para>Wondershaper allows you to define a set of hosts and/or ports
1329 you want to classify as low priority. To achieve this , you have to
1330 add these hosts to tcrules and set the mark to 3 (true if you use
1331 the example configuration files).</para>
1332 </section>
1333
1334 <section id="lowpro">
1335 <title>Setting hosts to low priority</title>
1336
1337 <para>lets assume the following settings from your old wondershaper
1338 script (don't assume these example values are really useful, they
1339 are only used for demonstrating ;-):</para>
1340
1341 <programlisting>
1342 # low priority OUTGOING traffic - you can leave this blank if you want
1343 # low priority source netmasks
1344 NOPRIOHOSTSRC="192.168.1.128/25 192.168.3.28"
1345
1346 # low priority destination netmasks
1347 NOPRIOHOSTDST=60.0.0.0/24
1348
1349 # low priority source ports
1350 NOPRIOPORTSRC="6662 6663"
1351
1352 # low priority destination ports
1353 NOPRIOPORTDST="6662 6663" </programlisting>
1354
1355 <para>This would result in the following additional settings to the
1356 mangle file:</para>
1357
1358 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT USER
1359 MARK(3) 192.168.1.128/25 0.0.0.0/0 all
1360 MARK(3) 192.168.3.28 0.0.0.0/0 all
1361 MARK(3) 0.0.0.0/0 60.0.0.0/24 all
1362 MARK(3) 0.0.0.0/0 0.0.0.0/0 udp 6662,6663
1363 MARK(3) 0.0.0.0/0 0.0.0.0/0 udp - 6662,6663
1364 MARK(3) 0.0.0.0/0 0.0.0.0/0 tcp 6662,6663
1365 MARK(3) 0.0.0.0/0 0.0.0.0/0 tcp - 6662,6663</programlisting>
1366 </section>
1367 </section>
1368
1369 <section id="simiple">
1370 <title>A simple setup</title>
1371
1372 <para>This is a simple setup for people sharing an Internet connection
1373 and using different computers for this. It just basically shapes
1374 between 2 hosts which have the ip addresses 192.168.2.23 and
1375 192.168.2.42</para>
1376
1377 <section id="simpletcd">
1378 <title>tcdevices file</title>
1379
1380 <programlisting>#INTERFACE IN_BANDWITH OUT_BANDWIDTH
1381 ppp0 6000kbit 700kbit</programlisting>
1382
1383 <para>We have 6mbit down and 700kbit upstream.</para>
1384 </section>
1385
1386 <section id="simpletcc">
1387 <title>tcclasses file</title>
1388
1389 <programlisting>#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
1390 ppp0 1 10kbit 50kbit 1 tcp-ack,tos-minimize-delay
1391 ppp0 2 300kbit full 2
1392 ppp0 3 300kbit full 2
1393 ppp0 4 90kbit 200kbit 3 default</programlisting>
1394
1395 <para>We add a class for tcp ack packets with highest priority, so
1396 that downloads are fast. The following 2 classes share most of the
1397 bandwidth between the 2 hosts, if the connection is idle, they may
1398 use full speed. As the hosts should be treated equally they have the
1399 same priority. The last class is for the remaining traffic.</para>
1400 </section>
1401
1402 <section id="simpletcr">
1403 <title>mangle file</title>
1404
1405 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT USER
1406 MARK(1):F 0.0.0.0/0 0.0.0.0/0 icmp echo-request
1407 MARK(1):F 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
1408 MARK(2):F 192.168.2.23 0.0.0.0/0 all
1409 MARK(3):F 192.168.2.42 0.0.0.0/0 all</programlisting>
1410
1411 <para>Corresponding tcrules file:</para>
1412
1413 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT USER
1414 1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-request
1415 1:F 0.0.0.0/0 0.0.0.0/0 icmp echo-reply
1416 2:F 192.168.2.23 0.0.0.0/0 all
1417 3:F 192.168.2.42 0.0.0.0/0 all</programlisting>
1418
1419 <para>We mark icmp ping and replies so they will go into the fast
1420 interactive class and set a mark for each host.</para>
1421 </section>
1422 </section>
1423 </section>
1424 </section>
1425
1426 <section id="Xen">
1427 <title>A Warning to Xen Users</title>
1428
1429 <para>If you are running traffic shaping in your dom0 and traffic shaping
1430 doesn't seem to be limiting outgoing traffic properly, it may be due to
1431 "checksum offloading" in your domU(s). Check the output of "shorewall show
1432 tc". Here's an excerpt from the output of that command:</para>
1433
1434 <programlisting>class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1500 rate 76000bit <emphasis
1435 role="bold">ceil 230000bit</emphasis> burst 1537b/8 mpu 0b overhead 0b cburst 1614b/8 mpu 0b overhead 0b level 0
1436 Sent 559018700 bytes 75324 pkt (dropped 0, overlimits 0 requeues 0)
1437 <emphasis role="bold">rate 299288bit</emphasis> 3pps backlog 0b 0p requeues 0
1438 lended: 53963 borrowed: 21361 <emphasis role="bold">giants: 90174</emphasis>
1439 tokens: -26688 ctokens: -14783</programlisting>
1440
1441 <para>There are two obvious problems in the above output:</para>
1442
1443 <orderedlist>
1444 <listitem>
1445 <para>The rate (299288) is considerably larger than the ceiling
1446 (230000).</para>
1447 </listitem>
1448
1449 <listitem>
1450 <para>There are a large number (90174) of giants reported.</para>
1451 </listitem>
1452 </orderedlist>
1453
1454 <para>This problem will be corrected by disabling "checksum offloading" in
1455 your domU(s) using the <command>ethtool</command> utility. See the <ulink
1456 url="XenMyWay-Routed.html">one of the Xen articles</ulink> for
1457 instructions.</para>
1458 </section>
1459
1460 <section id="HFSC">
1461 <title>An HFSC Example</title>
1462
1463 <para>As mentioned at the top of this article, there is an excellent
1464 introduction to HFSC at <ulink
1465 url="http://linux-ip.net/articles/hfsc.en/">http://linux-ip.net/articles/hfsc.en/</ulink>.
1466 At the end of that article are 'tc' commands that implement the
1467 configuration in the article. Those tc commands correspond to the
1468 following Shorewall traffic shaping configuration.</para>
1469
1470 <para><filename>/etc/shorewall/tcdevices</filename>:</para>
1471
1472 <programlisting>#INTERFACE IN_BANDWITH OUT_BANDWIDTH OPTIONS
1473 eth0 - 1000kbit hfsc</programlisting>
1474
1475 <para><filename>/etc/shorewall/tcclasses</filename>:</para>
1476
1477 <programlisting>#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
1478 1:10 1 500kbit full 1
1479 1:20 2 500kbit full 1
1480 1:10:11 3 400kbit:53ms:1500b full 2
1481 1:10:12 4 100kbit:30ms:1500b full 2</programlisting>
1482
1483 <para>The following sub-section offers some notes about the
1484 article.</para>
1485
1486 <section id="MajicNumbers">
1487 <title>Where Did all of those Magic Numbers come from?</title>
1488
1489 <para>As you read the article, numbers seem to be introduced out of thin
1490 air. I'll try to shed some light on those.</para>
1491
1492 <para>There is very clear development of these numbers:</para>
1493
1494 <itemizedlist>
1495 <listitem>
1496 <para>12ms to transfer a 1500b packet at 1000kbits/second.</para>
1497 </listitem>
1498
1499 <listitem>
1500 <para>100kbits per second with 1500b packets, requires 8 packets per
1501 second.</para>
1502 </listitem>
1503
1504 <listitem>
1505 <para>A packet from class 1:12 must be sent every 120ms.</para>
1506 </listitem>
1507
1508 <listitem>
1509 <para>Total transmit delay can be no more than 132ms (120 +
1510 12).</para>
1511 </listitem>
1512 </itemizedlist>
1513
1514 <para>We then learn that the queuing latency can be reduced to 30ms if
1515 we use a two-part service curve whose first part is 400kbits/second.
1516 Where did those come from?</para>
1517
1518 <itemizedlist>
1519 <listitem>
1520 <para>The latency is calculated from the rate. If it takes 12ms to
1521 transmit a 1500 byte packet at 1000kbits/second, it takes 30ms to
1522 transmit a 1500b at 400kbits/second.</para>
1523 </listitem>
1524
1525 <listitem>
1526 <para>For the slope of the first part of the service curve, in
1527 theory we can pick any number between 100 (the rate of class 1:12)
1528 and 500 (the rate of the parent class) with higher numbers providing
1529 lower latency.</para>
1530 </listitem>
1531 </itemizedlist>
1532
1533 <para>The final curious number is the latency for class 1:11 - 52.5ms.
1534 It is a consequence of everything that has gone before.</para>
1535
1536 <para>To acheive 400kbits/second with 1500-byte packets, 33.33 packets
1537 per second are required. So a packet from class 1:11 must be sent every
1538 30 ms. As the article says, "...the maximum transmission delay of this
1539 class increases from 30ms to a total of 52.5 ms.". So we are looking for
1540 an additional 22.5 ms.</para>
1541
1542 <para>Assume that both class 1:11 and 1:12 transmit for 30 ms at
1543 400kbits/second. That is a total of 800kbits/second for 30ms. So Class
1544 1:11 is punished for the excess. How long is the punishment? The two
1545 classes sent 24,000 bits in 30ms; they are only allowed 0.030 * 500,000
1546 = 15,000. So they are 9,000 bits over their quota. The amount of time
1547 required to transmit 9,000 bits at 400,000 bits/second is
1548 22.5ms!.</para>
1549 </section>
1550 </section>
1551
1552 <section id="IFB">
1553 <title>Intermediate Functional Block (IFB) Devices</title>
1554
1555 <para>The principles behind an IFB is fairly simple:</para>
1556
1557 <itemizedlist>
1558 <listitem>
1559 <para>It looks like a network interface although it is never given an
1560 IPv4 configuration.</para>
1561 </listitem>
1562
1563 <listitem>
1564 <para>Because it is a network interface, queuing disciplines can be
1565 associated with an IFB.</para>
1566 </listitem>
1567 </itemizedlist>
1568
1569 <para>The magic of an IFB comes in the fact that a filter may be defined
1570 on a real network interface such that each packet that arrives on that
1571 interface is queued for the IFB! In that way, the IFB provides a means for
1572 shaping input traffic.</para>
1573
1574 <para>To use an IFB, you must have IFB support in your kernel
1575 (configuration option CONFIG_IFB). Assuming that you have a modular
1576 kernel, the name of the IFB module is 'ifb' and may be loaded using the
1577 command <command>modprobe ifb</command> (if you have modprobe installed)
1578 or <command>insmod /path/to/module/ifb</command>.</para>
1579
1580 <para>By default, two IFB devices (ifb0 and ifb1) are created. You can
1581 control that using the numifbs option (e.g., <command>modprobe ifb
1582 numifbs=1</command>).</para>
1583
1584 <para>To create a single IFB when Shorewall starts, place the following
1585 two commands in <filename>/etc/shorewall/init</filename>:</para>
1586
1587 <programlisting><command>modprobe ifb numifbs=1
1588 ip link set ifb0 up</command></programlisting>
1589
1590 <para>Entries in <filename>/etc/shorewall/mangle</filename> or
1591 <filename>/etc/shorewall/tcrules</filename> have no effect on shaping
1592 traffic through an IFB unless the IFB is defined in shorewall-tcclasses(5)
1593 with the <emphasis role="bold">connmark</emphasis> option. To allow
1594 classification of such traffic, the /etc/shorewall/tcfilters file has been
1595 added. Entries in that file create <ulink
1596 url="http://b42.cz/notes/u32_classifier/">u32 classification
1597 rules</ulink>.</para>
1598
1599 <section id="tcfilters">
1600 <title>/etc/shorewall/tcfilters</title>
1601
1602 <para>While this file was created to allow shaping of traffic through an
1603 IFB, the file may be used for general traffic classification as well.
1604 The file is similar to <ulink
1605 url="shorewall-mangle.html">shorewall-mangle</ulink>(5) with the
1606 following key exceptions:</para>
1607
1608 <itemizedlist>
1609 <listitem>
1610 <para>The first match determines the classification, whereas in the
1611 mangle file, the last match determines the classification.</para>
1612 </listitem>
1613
1614 <listitem>
1615 <para>ipsets are not supported</para>
1616 </listitem>
1617
1618 <listitem>
1619 <para>DNS Names are not supported</para>
1620 </listitem>
1621
1622 <listitem>
1623 <para>Address ranges and lists are not supported</para>
1624 </listitem>
1625
1626 <listitem>
1627 <para>Exclusion is not supported.</para>
1628 </listitem>
1629
1630 <listitem>
1631 <para>filters are applied to packets as they <emphasis>appear on the
1632 wire</emphasis>. So incoming packets will not have DNAT applied yet
1633 (the destination IP address will be the external address) and
1634 outgoing packets will have had SNAT applied.</para>
1635 </listitem>
1636 </itemizedlist>
1637
1638 <para>The last point warrants elaboration. When looking at traffic being
1639 shaped by an IFB, there are two cases to consider:</para>
1640
1641 <orderedlist>
1642 <listitem>
1643 <para>Requests — packets being sent from remote clients to local
1644 servers. These packets may undergo subsequent DNAT, either as a
1645 result of entries in <filename>/etc/shorewall/nat</filename> or as a
1646 result of DNAT or REDIRECT rules.</para>
1647
1648 <para>Example: <filename>/etc/shorewall/rules</filename>:</para>
1649
1650 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT ORIGDEST
1651 DNAT net dmz:192.168.4.5 tcp 80 - 206.124.146.177</programlisting>
1652
1653 <para>Requests redirected by this rule will have destination IP
1654 address 206.124.146.177 and destination port 80.</para>
1655 </listitem>
1656
1657 <listitem>
1658 <para>Responses — packets being sent from remote servers to local
1659 clients. These packets may undergo subsequent DNAT as a result of
1660 entries in <filename>/etc/shorewall/nat</filename> or in
1661 <filename>/etc/shorewall/masq</filename>. The packet's destination
1662 IP address will be the external address specified in the
1663 entry.</para>
1664
1665 <para>Example:
1666 <filename>/etc/shorewall/masq</filename>:<programlisting>#INTERFACE SOURCE ADDRESS
1667 eth0 192.168.1.0/24 206.124.146.179</programlisting></para>
1668
1669 <para>When running Shorewall 5.0.14 or later, the equivalent
1670 <filename>/etc/shorewall/snat</filename> would be:</para>
1671
1672 <programlisting>#ACTION SOURCE DEST ...
1673 SNAT(206.124.146.179) 192.168.1.0/24 eth0</programlisting>
1674
1675 <para>HTTP response packets corresponding to requests that fall
1676 under that rule will have destination IP address 206.124.146.179 and
1677 <emphasis role="bold">source</emphasis> port 80.</para>
1678 </listitem>
1679 </orderedlist>
1680
1681 <para>Beginning with Shorewall 4.4.15, both IPv4 and IPv6 rules can be
1682 defined in this file. See <ulink
1683 url="manpages/shorewall-tcfilters.html">shorewall-tcfilters</ulink> (5)
1684 for details.</para>
1685
1686 <para>Columns in the file are as follow. As in all Shorewall
1687 configuration files, a hyphen ("-") may be used to indicate that no
1688 value is supplied in the column.</para>
1689
1690 <variablelist>
1691 <varlistentry>
1692 <term>CLASS</term>
1693
1694 <listitem>
1695 <para>The interface name or number followed by a colon (":") and
1696 the class number.</para>
1697 </listitem>
1698 </varlistentry>
1699
1700 <varlistentry>
1701 <term>SOURCE</term>
1702
1703 <listitem>
1704 <para>SOURCE IP address (host or network). DNS names are not
1705 allowed.</para>
1706 </listitem>
1707 </varlistentry>
1708
1709 <varlistentry>
1710 <term>DEST</term>
1711
1712 <listitem>
1713 <para>DESTINATION IP address (host or network). DNS names are not
1714 allowed.</para>
1715 </listitem>
1716 </varlistentry>
1717
1718 <varlistentry>
1719 <term>PROTO</term>
1720
1721 <listitem>
1722 <para>Protocol name or number.</para>
1723 </listitem>
1724 </varlistentry>
1725
1726 <varlistentry>
1727 <term>DPORT</term>
1728
1729 <listitem>
1730 <para>Comma-separated list of destination port names or numbers.
1731 May only be specified if the protocol is TCP, UDP, SCTP or ICMP.
1732 Port ranges are supported except for ICMP.</para>
1733 </listitem>
1734 </varlistentry>
1735
1736 <varlistentry>
1737 <term>SPORT</term>
1738
1739 <listitem>
1740 <para>Comma-separated list of source port names or numbers. May
1741 only be specified if the protocol is TCP, UDP or SCTP. Port ranges
1742 are supported.</para>
1743 </listitem>
1744 </varlistentry>
1745
1746 <varlistentry>
1747 <term>TOS</term>
1748
1749 <listitem>
1750 <para>Specifies the value of the TOS field. The value can be any
1751 of the following:</para>
1752
1753 <itemizedlist>
1754 <listitem>
1755 <para><option>tos-minimize-delay</option></para>
1756 </listitem>
1757
1758 <listitem>
1759 <para><option>tos-maximuze-throughput</option></para>
1760 </listitem>
1761
1762 <listitem>
1763 <para><option>tos-maximize-reliability</option></para>
1764 </listitem>
1765
1766 <listitem>
1767 <para><option>tos-minimize-cost</option></para>
1768 </listitem>
1769
1770 <listitem>
1771 <para><option>tos-normal-service</option></para>
1772 </listitem>
1773
1774 <listitem>
1775 <para><replaceable>hex-number</replaceable></para>
1776 </listitem>
1777
1778 <listitem>
1779 <para><replaceable>hex-number</replaceable>/<replaceable>hex-number</replaceable></para>
1780 </listitem>
1781 </itemizedlist>
1782
1783 <para>The <replaceable>hex-number</replaceable>s must be exactly
1784 two digits (e.g., 0x04).</para>
1785 </listitem>
1786 </varlistentry>
1787
1788 <varlistentry>
1789 <term>LENGTH</term>
1790
1791 <listitem>
1792 <para>Must be a power of 2 between 32 and 8192 inclusive. Packets
1793 with a total length that is strictly less than the specified value
1794 will match the rule.</para>
1795 </listitem>
1796 </varlistentry>
1797 </variablelist>
1798
1799 <para>Example:</para>
1800
1801 <para>I've used this configuration on my own firewall. The IFB portion
1802 is more for test purposes rather than to serve any well-reasoned QOS
1803 strategy.</para>
1804
1805 <para><filename>/etc/shorewall/init</filename>:<programlisting>qt modprobe ifb numifbs=1
1806 qt ip link set dev ifb0 up</programlisting></para>
1807
1808 <para><filename>/etc/shorewall/interfaces</filename>:</para>
1809
1810 <programlisting>#ZONE INTERFACE BROADCAST
1811 - ifb0</programlisting>
1812
1813 <para><filename>/etc/shorewall/tcdevices</filename>:</para>
1814
1815 <para><programlisting>
1816 #INTERFACE IN_BANDWITH OUT_BANDWIDTH OPTIONS REDIRECT
1817 1:eth0 - 384kbit classify
1818 2:ifb0 - 1300kbit - eth0</programlisting>
1819 <filename>/etc/shorewall/tcclasses</filename>:<programlisting>#INTERFACE MARK RATE CEIL PRIORITY OPTIONS
1820 1:110 - 5*full/10 full 1 tcp-ack,tos-minimize-delay
1821 1:120 - 2*full/10 6*full/10 2 default
1822 1:130 - 2*full/10 6*full/10 3
1823 2:110 - 5*full/10 full 1 tcp-ack,tos-minimize-delay
1824 2:120 - 2*full/10 6*full/10 2 default
1825 2:130 - 2*full/10 6*full/10 3</programlisting><filename>/etc/shorewall/tcfilters</filename>:<programlisting>#INTERFACE: SOURCE DEST PROTO DPORT SPORT
1826 #
1827 # OUTGOING TRAFFIC
1828 #
1829 1:130 206.124.146.178 - tcp - 49441,49442 #BITTORRENT on wookie
1830 1:110 206.124.146.178 #wookie
1831 1:110 206.124.146.179 #SNAT of internal systems
1832 1:110 206.124.146.180 #Work Laptop
1833 1:110 - - icmp echo-request,echo-reply
1834 1:110 - - icmp echo-reply
1835 1:130 206.124.146.177 - tcp - 873,25 #Bulk Traffic
1836 #
1837 # INCOMING TRAFFIC
1838 #
1839 2:110 - 206.124.146.178 #Wookie
1840 2:110 - 206.124.146.179 #SNAT Responses
1841 2:110 - 206.124.146.180 #Work Laptop
1842 2:130 - 206.124.146.177 tcp 25 #Incoming Email.</programlisting></para>
1843
1844 <para>You can examine the installed filters with the <command>shorewall
1845 show filters</command> command. What follows shows the output for
1846 <filename class="devicefile">eth0</filename> with the filters shown
1847 above. <emphasis role="bold">Bold font</emphasis> are comments
1848 explaining the rules.<programlisting>gateway:~ # shorewall-lite show filters
1849 Shorewall Lite 4.1.6 Classifiers at gateway - Fri Mar 21 08:06:47 PDT 2008
1850
1851 Device eth1:
1852
1853 Device eth2:
1854
1855 Device eth0:
1856 filter parent 1: protocol ip pref 10 u32
1857 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 3:</emphasis> ht divisor 1 <emphasis
1858 role="bold"> <========= Start of table 3. parses TCP header</emphasis>
1859
1860 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 3</emphasis>::800 order 2048 key ht 3 bkt 0 <emphasis
1861 role="bold">flowid 1:130</emphasis> (rule hit 102 success 0)
1862 match 03690000/ffff0000 at nexthdr+0 (success 0 ) <emphasis
1863 role="bold"> <========= SOURCE PORT 873 goes to class 1:130</emphasis>
1864
1865 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 2:</emphasis> ht divisor 1 <emphasis
1866 role="bold"> <========= Start of table 2. parses ICMP header</emphasis>
1867
1868 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 2</emphasis>::800 order 2048 key ht 2 bkt 0 <emphasis
1869 role="bold">flowid 1:110</emphasis> (rule hit 0 success 0)
1870 match 08000000/ff000000 at nexthdr+0 (success 0 ) <emphasis
1871 role="bold"> <========= ICMP Type 8 goes to class 1:110</emphasis>
1872
1873 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 2</emphasis>::801 order 2049 key ht 2 bkt 0 <emphasis
1874 role="bold">flowid 1:110</emphasis> (rule hit 0 success 0)
1875 match 00000000/ff000000 at nexthdr+0 (success 0 ) <emphasis
1876 role="bold"> <========= ICMP Type 0 goes to class 1:110</emphasis>
1877
1878 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 1:</emphasis> ht divisor 1 <emphasis
1879 role="bold"> <========= Start of table 1. parses TCP header</emphasis>
1880
1881 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 1:</emphasis>:800 order 2048 key ht 1 bkt 0 <emphasis
1882 role="bold">flowid 1:130</emphasis> (rule hit 0 success 0)
1883 match c1210000/ffff0000 at nexthdr+0 (success 0 ) <emphasis
1884 role="bold"> <========= SOURCE PORT 49441 goes to class 1:130</emphasis>
1885
1886 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 1</emphasis>::801 order 2049 key ht 1 bkt 0 <emphasis
1887 role="bold">flowid 1:130</emphasis> (rule hit 0 success 0)
1888 match c1220000/ffff0000 at nexthdr+0 (success 0 ) <emphasis
1889 role="bold"> <========= SOURCE PORT 49442 goes to class 1:130</emphasis>
1890
1891 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis> ht divisor 1 <emphasis
1892 role="bold"><========= Start of Table 800. Packets start here!</emphasis>
1893
1894 <emphasis role="bold">=============== The following 2 rules are generated by the class definition in /etc/shorewall/classes ==================</emphasis>
1895
1896 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:800 order 2048 key ht 800 bkt 0 <emphasis
1897 role="bold">flowid 1:110</emphasis> (rule hit 2204 success 138)
1898 match 00060000/00ff0000 at 8 (success 396 ) <emphasis
1899 role="bold"><========= TCP </emphasis>
1900 match 05000000/0f00ffc0 at 0 (success 250 ) <emphasis
1901 role="bold"><========= Header length 20 and Packet Length < 64</emphasis>
1902 match 00100000/00ff0000 at 32 (success 138 ) <emphasis
1903 role="bold"><========= ACK</emphasis>
1904
1905 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:801 order 2049 key ht 800 bkt 0 <emphasis
1906 role="bold">flowid 1:110</emphasis> (rule hit 2066 success 0)
1907 match 00100000/00100000 at 0 (success 0 ) <emphasis
1908 role="bold"><========= Minimize-delay</emphasis><emphasis
1909 role="bold"> goes to class 1:110</emphasis>
1910
1911 <emphasis role="bold"> =============== Jump to Table 1 if the matches are met ==================</emphasis>
1912
1913 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:802 order 2050 key ht 800 bkt 0 <emphasis
1914 role="bold">link 1:</emphasis> (rule hit 2066 success 0)
1915 match ce7c92b2/ffffffff at 12 (success 1039 ) <emphasis
1916 role="bold"><========= SOURCE 206.124.146.178 </emphasis>
1917 match 00060000/00ff0000 at 8 (success 0 ) <emphasis
1918 role="bold"><========= PROTO TCP</emphasis>
1919 offset 0f00>>6 at 0 eat
1920
1921 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:803 order 2051 key ht 800 bkt 0 <emphasis
1922 role="bold">flowid 1:110</emphasis> (rule hit 2066 success 1039)
1923 match ce7c92b2/ffffffff at 12 (success 1039 ) <emphasis
1924 role="bold"><========= SOURCE 206.124.146.178 goes to class 1:110</emphasis>
1925
1926 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:804 order 2052 key ht 800 bkt 0 <emphasis
1927 role="bold">flowid 1:110</emphasis> (rule hit 1027 success 132)
1928 match ce7c92b3/ffffffff at 12 (success 132 ) <emphasis
1929 role="bold"> <========= SOURCE 206.124.146.179 goes to class 1:110</emphasis>
1930
1931 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:805 order 2053 key ht 800 bkt 0 <emphasis
1932 role="bold">flowid 1:110</emphasis> (rule hit 895 success 603)
1933 match ce7c92b4/ffffffff at 12 (success 603 ) <emphasis
1934 role="bold"><========= SOURCE 206.124.146.180 goes to class 1:110</emphasis>
1935
1936 <emphasis role="bold"> =============== Jump to Table 2 if the matches are met ==================</emphasis>
1937
1938 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:806 order 2054 key ht 800 bkt 0 <emphasis
1939 role="bold">link 2:</emphasis> (rule hit 292 success 0)
1940 match 00010000/00ff0000 at 8 (success 0 ) <emphasis
1941 role="bold"><========= PROTO ICMP</emphasis>
1942 offset 0f00>>6 at 0 eat
1943
1944 <emphasis role="bold"> =============== Jump to Table 3 if the matches are met ==================</emphasis>
1945
1946 filter parent 1: protocol ip pref 10 u32 <emphasis role="bold">fh 800:</emphasis>:807 order 2055 key ht 800 bkt 0 <emphasis
1947 role="bold">link 3:</emphasis> (rule hit 292 success 0)
1948 match ce7c92b1/ffffffff at 12 (success 265 ) <emphasis
1949 role="bold"><========= SOURCE 206.124.146.177</emphasis>
1950 match 00060000/00ff0000 at 8 (success 102 ) <emphasis
1951 role="bold"><========= PROTO TCP</emphasis>
1952 offset 0f00>>6 at 0 eat </programlisting></para>
1953 </section>
1954
1955 <section>
1956 <title>IFBs and SNAT/MASQUERADE</title>
1957
1958 <para>IFB traffic shaping takes place immediately after the traffic is
1959 received by the incoming interface and before it has been passed to any
1960 Netfilter hook. This has two consequences:</para>
1961
1962 <itemizedlist>
1963 <listitem>
1964 <para>There is no opportunity to mark the packets before they are
1965 processed by the IFBs traffic shaping rules.</para>
1966 </listitem>
1967
1968 <listitem>
1969 <para>The DEST IP address is still the IP address of the external
1970 interface on which the traffic arrived.</para>
1971 </listitem>
1972 </itemizedlist>
1973
1974 <para>As a result, in the tcdevices file description above, a <emphasis
1975 role="bold">connmark</emphasis> option was added to that file in
1976 Shorewall 5.2.7. The <emphasis role="bold">connmark</emphasis> option
1977 allows firewall marks to be used to segregate traffic by DEST IP.</para>
1978
1979 <para>Example (based closely on one supplied by Rodrigo Araujo, who also
1980 wrote much of the code supporting the <emphasis
1981 role="bold">connmark</emphasis> option):</para>
1982
1983 <para><emphasis
1984 role="bold">/etc/shorewall/shorewall.conf:</emphasis></para>
1985
1986 <programlisting>...
1987 TC_ENABLED=Internal
1988 ...</programlisting>
1989
1990 <para><emphasis role="bold">/etc/shorewall/interfaces:</emphasis></para>
1991
1992 <programlisting>##############################################################################
1993 ?FORMAT 2
1994 ###############################################################################
1995 #ZONE INTERFACE OPTIONS
1996 net NET_IF dhcp,tcpflags,nosmurfs,routefilter,logmartians,sourceroute=0,physical=eth0
1997 loc LOC_IF tcpflags,nosmurfs,routefilter,logmartians,physical=eth1</programlisting>
1998
1999 <para><emphasis role="bold">/etc/shorewall/snat:</emphasis></para>
2000
2001 <programlisting>?FORMAT 2
2002 #ACTION SOURCE DEST PROTO PORT IPSEC MARK
2003 USER SWITCH ORIGDEST PROBABILITY
2004 MASQUERADE - NET_IF</programlisting>
2005
2006 <para><emphasis role="bold">/etc/shorewall/tcdevices:</emphasis></para>
2007
2008 <programlisting>#INTERFACE IN_BANDWITH OUT_BANDWIDTH OPTIONS REDIRECT
2009 ## net upload
2010 10:NET_IF - 1000mbit htb
2011 ## net download
2012 11:ifb0 - 1000mbit htb,connmark NET_IF</programlisting>
2013
2014 <para><emphasis role="bold">/etc/shorewall/tcclasses:</emphasis></para>
2015
2016 <programlisting>#INTERFACE MARK RATE CEIL PRIO OPTIONS
2017 10:5000 111 500kbit full 10 tcp-ack,tos-minimize-delay
2018 11:5000 110 500kbit full 10 tcp-ack,tos-minimize-delay
2019
2020 10:1000 100 full-50500 full 20 default
2021 11:1000 101 full-100500 full 20 default
2022
2023 10:50 10 50mbit 50mbit 101 flow=nfct-src
2024 11:100 11 100mbit 100mbit 101 flow=dst</programlisting>
2025
2026 <para><emphasis role="bold">/etc/shorewall/tcfilters:</emphasis></para>
2027
2028 <programlisting>#CLASS SOURCE DEST PROTO DPORT SPORT TOS LENGTH
2029 ## limit LAN upload - works
2030 10:50 10.100.100.0/24
2031 ## limit LAN download - DOESN'T WORK BECAUSE OF MASQUERADE ON eth0 !!!! (snat file)
2032 #11:100 - 10.100.100.0/24</programlisting>
2033
2034 <para><emphasis role="bold">/etc/shorewall/mangle:</emphasis></para>
2035
2036 <programlisting>#ACTION SOURCE DEST PROTO DPORT SPORT USER TEST LENGTH TOS CONNBYTES HELPER PROBABILITY DSCP SWITCH
2037 ## this only works with the aforementioned conntrack option
2038 ## and LAN users' download traffic will get the 11:100 class (defined in tcclasses) applied
2039 CONNMARK(11):F 10.100.100.0/24 - { TEST=0x0/0xff }</programlisting>
2040 </section>
2041 </section>
2042
2043 <section id="show">
2044 <title>Understanding the output of 'shorewall show tc'</title>
2045
2046 <para>The <command>shorewall show tc</command> (<command>shorewall-lite
2047 show tc</command>) command displays information about the current state of
2048 traffic shaping. For each device, it executes the following
2049 commands:</para>
2050
2051 <programlisting> echo Device $device:
2052 tc -s -d qdisc show dev $device
2053 echo
2054 tc -s -d class show dev $device
2055 echo </programlisting>
2056
2057 <para>So, the traffic-shaping output is generated entirely by the
2058 <command>tc</command> utility.</para>
2059
2060 <para>Here's the output of for eth0. The configuration is the one shown in
2061 the preceding section (the output was obtained almost 24 hours later than
2062 the <command>shorewall show filters</command> output shown above).</para>
2063
2064 <programlisting>Device eth0:
2065
2066 <emphasis role="bold"> ============== The primary queuing discipline is HTB (Hierarchical Token Bucket) ==================== </emphasis>
2067
2068 qdisc htb 1: r2q 10 default 120 direct_packets_stat 9 ver 3.17
2069 Sent 2133336743 bytes 4484781 pkt (dropped 198, overlimits 4911403 requeues 21) <emphasis
2070 role="bold"><=========== Note the overlimits and dropped counts</emphasis>
2071 rate 0bit 0pps backlog 0b 8p requeues 21
2072
2073 <emphasis role="bold">============== The ingress filter. If you specify IN-BANDWIDTH, you can see the 'dropped' count here. =========</emphasis>
2074
2075 <emphasis role="bold">In this case, the packets are being sent to the IFB for shaping</emphasis>
2076
2077 qdisc ingress ffff: ----------------
2078 Sent 4069015112 bytes 4997252 pkt (dropped 0, overlimits 0 requeues 0)
2079 rate 0bit 0pps backlog 0b 0p requeues 0
2080
2081 <emphasis role="bold">============ Each of the leaf HTB classes has an SFQ qdisc to ensure that each flow gets its turn ============</emphasis>
2082
2083 qdisc sfq 110: parent 1:110 limit 128p quantum 1514b flows 128/1024 perturb 10sec
2084 Sent 613372519 bytes 2870225 pkt (dropped 0, overlimits 0 requeues 6)
2085 rate 0bit 0pps backlog 0b 0p requeues 6
2086 qdisc sfq 120: parent 1:120 limit 128p quantum 1514b flows 128/1024 perturb 10sec
2087 Sent 18434920 bytes 60961 pkt (dropped 0, overlimits 0 requeues 0)
2088 rate 0bit 0pps backlog 0b 0p requeues 0
2089 qdisc sfq 130: parent 1:130 limit 128p quantum 1514b flows 128/1024 perturb 10sec
2090 Sent 1501528722 bytes 1553586 pkt (dropped 198, overlimits 0 requeues 15)
2091 rate 0bit 0pps backlog 11706b 8p requeues 15
2092
2093 <emphasis role="bold">============= Class 1:110 -- the high-priority class ===========
2094
2095
2096 Note the rate and ceiling calculated from 'full'</emphasis>
2097
2098 class htb <emphasis role="bold">1:110</emphasis> parent 1:1 leaf 110: prio 1 quantum 4800 <emphasis
2099 role="bold">rate 192000bit ceil 384000bit</emphasis> burst 1695b/8 mpu 0b overhead 0b cburst 1791b/8 mpu 0b overhead 0b level 0
2100 Sent 613372519 bytes 2870225 pkt (dropped 0, overlimits 0 requeues 0)
2101 <emphasis role="bold">rate 195672bit 28pps backlog 0b 0p</emphasis> requeues 0 <emphasis
2102 role="bold"><=========== Note the current rate of traffic. There is no queuing (no packet backlog)</emphasis>
2103 lended: 2758458 borrowed: 111773 giants:
2104 tokens: 46263 ctokens: 35157
2105
2106 <emphasis role="bold">============== The root class ============</emphasis>
2107
2108 class htb <emphasis role="bold">1:1 root</emphasis> rate 384000bit ceil 384000bit burst 1791b/8 mpu 0b overhead 0b cburst 1791b/8 mpu 0b overhead 0b level 7
2109 <emphasis role="bold">Sent 2133276316 bytes 4484785 pkt</emphasis> (dropped 0, overlimits 0 requeues 0) <emphasis
2110 role="bold"><==== Total output traffic since last 'restart'</emphasis>
2111 rate 363240bit 45pps backlog 0b 0p requeues 0
2112 lended: 1081936 borrowed: 0 giants: 0
2113 tokens: -52226 ctokens: -52226
2114
2115 <emphasis role="bold">============= Bulk Class (outgoing rsync, email and bittorrent) ============</emphasis>
2116
2117 class htb 1:130 parent 1:1 leaf 130: prio 3 quantum 1900 rate 76000bit ceil 230000bit burst 1637b/8 mpu 0b overhead 0b cburst 1714b/8 mpu 0b overhead 0b level 0
2118 Sent 1501528722 bytes 1553586 pkt (dropped 198, overlimits 0 requeues 0)
2119 <emphasis role="bold">rate 162528bit 14pps backlog 0b 8p</emphasis> requeues 0 <emphasis
2120 role="bold"><======== Queuing is occurring (8 packet backlog). The rate is still below the ceiling.</emphasis>
2121 lended: 587134 borrowed: 966459 giants: 0 <emphasis role="bold">During peak activity, the rate tops out at around 231000 (just above ceiling).</emphasis>
2122 tokens: -30919 ctokens: -97657
2123
2124 <emphasis role="bold">================= Default class (mostly serving web pages) ===============</emphasis>
2125
2126 class htb 1:120 parent 1:1 leaf 120: prio 2 quantum 1900 rate 76000bit ceil 230000bit burst 1637b/8 mpu 0b overhead 0b cburst 1714b/8 mpu 0b overhead 0b level 0
2127 Sent 18434920 bytes 60961 pkt (dropped 0, overlimits 0 requeues 0)
2128 rate 2240bit 2pps backlog 0b 0p requeues 0
2129 lended: 57257 borrowed: 3704 giants: 0
2130 tokens: 156045 ctokens: 54178
2131
2132 </programlisting>
2133 </section>
2134
2135 <section id="External">
2136 <title id="tcstart">Using your own tc script</title>
2137
2138 <section id="owntcstart">
2139 <title>Replacing builtin tcstart file</title>
2140
2141 <para>If you prefer your own tcstart file, just install it in
2142 /etc/shorewall/tcstart.</para>
2143
2144 <para>In your tcstart script, when you want to run the <quote>tc</quote>
2145 utility, use the run_tc function supplied by Shorewall if you want tc
2146 errors to stop the firewall.</para>
2147
2148 <orderedlist>
2149 <listitem>
2150 <para>Set TC_ENABLED=Yes and CLEAR_TC=Yes</para>
2151 </listitem>
2152
2153 <listitem>
2154 <para>Supply an /etc/shorewall/tcstart script to configure your
2155 traffic shaping rules.</para>
2156 </listitem>
2157
2158 <listitem>
2159 <para>Optionally supply an /etc/shorewall/tcclear script to stop
2160 traffic shaping. That is usually unnecessary.</para>
2161 </listitem>
2162
2163 <listitem>
2164 <para>If your tcstart script uses the <quote>fwmark</quote>
2165 classifier, you can mark packets using entries in
2166 /etc/shorewall/mangle or /etc/shorewall/tcrules.</para>
2167 </listitem>
2168 </orderedlist>
2169 </section>
2170
2171 <section id="Start">
2172 <title>Traffic control outside Shorewall</title>
2173
2174 <para>To start traffic shaping when you bring up your network
2175 interfaces, you will have to arrange for your traffic shaping
2176 configuration script to be run at that time. How you do that is
2177 distribution dependent and will not be covered here. You then
2178 should:</para>
2179
2180 <orderedlist>
2181 <listitem>
2182 <para>Set TC_ENABLED=No and CLEAR_TC=No</para>
2183 </listitem>
2184
2185 <listitem>
2186 <para>If your script uses the <quote>fwmark</quote> classifier, you
2187 can mark packets using entries in /etc/shorewall/mangle or
2188 /etc/shorewall/tcrules.</para>
2189 </listitem>
2190 </orderedlist>
2191 </section>
2192 </section>
2193
2194 <section id="Testing">
2195 <title>Testing Tools</title>
2196
2197 <para>At least one Shorewall user has found this tool helpful: <ulink
2198 url="http://e2epi.internet2.edu/network-performance-toolkit.html">http://e2epi.internet2.edu/network-performance-toolkit.html</ulink></para>
2199 </section>
2200 </article>