"Fossies" - the Fresh Open Source Software Archive

Member "mod_throttle-3.1.2/index.shtml" (3 Dec 2000, 26492 Bytes) of package /linux/www/apache_httpd_modules/old/mod_throttle312.tgz:


Caution: In this restricted "Fossies" environment the current HTML page may not be correctly presentated and may have some non-functional links. You can here alternatively try to browse the pure source code or just view or download the uninterpreted raw source code. If the rendering is insufficient you may try to find and view the page on the mod_throttle312.tgz project site itself.

Snert . Com mod_throttle/3.1.2
      from   Snert . Com

Bandwidth & Request Throttling for Apache 1.3

| Description | Download | Installation | Configuration | Notes | Changes |
| License | Disclaimer | Credits |
| Welcome | Software |

Description

This Apache module is intended to reduce the load on your server & bandwidth generated by popular virtual hosts, directories, locations, or users according to supported polices (see below) that decide when to delay or refuse requests. Also mod_throttle can track and throttle incoming connections by IP address or by authenticated remote user.

Every request now passes through four levels of throttling, which are: by client's IP address (ThrottleClientIP), by authenticated remote user name (ThrottleRemoteUser), by local user ID (ThrottleUser), and by directory, location, virtual host, or server (ThrottlePolicy).

In the list below, is a detailed description of each policy. Every policy expects two parameters: a limit and a time period.

None
This policy imposes no restrictions on a request and used as a place holder to allow monitoring. The limit currently serves no purpose. The period specifies how long data is accumulated before the counters are reset.

Concurrent
Impose a limit on the number of concurrent requests at any one time. The period specifies how long data is accumulated before the counters are reset. This policy cannot be used with either ThrottleClientIP or ThrottleRemoteUser.

Document
Excluding requests for HTML page elements such as images and style sheets, impose a limit on the number of requests per period. When this limit is exceeded all further requests are refused, until the elapsed time exceeds the period length, at which point the elapsed time and the counters are reset. Note that the requests (hits) column of the throttle status display does not include the requests for page elements.

Idle
Impose a mimimum idle time between requests. When the miminum is not reached, then the request incurs a calculated delay penalty or is refused.

First, whenever the elapsed time exceeds the period length, then the counters are reset.

Second, if the idle time between requests exceeds the minimum, then the the request proceeds without delay. Otherwise the request is delayed between one and ThrottleMaxDelay seconds. If the delay would exceed ThrottleMaxDelay, then we refuse the request entirely to avoid occupying servers unnecessarily.

The delay is computed as the policy minimum less the idle time between requests.

Original
Original mod_throttle 2.0 heuristic.

Impose a limit on the volume (kbytes sent) per period, which when exceeded the request incurs a counter-based delay penalty or is refused.

First, whenever the elapsed time exceeds the period length, then the volume and elapsed time are halved.

Second, if the volume is below the limit, then the delay counter is decreased by one second if it is not yet zero. Otherwise, when the limit is exeeded, the delay counter is increased by one second. The delay can be between zero and ThrottleMaxDelay seconds, after which the request will be refused to avoid occupying servers unnecessarily.

Random
Randomly accept a percentage (limit) of the requests. If the percentage is zero (0), then every request is refused; if the percentage is 100, then all requests are accepted. The period specifies how long data is accumulated before the counters are reset.

Request
Impose a limit on the number of requests per period. When this limit is exceeded all further requests are refused until the elapsed time exceeds the period length, at which point the elapsed time and counters are reset.

Speed
Impose a limit on the volume (kbytes sent) per period, which when exceeded the request incurs a calculated delay penalty or is refused.

First, whenever the elapsed time exceeds the period length, then the limit (allowance) is deducted from the volume, which cannot be a negative result; also the period length is deducted from the elapse time.

Second, if the volume is below the limit, in which case the request proceeds without delay. Otherwise the request is delayed between one and ThrottleMaxDelay seconds. If the delay would exceed ThrottleMaxDelay, then we refuse the request entirely to avoid occupying servers unnecessarily.

The delay is computed as one plus the integer result of the volume times 10 divided by the limit.

Volume
Impose a limit on the volume (kbytes sent) per period. When this limit is exceeded all further requests are refused, until the end of the period at which point the elapsed time and counters are reset.

Installation

  1. Download mod_throttle312.tgz and unpack the archive. (The previous series mod_throttle/2.11 can also be downloaded.)

  2. Depending on your operating system, you must edit the #undef and #define found at the top of mod_throttle.c source file to specify the semaphore & shared memory API to use.

    For the semaphore support you must #define only one of the following and #undef the rest. If you do not #define one of these, then mod_throttle will operate without semaphore support, which will have undefined results:

    USE_POSIX_SERIALIZATION
    USE_FCNTL_SERIALIZATION
    USE_FLOCK_SERIALIZATION
    USE_SYSTEM_V_SERIALIZATION
    

    For the shared memory support you must #define only one of the following and #undef the rest. If you do not #define one of these, then mod_throttle will operate without shared memory, which will have undefined results:

    USE_POSIX_SHARED_MEMORY
    USE_SYSTEM_V_SHARED_MEMORY
    

  3. To build as DSO module, just type:

    cd (path to)/mod_throttle-3.1
    make install
    

    To build as a static module:

    cd /usr/local/apache_1.3.14	(or where ever you keep the Apache source)
    ./configure \			(see below in the Notes for additional information)
        --disable-shared=throttle \
        --add-module=(path to)/mod_throttle-3.1/mod_throttle.c \
        ... (your other configuration options) ...
    make install
    

  4. Configure the httpd.conf file. At the very least add following:

    <IfModule mod_throttle.c>
    	ThrottlePolicy none
    
    	<Location /throttle-status>
    	SetHandler throttle-status
    	</Location>
    
    	<Location /throttle-me>
    	SetHandler throttle-me
    	</Location>
    
    	<Location /~*/throttle-me>
    	SetHandler throttle-me
    	</Location>
    </IfModule>
    

  5. Restart the server: apachectl stop; apachectl start

  6. View the status display http://www.mydomain.com/throttle-status.

Configuration

The commands below can be added to the general Apache configuration file, httpd.conf.

SetHandler throttle-client-ip
Context: server, <VirtualHost>, <Directory>, <Location>

Displays the complete list of recently connected client IP addresses. There are command links to reset all the clients, or reset individual clients.

Each row shows a client IP address, the percentage of the limit reached, the total number of requests (hits) made by this client, number of requests refused to the this client, volume (kbytes sent) per period, average volume per request, the current delay, the elapsed period time, and the time idle since the last request.

SetHandler throttle-remote-user
Context: server, <VirtualHost>, <Directory>, <Location>

Displays the complete list of recently authenticated remote users. There are command links to reset all or individual remote users.

Each row shows the authenticated remote user name, the percentage of the limit reached, the total number of requests (hits) made by this remote user, number of requests refused to the this remote user, volume (kbytes sent) per period, average volume per request, the current delay, the elapsed period time, and the time idle since the last request.

SetHandler throttle-status
Context: server, <VirtualHost>, <Directory>, <Location>

Displays the complete list of server, virtual hosts, directories, locations, and local users who are to be throttled. There are command links to reset all or individual entries, and preserve & restore runtime data.

Each row shows the item being throttled, the percentage of the limit reached, the total number of requests (hits) received, number of requests refused, volume (kbytes sent) per period, average volume per request, the current delay, policy, limit, period, elapsed period time, and the time idle since the last request.

http://my.domain.com/throttle-status
SetHandler throttle-me
Context: server, <VirtualHost>, <Directory>, <Location>

Similar to SetHandler throttle-status in content, except it only displays one individual entry for either a ~user or the server of which the request was made. The intent of this handler is to provide the system administrator a means by which customers can monitor their throttles without seeing those belonging to others.

Assuming the handler has been installed as outlined above, then a URL of the form:

http://my.domain.com/~user/throttle-me

will display the throttle status for user, if a ThrottleUser was specified for the user in question; otherwise a Not Found error page. A URL of the form:

http://my.domain.com/throttle-me

will display the throttle status of my.domain.com, if it has a ThrottlePolicy; otherwise a Not Found error page.

ThrottleClientIP size policy limit period
Context: server

Specify the size of the client IP address list and the global policy to be applied to all incoming connections based on information recorded for a client IP address. The policy Concurrent is ignored.

When the list size is greater than zero, then throttling by client IP address is enabled (the default is zero). Requests from a client IP address can be tracked for a short period of time and subject to the policy specified. The period of time that a client IP address is tracked depends on the size of the client IP address list, which is ordered most recent request to oldest. Every time a new client IP address connects, the oldest entry in the list is lost and reassigned to the new client IP address. Every time an existing entry makes a request, it is moved to the top of the list.

ThrottleContentType string
Context: server

Specifies the default Content-Type to be used for the throttle status display. Only two supported types: text/html, text/plain. text/html is a nicely formatted and coloured table. text/plain is a tab separated list of columns as seen for text/html, without headers and footers. When not specified, the default is text/html. This can be overridden by specifying the content-type in the URL, for example:

http://my.domain.com/throttle-status?content-type=text/plain
ThrottleIndicator green percentage
ThrottleIndicator yellow percentage
ThrottleIndicator red percentage
Context: server

Specify the default thresholds used for the visual warnings by the throttle status display. The default indicators are green 50, yellow 75, and red 90.

ThrottleLockFile string
Context: server

The lock file used with fcntl() or flock() serialization. Must be stored on a local disk.

ThrottleMaxDelay seconds
Context: server

This directive affects all policies that use delays. It imposes an upper limit on the throttle delay that can be applied by a policy. The default is 60 seconds. A value of zero (0) disables the limit, allowing a delay to be as high as the policy wants.

ThrottlePolicy policy limit period
Context: server, <VirtualHost>, <Directory>, <Location>

The policy parameter can be one of the previously above mentioned policies.

The limit is a number followed by an optional suffix K, M, or G for kilo-bytes, mega-bytes, or giga-bytes respectively. When no suffix is given the default is kilo-bytes.

The period is a number followed by an optional suffix s, m, h, d, or w for seconds, minutes, hours, days, or weeks respectively. When no suffix is given the default is seconds.

ThrottleRefresh seconds
Context: server

Specifies the refresh time for the throttle status display. When not specified, the default is 60 seconds. This can be overridden by specifying the refresh time in the URL as well, for example:

http://www.domain.com/throttle-status?refresh=20
ThrottleRemoteUser size policy limit period
Context: server

Specify the size of the authenticated remote user list and the global policy to be applied to all requests based on information recorded for an authenticated remote user. The policy Concurrent is ignored.

When the list size is greater than zero, then throttling by authenticated remote user is enabled (the default is zero). When the web server restricts access to a resource, authentication is required to be sent by the remote user. The user name supplied for that authentication is then used to control short term usage by the policy specified. The period of time that a remote user can be tracked depends on the size of the remote user list, which is ordered most recent request to oldest. Every time a new remote user connects, the oldest entry in the list is lost and reassigned to the new remote user. Every time an existing entry makes a request, it is moved to the top of the list.

ThrottleRuntimeFile filename
Context: server

The location and file name of the runtime data file used to preserve state information across shutdowns & restarts.

ThrottleUser user policy limit period
Context: server

Each request for files belonging to the given user name or #ID are throttled according to the selected policy as described above. If the user is an asterisk (*), then all the users from the system user databases are throttled with the same policy, limit, and period. If the user is an absolute path to an htpasswd file, /etc/passwd, or a file with one user per line, then all the users from that file are throttled with the same policy, limit, and period. User names with no corresponding system user ID are ignored.

Notes

  • mod_throttle/3.0 tested on following platforms (please email me your platform if you've made it work on something different):

    FreeBSD 4.1.1, Apache 1.3.14
    The USE_SYSTEM_V_SERIALIZATION, and USE_SYSTEM_V_SHARED_MEMORY code was successfully tested on this platform.

    The USE_POSIX_SERIALIZATION and USE_POSIX_SHARED_MEMORY compiles and links (with & without -pthread), but fails on sem_init() when execute - help anyone?.

    RedHat Linux 5.1+ (MIPS & Intel), Apache 1.3.14
    The USE_FCNTL_SERIALIZATION, USE_FLOCK_SERIALIZATION, USE_SYSTEM_V_SERIALIZATION, and USE_SYSTEM_V_SHARED_MEMORY code was successfully tested on this platform.


    Solaris 5.7, Apache 1.3.12
    To build as a DSO module, Apache must be built with --enable-rule=SHARED_CORE. The USE_POSIX_SERIALIZATION and USE_POSIX_SHARED_MEMORY code was successfully tested on this platform, but must be linked with the library "-lrt"; when compiling DSO, edit the Makefile and uncomment the LIB= macro; when compiling statically, specify: LIBS='-lrt' ./configure ...

  • The context diff "patch.proxy_util.c" should be applied to /src/modules/proxy/proxy_util.c from the Apache 1.3.14 distribution. This patch allows mod_throttle/3.0 to monitor proxy requests when mod_proxy is in use. (This patch has been submitted to the Apache Server project.)

  • The policies Original, Speed, and Volume all impose limits on volume (kilobytes sent). The differences lie in how adjustments are made each period and whether to delay or refuse the conection.

  • Here is a shell command that can be used for a cron job to restart the Apache server on a set schedule in order to reload your configuration file. This assumes that you do not change the location of the "PidFile" in your configuration and that httpd is on root's PATH.

    kill -USR1 $(cat $(httpd -V | sed -n '/DEFAULT_PIDLOG/s/.*"\(.*\)"/\1/p'))
    

License

This source distribution is made freely available and there is no charge for its use, provided you retain this notice, disclaimers, author's copyright, and credits.

Disclaimer

THIS SOFTWARE IS PROVIDE "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO WAY SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Credits

Original design for mod_throttle/1.0 goes to Mark Lovell <mlovell@bigrock.com>.

Elements of the critical & shared memory code, as of mod_throttle/3.0, originally derived from the Apache Web Server source code.

Thank You to Lu Vo <LVo@SUPERB.NET> for providing a Solaris POSIX compliant machine to test on; to Travis Doherty <travis@terradelta.net> for a FreeBSD machine for testing and several suggestions; to David M. Shirley <david@webquarry.com> for constructive and helpful reports concerning design and testing.


| Description | Download | Installation | Configuration | Notes | Changes |
| License | Disclaimer | Credits |
| Welcome | Software |

Copyright 1999, 2000 by Anthony C Howe. All rights reserved.
fluffy bunnies since 15 Nov. 2000