"Fossies" - the Fresh Open Source Software Archive

Member "incubator-pagespeed-mod-1.14.36.1/html/doc/config_filters.html" (28 Feb 2020, 40234 Bytes) of package /linux/www/apache_httpd_modules/incubator-pagespeed-mod-1.14.36.1.tar.gz:


Caution: In this restricted "Fossies" environment the current HTML page may not be correctly presentated and may have some non-functional links. You can here alternatively try to browse the pure source code or just view or download the uninterpreted raw source code. If the rendering is insufficient you may try to find and view the page on the incubator-pagespeed-mod-1.14.36.1.tar.gz project site itself.

Configuring PageSpeed Filters

Rewriting Level

PageSpeed offers three "levels" to simplify configuration: PassThrough, CoreFilters, and OptimizeForBandwidth. The CoreFilters set contains filters that the PageSpeed team believes are safe for most web sites. By using the CoreFilters set, as PageSpeed is updated with new filters, your site will get faster. The OptimizeForBandwidth setting provides a stronger guarantee of safety and is suitable as a default setting for use with sites that are not aware of PageSpeed.

To disable the CoreFilters, you can specify

Apache:
ModPagespeedRewriteLevel PassThrough
Nginx:
pagespeed RewriteLevel PassThrough;

and then enable specific filters with the EnableFilters directive. The default level is CoreFilters. The core set of filters contains:

   add_head
   combine_css
   combine_javascript
   convert_meta_tags
   extend_cache
   fallback_rewrite_css_urls
   flatten_css_imports
   inline_css
   inline_import_to_link
   inline_javascript
   rewrite_css
   rewrite_images
   rewrite_javascript
   rewrite_style_attributes_with_url

Enabling, Disabling, And Forbidding Specific Filters

To turn off specific filters in the core set, specify:

Apache:
ModPagespeedDisableFilters filtera,filterb
Nginx:
pagespeed DisableFilters filtera,filterb;

For example, if you want to use the core set of filters, but specifically disable rewrite_images and combine_css, you can use:

Apache:
ModPagespeedDisableFilters rewrite_images,combine_css
Nginx:
pagespeed DisableFilters rewrite_images,combine_css;

To turn off specific filters and forbid them from being turned on by query parameters, request headers, or in a location-specific configuration section, specify (for example):

Apache:
ModPagespeedForbidFilters rewrite_css,rewrite_javascript
Nginx:
pagespeed ForbidFilters rewrite_css,rewrite_javascript;

You can use any number of the DisableFilters and/or ForbidFilters directives, each of which can contain multiple filter names separated by commas.

The EnableFilters configuration file directive allows specification of one or more filters by name, separated by commas. You can use any number of EnableFilters directives, each of which can contain multiple filter names separated by commas. For example:

Apache:
ModPagespeedRewriteLevel PassThrough
ModPagespeedEnableFilters combine_css,extend_cache,rewrite_images
ModPagespeedEnableFilters rewrite_css,rewrite_javascript
Nginx:
pagespeed RewriteLevel PassThrough;
pagespeed EnableFilters combine_css,extend_cache,rewrite_images;
pagespeed EnableFilters rewrite_css,rewrite_javascript;

The order of the directives in the configuration file is not important. the rewriters are run in the pre-defined order presented in the table:

.
Filter Name In CoreFilters In OptimizeForBandwidth Brief Description
responsive_images NoNoMakes images responsive by adding srcset with images optimized for various resolutions.
add_head YesNo Adds a <head> element to the document if not already present.
combine_heads NoNo Combines multiple <head> elements found in document into one.
inline_import_to_link YesNo Inlines <style> tags comprising only CSS @imports by converting them to equivalent <link> tags.
outline_css NoNoExternalize large blocks of CSS into a cacheable file.
outline_javascript NoNoExternalize large blocks of JS into a cacheable file.
move_css_above_scripts NoNo Moves CSS elements above <script> tags.
move_css_to_head NoNoMoves CSS elements into the <head>.
combine_css YesNoCombines multiple CSS elements into one.
rewrite_css YesYes Rewrites CSS files to remove excess whitespace and comments, and, if enabled, rewrite or cache-extend images referenced in CSS files. In OptimizeForBandwidth mode, the minification occurs in-place without changing URLs.
fallback_rewrite_css_urls YesNo Rewrites resources referenced in any CSS file that cannot otherwise be parsed and minified.
rewrite_style_attributes NoNo Rewrite the CSS in style attributes by applying the configured rewrite_css filter to it.
rewrite_style_attributes_with_url YesNo Rewrite the CSS in style attributes if it contains the text 'url(' by applying the configured rewrite_css filter to it
flatten_css_imports YesNoInline CSS by flattening all @import rules.
prioritize_critical_css NoNoReplace CSS tags with inline versions that include only the CSS used by the page.
make_google_analytics_async NoNoConvert synchronous use of Google Analytics API to asynchronous
make_show_ads_async NoNoConvert synchronous use of Google AdSense API to asynchronous
rewrite_javascript YesYes Rewrites JavaScript files to remove excess whitespace and comments. In OptimizeForBandwidth mode, the minification occurs in-place without changing URLs.
rewrite_javascript_external YesYes Implied by rewrite_javascript. Rewrites JavaScript external files to remove excess whitespace and comments. In OptimizeForBandwidth mode, the minification occurs in-place without changing URLs.
rewrite_javascript_inline YesYes Implied by rewrite_javascript. Rewrites inline JavaScript blocks to remove excess whitespace and comments.
include_js_source_maps NoNo Adds source maps to rewritten JavaScript files.
combine_javascript YesNoCombines multiple script elements into one.
canonicalize_javascript_libraries NoNoRedirects JavaScript libraries to a JavaScript hosting service.
inline_css YesNoInlines small CSS files into the HTML document.
inline_google_font_css NoNoInlines small CSS files used by fonts.googleapis.com into the HTML document.
inline_javascript YesNoInlines small JS files into the HTML document.
local_storage_cache NoNoCache inlined resources in HTML5 local storage.
insert_ga NoNoAdds the Google Analytics snippet to each HTML page.
rewrite_images YesNoOptimizes images, re-encoding them, removing excess pixels, and inlining small images. In OptimizeForBandwidth mode, the minification occurs in-place without changing URLs.
convert_jpeg_to_progressive YesYesConverts larger jpegs to progressive format. Implied by recompress images.
convert_png_to_jpeg YesYesConverts gif and png images into jpegs if they appear to be less sensitive to compression artifacts and lack alpha transparency. Implied by recompress images.
convert_jpeg_to_webp YesYes Producess lossy webp rather than jpeg images for browsers that support webp. Implied by recompress images.
convert_to_webp_animated NoNo Replaces animated gif images with webp images on browsers that support the format.
convert_to_webp_lossless YesNo Implied by rewrite_images. Replaces png and non-animated gif images with webp images on browsers that support the format.
insert_image_dimensions NoNo Adds width and height attributes to <img> tags that lack them.
inline_images YesNo Implied by rewrite_images. Replaces small images by data: urls.
recompress_images YesYes Implied by rewrite_images. Recompresses images, removing excess metadata and transforming gifs into pngs.
recompress_jpeg YesYes Implied by recompress_images. Recompresses jpegs, removing excess metadata.
recompress_png YesYes Implied by recompress_images. Recompresses pngs, removing excess metadata.
recompress_webp YesYes Implied by recompress_images. Recompresses webps, removing excess metadata.
convert_gif_to_png YesYes Implied by recompress_images. Optimizes gifs to pngs.
strip_image_color_profile YesYesImplied by recompress_images. Strips color profile info from images.
strip_image_meta_data YesYesImplied by recompress_images. Strips EXIF meta data from images.
jpeg_sampling YesYesImplied by recompress_images. Reduces the color sampling of jpeg images to 4:2:0.
resize_images YesNoImplied by rewrite_images. Resizes images when the corresponding <img> tag specifies a smaller width and height.
resize_rendered_image_dimensions YesNoImplied by rewrite_images. Resizes an image when the rendered dimensions of the image are smaller than the actual image.
inline_preview_images NoNo Uses inlined low-quality images as placeholders which will be replaced with original images once the web page is loaded.
resize_mobile_images NoNo Works just like inline_preview_images, but uses smaller placeholder images and only serves them to mobile browsers.
remove_comments NoNo Removes comments in HTML files (but not in inline JavaScript or CSS).
collapse_whitespace NoNo Removes excess whitespace in HTML files (avoiding <pre>, <script>, <style>, and <textarea>).
elide_attributes NoNo Removes attributes which are not significant according to the HTML spec.
extend_cache YesNo Extends cache lifetime of CSS, JS, and image resources that have not otherwise been optimized, by signing URLs with a content hash.
extend_cache_css YesNo Implied by extend_cache. Extends cache lifetime of otherwise unoptimized CSS resources by signing URLs with a content hash.
extend_cache_images YesNo Implied by extend_cache. Extends cache lifetime of otherwise unoptimized images by signing URLs with a content hash.
extend_cache_scripts YesNo Implied by extend_cache. Extends cache lifetime of otherwise unoptimized scripts by signing URLs with a content hash.
extend_cache_pdfs NoNo Extends cache lifetime of PDFs by signing URLs with a content hash.
sprite_images NoNo Combine background images in CSS files into one sprite.
rewrite_domains NoNo Rewrites the domains of resources not otherwise touched by PageSpeed, based on MapRewriteDomain and ShardDomain settings in the config file.
trim_urls NoNo Shortens URLs by making them relative to the base URL.
pedantic NoNo Add default types for <script> and <style> tags if the type attribute is not present and the page is not HTML5. The purpose of this filter is to help ensure that PageSpeed does not break HTML4 validation.
remove_quotes NoNo Removes quotes around HTML attributes that are not lexically required.
add_instrumentation NoNo Adds JavaScript to page to measure latency and send back to the server.
convert_meta_tags YesNo Adds a response header for each meta tag with an http-equiv attribute.
defer_javascript NoNo Defers the execution of JavaScript in HTML until page load complete.
dedup_inlined_images NoNo Replaces repeated inlined images with JavaScript that loads the image from the first occurence of the image.
lazyload_images NoNo Loads images when they become visible in the client viewport.
insert_dns_prefetch NoNo Inserts <link rel="dns-prefetch" href="//www.example.com"> tags to reduce DNS resolution time.
hint_preload_subresources NoNo Inserts Link:</example.css>; rel=preload headers to permit earlier fetching of important resources.
in_place_optimize_for_browser NoYes Perform browser-dependent in-place resource optimizations.

Forbidding All Disabled Filters

You can enable filters for a specific request using either query parameters or request headers, and you can enable filters in sub-directories using the EnableFilters directive.

In both cases you can enable filters that are disabled or not explicitly enabled in the configuration file, however there are situations where this is undesirable, such as when a filter has been expressly disabled because it breaks a page, or because a filter imposes too great a load on the server.

All disabled filters can be forced off with:

Apache:
ModPagespeedForbidAllDisabledFilters true
Nginx:
pagespeed ForbidAllDisabledFilters true;

Note that in this context disabled filters means all filters that are not enabled by the RewriteLevel or EnableFilters directives.

This directive can be used in location-specific configuration sections.

Checking Which Filters Are Enabled

If you want to see exactly which filters are enabled on a virtual host, you can do so by going to that host's admin or statistics page.

Tuning the Filters

Once the rewriters are selected, some of them may also be tuned. These parameters control the inlining and outlining thresholds of various resources.

CssFlattenMaxBytes                       102400 (was 2048 prior to 1.9.32.1)
CssImageInlineMaxBytes                        0
CssInlineMaxBytes                          2048
CssOutlineMinBytes                         3000
ImageInlineMaxBytes                        3072
ImageJpegNumProgressiveScans                 -1
ImageJpegNumProgressiveScansForSmallScreens  -1
ImageLimitOptimizedPercent                  100
ImageLimitResizeAreaPercent                 100
ImageRecompressionQuality                    85
ImageResolutionLimitBytes              32000000
JpegRecompressionQuality                     -1
JpegRecompressionQualityForSmallScreens      70
WebpRecompressionQuality                     80
WebpAnimatedRecompressionQuality             70
WebpRecompressionQualityForSmallScreens      70
JsInlineMaxBytes                           2048
JsOutlineMinBytes                          3000
MaxInlinedPreviewImagesIndex                 -1
MinImageSizeLowResolutionBytes             3072
RetainComment                             "[WILDCARD PATTERN]"
RewriteRandomDropPercentage                   0

Note: The default settings are reasonable and intuitive, but have not been experimentally tuned.

These directives can be used in location-specific configuration sections.

Controlling the use of beacons

The lazyload_images, inline_preview_images, and inline_images filters, use a beacon to collect information about the rewritten page so as to optimize the rewriting process. The beacon is a POST request sent back by JavaScript inserted into the page by the filter. The use of this beacon is on by default but it can be disabled using:

Apache:
ModPagespeedCriticalImagesBeaconEnabled false
Nginx:
pagespeed CriticalImagesBeaconEnabled false;

If you disable image beacons but enable filters that use them, the filters will work but not as well as when beacons are enabled.

This directive can be used in all scopes including location-specific configuration sections.

Controlling beacon expiry

Note: New option as of 1.12.34.1

By default beacon data is considered valid for two hours, but if your site has a lot of pages that change rarely and get few hits you might want to raise this limit:

Apache:
ModPagespeedFinderPropertiesCacheExpirationTimeMs TtlInMs
Nginx:
pagespeed FinderPropertiesCacheExpirationTimeMs TtlInMs;

Preserving URLs in HTML

PageSpeed filters often modify the URLs of resources in HTML pages. This is generally harmless but it has the potential to break pages whose JavaScript expects to read or modify the URLs in the page.

image_preserve_urls, css_preserve_urls, and js_preserve_urls will suppress URL rewriting actions for the respective resource types. Those filters that require modifications to the URL are disabled by the preserve directives.

Note: Even though resource URLs are unchanged that does not mean that they cannot still be optimized. For instance, InPlaceResourceOptimization still works since it does not alter URLs. Turning on in place resource optimization is recommended when enabling any of the options to preserve URLs. In version 1.9.32.1 and later in place resource optimization is enabled by default.

Enabling image_preserve_urls will forbid the use of the following filters: inline_preview_images, lazyload_images, extend_cache_images, inline_images, and sprite_images.

Enabling css_preserve_urls will forbid the use of the following filters: combine_css, extend_cache_css, inline_css, inline_import_to_link, and outline_css.

Enabling js_preserve_urls will forbid the use of the following filters: canonicalize_javascript_libraries, combine_javascript, defer_javascript, extend_cache_javascript, inline_javascript, and outline_javascript.

Reducing Load by Randomly Dropping Expensive Rewrites

To reduce processing load, PageSpeed can be configured to optimize the most frequently fetched resources, leaving infrequently fetched resources alone. This is accomplished by randomly dropping expensive (CSS and image) rewrites. Frequently fetched resources will have a higher probability of being rewritten than infrequently fetched resources. Over time, frequently accessed resources will be optimized and cached so a page will be fully optimized. Infrequently accessed pages will be left unoptimized or partially optimized, saving CPU time and cache space.
Apache:
ModPagespeedRewriteRandomDropPercentage Percent
Nginx:
pagespeed RewriteRandomDropPercentage Percent;

This is a load-tuning parameter (integer between 0 and 100 inclusive) that controls the percentage of resource rewrites that are randomly dropped. Currently only CSS and image rewrites are randomly dropped, as they are the CPU intensive rewrite tasks. A value of 100 means all such rewrites are dropped and a value of 0 means no rewrites are dropped. A value of 75 means that 75% of image and CSS rewrites (selected at random) are dropped. Do not set this parameter to 100 in order prevent optimization of images and CSS files, it is more efficient to instead disable the image and/or CSS filters.

As an example, if the value is 90 then an image fetched only once will be optimized with 10% probability while an image fetched 50 times will be optimized with 99.65% probability (1 - 0.9^50). You may need to tune this parameter to find a value that provides the right load on your servers and still provides sufficient image and CSS optimization.

Note: Images within CSS files are not randomly dropped as this would lead to partially optimized CSS resources.

Configuring for Multiple Servers

When running PageSpeed on multiple servers, it is important that each have the same configuration file. This ensures that when a browser requests an image or other resource from one server, it will be optimized using the same options that were used to compute the optimized resource when HTML was served. It is helpful to use memcached to share cache between servers as it improves multi-server performance and scalability, but it is still important that the configurations are consistent to get the desired behavior when optimized images are evicted from cache.

Note also that location-specific configuration settings should be consistent between the HTML paths and the resource paths.

In some sites, the URL path layout or network deployment strategy may not allow for consistent configuration between HTML and images. PageSpeed offers a workaround for such sites by encoding relevant configuration settings for each rewritten resource into the URLs:

Apache:
ModPagespeedAddOptionsToUrls on
Nginx:
pagespeed AddOptionsToUrls on;

This adds an encoding of the options that are relevant to each rewritten resource to the URLs. While the produced URLs are larger, this provides a mechanism to propagate configuration without having to share a configuration file. For example, a site with image recompression on and JPEG compression set to 85 would see URLs like http://example.com/ximage.jpg.pagespeed.gp+jw+pj+rj+rp+rw+iq=85.ic.HASH.jpg. While it is better to have the extra configuration details in the configuration file, this option offers a fallback plan when that is not practical.

Custom Fetch Headers

When not using LoadFromFile, PageSpeed has to make HTTP requests for sub-resources of a >page in order to rewrite them. Consider the following HTML snippet:

    ...
    <body>
      <img src="example.jpg">
      ...

If the rewrite_images filter is enabled, PageSpeed needs to fetch example.jpg in order to inline, compress, or otherwise optimize it. If you would like custom headers to be sent with all sub-resource fetches like this one, you can use the CustomFetchHeader directive:

Apache:
ModPagespeedCustomFetchHeader CustomHeader CustomHeaderValue
ModPagespeedCustomFetchHeader AnotherCustomHeader AnotherValue
Nginx:
pagespeed CustomFetchHeader CustomHeader CustomHeaderValue;
pagespeed CustomFetchHeader AnotherCustomHeader AnotherValue;

Unsupported Filters

The PageSpeed code base contains a number of additional filters whose use is unsupported. Some of these are experimental; note that using experimental filters is likely to result in crashes or site breakage. Others are used for debugging specific problems with PageSpeed:

Debugging filter name Brief Description
add_base_tag Adds a <base> element to the beginning of the <head> that reflects the base url PageSpeed is using to resolve relative url references in the page.
debug Adds comments to the page describing actions by certain filters, and attempts to serve JavaScript injected by PageSpeed in source form rather than compiled and minified.
deterministic_js Attempts to provide deterministic JavaScript behavior on each page, for example by replacing the timer and random number generator with functions that return the same sequence of values on every page load.
strip_scripts Removes all script tags from the document.

Note: None of the above filters should be used to serve live traffic.

Remote Configuration

Note: New feature as of 1.10.33.0

PageSpeed filters and configurations can also be specified by an external "Remote Configuration" file. The remote configuration will override the main configuration file, override .htaccess configurations, and be overridden by any query-level parameters.

To specify the RemoteConfigurationUrl:

Apache:
ModPagespeedRemoteConfigurationUrl https://example.com/remote.conf
Nginx:
pagespeed RemoteConfigurationUrl https://example.com/remote.conf;

The syntax for the remote configuration file is similar to that of the .htaccess configurations with a few notable exceptions. Directives don't use ModPagespeed as a prefix. Comments are specified by # and must be on their own line. Filters and options with DirectoryScope or higher may be applied with the remote configuration. Any invalid lines in the remote configuration will be skipped, a warning will be logged, and the remaining lines will still be parsed. The remote configuration terminates with a line beginning with EndRemoteConfig, and any lines after this are ignored. If the configuration file does not contain a line beginning with EndRemoteConfig no configuration will be applied. An example configuration for enabling the remove_comments filter is as follows.

  # Enable the remove_comments filter.
  EnableFilter remove_comments
  EndRemoteConfig
  # Everything after the previous line will be ignored.

The remote configuration file will be fetched on the server's startup, and cached for the extent determined by the remote server's Cache-Control and Expires headers. For example, if the remote configuration hosting server provides the header Cache-Control: max-age=3600, the next fetch of the remote configuration will happen at the first request after 3600 seconds. Failed fetches after successful fetches will continue to serve the stale config. The remote configuration should be used in addition to your original configuration. The remote configuration is not guaranteed to be fetched and applied to every request, so the site should not rely on the remote configuration in order to work.

The timeout for the fetching the remote configuration file can also be specified. The default timeout is one second. To specify the fetching timeout.
Apache:
ModPagespeedRemoteConfigurationTimeoutMs 1500
Nginx:
pagespeed RemoteConfigurationTimeoutMs 1500;
Fetching the remote configuration will block for up to the specified timeout.

Note: Remote configurations can not be fetched from the same server that is running the instance of pagespeed.