"Fossies" - the Fresh Open Source Software Archive

Member "mariadb-10.4.10/pcre/doc/pcretest.txt" (7 Nov 2019, 54466 Bytes) of package /linux/misc/mariadb-10.4.10.tar.gz:

As a special service "Fossies" has tried to format the requested text file into HTML format (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 PCRETEST(1)                 General Commands Manual                PCRETEST(1)
    5 NAME
    6        pcretest - a program for testing Perl-compatible regular expressions.
   10        pcretest [options] [input file [output file]]
   12        pcretest  was written as a test program for the PCRE regular expression
   13        library itself, but it can also be used for experimenting with  regular
   14        expressions.  This document describes the features of the test program;
   15        for details of the regular expressions themselves, see the  pcrepattern
   16        documentation. For details of the PCRE library function calls and their
   17        options, see the pcreapi , pcre16 and pcre32 documentation.
   19        The input for pcretest is a sequence of regular expression patterns and
   20        strings  to be matched, as described below. The output shows the result
   21        of each match. Options on the command line  and  the  patterns  control
   22        PCRE options and exactly what is output.
   24        As  PCRE has evolved, it has acquired many different features, and as a
   25        result, pcretest now has rather a lot of obscure  options  for  testing
   26        every possible feature. Some of these options are specifically designed
   27        for use in conjunction with the test script and  data  files  that  are
   28        distributed  as  part of PCRE, and are unlikely to be of use otherwise.
   29        They are all documented here, but without much justification.
   34        Input to pcretest is processed line by line, either by  calling  the  C
   35        library's fgets() function, or via the libreadline library (see below).
   36        In Unix-like environments, fgets() treats any bytes other than  newline
   37        as  data characters. However, in some Windows environments character 26
   38        (hex 1A) causes an immediate end of file, and no further data is  read.
   39        For  maximum  portability,  therefore,  it  is safest to use only ASCII
   40        characters in pcretest input files.
   42        The input is processed using using C's string functions,  so  must  not
   43        contain  binary  zeroes, even though in Unix-like environments, fgets()
   44        treats any bytes other than newline as data characters.
   49        From release 8.30, two separate PCRE libraries can be built. The origi-
   50        nal  one  supports  8-bit  character  strings, whereas the newer 16-bit
   51        library supports  character  strings  encoded  in  16-bit  units.  From
   52        release  8.32,  a  third  library  can  be  built, supporting character
   53        strings encoded in 32-bit units. The pcretest program can  be  used  to
   54        test all three libraries. However, it is itself still an 8-bit program,
   55        reading 8-bit input and writing 8-bit output.  When testing the  16-bit
   56        or  32-bit  library, the patterns and data strings are converted to 16-
   57        or 32-bit format before being passed to  the  PCRE  library  functions.
   58        Results are converted to 8-bit for output.
   60        References to functions and structures of the form pcre[16|32]_xx below
   61        mean "pcre_xx when using the 8-bit library, pcre16_xx  when  using  the
   62        16-bit library, or pcre32_xx when using the 32-bit library".
   67        -8        If  both the 8-bit library has been built, this option causes
   68                  the 8-bit library to be used (which is the default);  if  the
   69                  8-bit  library  has  not  been  built,  this option causes an
   70                  error.
   72        -16       If both the 8-bit or the 32-bit,  and  the  16-bit  libraries
   73                  have  been built, this option causes the 16-bit library to be
   74                  used. If only the 16-bit library has been built, this is  the
   75                  default  (so  has no effect). If only the 8-bit or the 32-bit
   76                  library has been built, this option causes an error.
   78        -32       If both the 8-bit or the 16-bit,  and  the  32-bit  libraries
   79                  have  been built, this option causes the 32-bit library to be
   80                  used. If only the 32-bit library has been built, this is  the
   81                  default  (so  has no effect). If only the 8-bit or the 16-bit
   82                  library has been built, this option causes an error.
   84        -b        Behave as if each pattern has the /B (show byte  code)  modi-
   85                  fier; the internal form is output after compilation.
   87        -C        Output the version number of the PCRE library, and all avail-
   88                  able  information  about  the  optional  features  that   are
   89                  included,  and  then  exit  with  zero  exit  code. All other
   90                  options are ignored.
   92        -C option Output information about a specific build-time  option,  then
   93                  exit.  This functionality is intended for use in scripts such
   94                  as RunTest. The following options output the  value  and  set
   95                  the exit code as indicated:
   97                    ebcdic-nl  the code for LF (= NL) in an EBCDIC environment:
   98                                 0x15 or 0x25
   99                                 0 if used in an ASCII environment
  100                                 exit code is always 0
  101                    linksize   the configured internal link size (2, 3, or 4)
  102                                 exit code is set to the link size
  103                    newline    the default newline setting:
  104                                 CR, LF, CRLF, ANYCRLF, or ANY
  105                                 exit code is always 0
  106                    bsr        the default setting for what \R matches:
  107                                 ANYCRLF or ANY
  108                                 exit code is always 0
  110                  The  following  options output 1 for true or 0 for false, and
  111                  set the exit code to the same value:
  113                    ebcdic     compiled for an EBCDIC environment
  114                    jit        just-in-time support is available
  115                    pcre16     the 16-bit library was built
  116                    pcre32     the 32-bit library was built
  117                    pcre8      the 8-bit library was built
  118                    ucp        Unicode property support is available
  119                    utf        UTF-8 and/or UTF-16 and/or UTF-32 support
  120                                 is available
  122                  If an unknown option is given, an error  message  is  output;
  123                  the exit code is 0.
  125        -d        Behave  as  if  each pattern has the /D (debug) modifier; the
  126                  internal form and information about the compiled  pattern  is
  127                  output after compilation; -d is equivalent to -b -i.
  129        -dfa      Behave  as if each data line contains the \D escape sequence;
  130                  this    causes    the    alternative    matching    function,
  131                  pcre[16|32]_dfa_exec(),  to  be  used instead of the standard
  132                  pcre[16|32]_exec() function (more detail is given below).
  134        -help     Output a brief summary these options and then exit.
  136        -i        Behave as if each pattern has the  /I  modifier;  information
  137                  about the compiled pattern is given after compilation.
  139        -M        Behave  as if each data line contains the \M escape sequence;
  140                  this causes PCRE to  discover  the  minimum  MATCH_LIMIT  and
  141                  MATCH_LIMIT_RECURSION  settings by calling pcre[16|32]_exec()
  142                  repeatedly with different limits.
  144        -m        Output the size of each compiled pattern after  it  has  been
  145                  compiled.  This  is  equivalent  to adding /M to each regular
  146                  expression. The size is given in bytes for both libraries.
  148        -O        Behave as if each pattern has the /O modifier, that  is  dis-
  149                  able auto-possessification for all patterns.
  151        -o osize  Set  the number of elements in the output vector that is used
  152                  when calling pcre[16|32]_exec() or pcre[16|32]_dfa_exec()  to
  153                  be  osize.  The  default  value is 45, which is enough for 14
  154                  capturing subexpressions for pcre[16|32]_exec() or 22 differ-
  155                  ent  matches for pcre[16|32]_dfa_exec().  The vector size can
  156                  be changed for individual matching calls by including  \O  in
  157                  the data line (see below).
  159        -p        Behave  as  if  each  pattern  has the /P modifier; the POSIX
  160                  wrapper API is used to call PCRE. None of the  other  options
  161                  has  any  effect when -p is set. This option can be used only
  162                  with the 8-bit library.
  164        -q        Do not output the version number of pcretest at the start  of
  165                  execution.
  167        -S size   On  Unix-like  systems, set the size of the run-time stack to
  168                  size megabytes.
  170        -s or -s+ Behave as if each pattern  has  the  /S  modifier;  in  other
  171                  words,  force each pattern to be studied. If -s+ is used, all
  172                  the JIT compile options are  passed  to  pcre[16|32]_study(),
  173                  causing  just-in-time  optimization  to  be  set  up if it is
  174                  available, for both full and partial matching.  Specific  JIT
  175                  compile options can be selected by following -s+ with a digit
  176                  in the range 1 to 7, which selects the JIT compile  modes  as
  177                  follows:
  179                    1  normal match only
  180                    2  soft partial match only
  181                    3  normal match and soft partial match
  182                    4  hard partial match only
  183                    6  soft and hard partial match
  184                    7  all three modes (default)
  186                  If  -s++  is used instead of -s+ (with or without a following
  187                  digit), the text "(JIT)" is added to the  first  output  line
  188                  after a match or no match when JIT-compiled code was actually
  189                  used.
  191                  Note that there are pattern options  that  can  override  -s,
  192                  either specifying no studying at all, or suppressing JIT com-
  193                  pilation.
  195                  If the /I or /D option is present on  a  pattern  (requesting
  196                  output  about  the  compiled  pattern), information about the
  197                  result of studying is not included when  studying  is  caused
  198                  only  by  -s  and neither -i nor -d is present on the command
  199                  line. This behaviour means that the output  from  tests  that
  200                  are  run with and without -s should be identical, except when
  201                  options that output information about the actual running of a
  202                  match are set.
  204                  The  -M,  -t,  and  -tm options, which give information about
  205                  resources used, are likely to produce different  output  with
  206                  and  without  -s.  Output may also differ if the /C option is
  207                  present on an individual pattern. This uses callouts to trace
  208                  the  the  matching process, and this may be different between
  209                  studied and non-studied patterns.  If  the  pattern  contains
  210                  (*MARK)  items  there  may  also be differences, for the same
  211                  reason. The -s command line option can be overridden for spe-
  212                  cific  patterns that should never be studied (see the /S pat-
  213                  tern modifier below).
  215        -t        Run each compile, study, and match many times with  a  timer,
  216                  and  output  the resulting times per compile, study, or match
  217                  (in milliseconds). Do not set -m with -t,  because  you  will
  218                  then get the size output a zillion times, and the timing will
  219                  be distorted. You can control the number of  iterations  that
  220                  are used for timing by following -t with a number (as a sepa-
  221                  rate item on the command line). For example, "-t 1000"  iter-
  222                  ates 1000 times.  The default is to iterate 500000 times.
  224        -tm       This is like -t except that it times only the matching phase,
  225                  not the compile or study phases.
  227        -T -TM    These behave like -t and -tm, but in addition, at the end  of
  228                  a run, the total times for all compiles, studies, and matches
  229                  are output.
  234        If pcretest is given two filename arguments, it reads  from  the  first
  235        and writes to the second. If it is given only one filename argument, it
  236        reads from that file and writes to stdout.  Otherwise,  it  reads  from
  237        stdin  and  writes to stdout, and prompts for each line of input, using
  238        "re>" to prompt for regular expressions, and "data>" to prompt for data
  239        lines.
  241        When  pcretest  is  built,  a  configuration option can specify that it
  242        should be linked with the libreadline library. When this  is  done,  if
  243        the input is from a terminal, it is read using the readline() function.
  244        This provides line-editing and history facilities. The output from  the
  245        -help option states whether or not readline() will be used.
  247        The program handles any number of sets of input on a single input file.
  248        Each set starts with a regular expression, and continues with any  num-
  249        ber of data lines to be matched against that pattern.
  251        Each  data line is matched separately and independently. If you want to
  252        do multi-line matches, you have to use the \n escape sequence (or \r or
  253        \r\n, etc., depending on the newline setting) in a single line of input
  254        to encode the newline sequences. There is no limit  on  the  length  of
  255        data  lines;  the  input  buffer is automatically extended if it is too
  256        small.
  258        An empty line signals the end of the data lines, at which point  a  new
  259        regular  expression is read. The regular expressions are given enclosed
  260        in any non-alphanumeric delimiters other than backslash, for example:
  262          /(a|bc)x+yz/
  264        White space before the initial delimiter is ignored. A regular  expres-
  265        sion  may be continued over several input lines, in which case the new-
  266        line characters are included within it. It is possible to  include  the
  267        delimiter within the pattern by escaping it, for example
  269          /abc\/def/
  271        If  you  do  so, the escape and the delimiter form part of the pattern,
  272        but since delimiters are always non-alphanumeric, this does not  affect
  273        its  interpretation.   If the terminating delimiter is immediately fol-
  274        lowed by a backslash, for example,
  276          /abc/\
  278        then a backslash is added to the end of the pattern. This  is  done  to
  279        provide  a  way of testing the error condition that arises if a pattern
  280        finishes with a backslash, because
  282          /abc\/
  284        is interpreted as the first line of a pattern that starts with  "abc/",
  285        causing pcretest to read the next line as a continuation of the regular
  286        expression.
  291        A pattern may be followed by any number of modifiers, which are  mostly
  292        single  characters,  though  some  of these can be qualified by further
  293        characters.  Following Perl usage, these are referred to below as,  for
  294        example,  "the  /i  modifier", even though the delimiter of the pattern
  295        need not always be a slash, and no slash is  used  when  writing  modi-
  296        fiers.  White  space may appear between the final pattern delimiter and
  297        the first modifier, and between the modifiers  themselves.  For  refer-
  298        ence,  here  is  a  complete  list of modifiers. They fall into several
  299        groups that are described in detail in the following sections.
  301          /8              set UTF mode
  302          /9              set PCRE_NEVER_UTF (locks out UTF mode)
  303          /?              disable UTF validity check
  304          /+              show remainder of subject after match
  305          /=              show all captures (not just those that are set)
  307          /A              set PCRE_ANCHORED
  308          /B              show compiled code
  309          /C              set PCRE_AUTO_CALLOUT
  310          /D              same as /B plus /I
  311          /E              set PCRE_DOLLAR_ENDONLY
  312          /F              flip byte order in compiled pattern
  313          /f              set PCRE_FIRSTLINE
  314          /G              find all matches (shorten string)
  315          /g              find all matches (use startoffset)
  316          /I              show information about pattern
  317          /i              set PCRE_CASELESS
  318          /J              set PCRE_DUPNAMES
  319          /K              show backtracking control names
  320          /L              set locale
  321          /M              show compiled memory size
  322          /m              set PCRE_MULTILINE
  323          /N              set PCRE_NO_AUTO_CAPTURE
  324          /O              set PCRE_NO_AUTO_POSSESS
  325          /P              use the POSIX wrapper
  326          /Q              test external stack check function
  327          /S              study the pattern after compilation
  328          /s              set PCRE_DOTALL
  329          /T              select character tables
  330          /U              set PCRE_UNGREEDY
  331          /W              set PCRE_UCP
  332          /X              set PCRE_EXTRA
  333          /x              set PCRE_EXTENDED
  334          /Y              set PCRE_NO_START_OPTIMIZE
  335          /Z              don't show lengths in /B output
  337          /<any>          set PCRE_NEWLINE_ANY
  338          /<anycrlf>      set PCRE_NEWLINE_ANYCRLF
  339          /<cr>           set PCRE_NEWLINE_CR
  340          /<crlf>         set PCRE_NEWLINE_CRLF
  341          /<lf>           set PCRE_NEWLINE_LF
  342          /<bsr_anycrlf>  set PCRE_BSR_ANYCRLF
  343          /<bsr_unicode>  set PCRE_BSR_UNICODE
  344          /<JS>           set PCRE_JAVASCRIPT_COMPAT
  347    Perl-compatible modifiers
  349        The /i, /m, /s, and /x modifiers set the PCRE_CASELESS, PCRE_MULTILINE,
  350        PCRE_DOTALL,    or    PCRE_EXTENDED    options,    respectively,   when
  351        pcre[16|32]_compile() is called. These four modifier letters  have  the
  352        same effect as they do in Perl. For example:
  354          /caseless/i
  357    Modifiers for other PCRE options
  359        The  following  table  shows additional modifiers for setting PCRE com-
  360        pile-time options that do not correspond to anything in Perl:
  362          /8              PCRE_UTF8           ) when using the 8-bit
  363          /?              PCRE_NO_UTF8_CHECK  )   library
  365          /8              PCRE_UTF16          ) when using the 16-bit
  366          /?              PCRE_NO_UTF16_CHECK )   library
  368          /8              PCRE_UTF32          ) when using the 32-bit
  369          /?              PCRE_NO_UTF32_CHECK )   library
  371          /9              PCRE_NEVER_UTF
  372          /A              PCRE_ANCHORED
  373          /C              PCRE_AUTO_CALLOUT
  374          /E              PCRE_DOLLAR_ENDONLY
  375          /f              PCRE_FIRSTLINE
  376          /J              PCRE_DUPNAMES
  377          /N              PCRE_NO_AUTO_CAPTURE
  378          /O              PCRE_NO_AUTO_POSSESS
  379          /U              PCRE_UNGREEDY
  380          /W              PCRE_UCP
  381          /X              PCRE_EXTRA
  382          /Y              PCRE_NO_START_OPTIMIZE
  383          /<any>          PCRE_NEWLINE_ANY
  384          /<anycrlf>      PCRE_NEWLINE_ANYCRLF
  385          /<cr>           PCRE_NEWLINE_CR
  386          /<crlf>         PCRE_NEWLINE_CRLF
  387          /<lf>           PCRE_NEWLINE_LF
  388          /<bsr_anycrlf>  PCRE_BSR_ANYCRLF
  389          /<bsr_unicode>  PCRE_BSR_UNICODE
  390          /<JS>           PCRE_JAVASCRIPT_COMPAT
  392        The modifiers that are enclosed in angle brackets are  literal  strings
  393        as  shown,  including the angle brackets, but the letters within can be
  394        in either case.  This example sets multiline matching with CRLF as  the
  395        line ending sequence:
  397          /^abc/m<CRLF>
  399        As  well  as  turning  on  the  PCRE_UTF8/16/32 option, the /8 modifier
  400        causes all non-printing characters in  output  strings  to  be  printed
  401        using the \x{hh...} notation. Otherwise, those less than 0x100 are out-
  402        put in hex without the curly brackets.
  404        Full details of the PCRE options are given in  the  pcreapi  documenta-
  405        tion.
  407    Finding all matches in a string
  409        Searching  for  all  possible matches within each subject string can be
  410        requested by the /g or /G modifier. After  finding  a  match,  PCRE  is
  411        called again to search the remainder of the subject string. The differ-
  412        ence between /g and /G is that the former uses the startoffset argument
  413        to  pcre[16|32]_exec()  to  start  searching  at a new point within the
  414        entire string (which is in effect what Perl does), whereas  the  latter
  415        passes  over  a  shortened  substring.  This  makes a difference to the
  416        matching process if the pattern  begins  with  a  lookbehind  assertion
  417        (including \b or \B).
  419        If  any  call  to  pcre[16|32]_exec() in a /g or /G sequence matches an
  420        empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART  and
  421        PCRE_ANCHORED  flags  set  in  order  to search for another, non-empty,
  422        match at the same point. If this second match fails, the  start  offset
  423        is  advanced,  and  the  normal match is retried. This imitates the way
  424        Perl handles such cases when using the /g modifier or the split() func-
  425        tion.  Normally,  the start offset is advanced by one character, but if
  426        the newline convention recognizes CRLF as a newline,  and  the  current
  427        character is CR followed by LF, an advance of two is used.
  429    Other modifiers
  431        There are yet more modifiers for controlling the way pcretest operates.
  433        The  /+ modifier requests that as well as outputting the substring that
  434        matched the entire pattern, pcretest  should  in  addition  output  the
  435        remainder  of  the  subject  string. This is useful for tests where the
  436        subject contains multiple copies of the same substring. If the +  modi-
  437        fier  appears  twice, the same action is taken for captured substrings.
  438        In each case the remainder is output on the following line with a  plus
  439        character  following  the  capture number. Note that this modifier must
  440        not immediately follow the /S modifier because /S+ and /S++ have  other
  441        meanings.
  443        The  /=  modifier  requests  that  the values of all potential captured
  444        parentheses be output after a match. By default, only those up  to  the
  445        highest one actually used in the match are output (corresponding to the
  446        return code from pcre[16|32]_exec()). Values in the offsets vector cor-
  447        responding  to higher numbers should be set to -1, and these are output
  448        as "<unset>". This modifier gives a way of checking that this  is  hap-
  449        pening.
  451        The  /B modifier is a debugging feature. It requests that pcretest out-
  452        put a representation of the compiled code after  compilation.  Normally
  453        this  information  contains length and offset values; however, if /Z is
  454        also present, this data is replaced by spaces. This is a  special  fea-
  455        ture  for  use  in the automatic test scripts; it ensures that the same
  456        output is generated for different internal link sizes.
  458        The /D modifier is a PCRE debugging feature, and is equivalent to  /BI,
  459        that is, both the /B and the /I modifiers.
  461        The  /F  modifier  causes pcretest to flip the byte order of the 2-byte
  462        and 4-byte fields in the compiled pattern. This facility is for testing
  463        the  feature  in PCRE that allows it to execute patterns that were com-
  464        piled on a host with a different endianness. This feature is not avail-
  465        able  when the POSIX interface to PCRE is being used, that is, when the
  466        /P pattern modifier is specified. See also the section about saving and
  467        reloading compiled patterns below.
  469        The  /I  modifier  requests  that pcretest output information about the
  470        compiled pattern (whether it is anchored, has a fixed first  character,
  471        and  so  on). It does this by calling pcre[16|32]_fullinfo() after com-
  472        piling a pattern. If the pattern is studied, the results  of  that  are
  473        also output. In this output, the word "char" means a non-UTF character,
  474        that is, the value of a single data item  (8-bit,  16-bit,  or  32-bit,
  475        depending on the library that is being tested).
  477        The  /K modifier requests pcretest to show names from backtracking con-
  478        trol verbs that are  returned  from  calls  to  pcre[16|32]_exec().  It
  479        causes  pcretest  to  create  a  pcre[16|32]_extra block if one has not
  480        already been created by a call to pcre[16|32]_study(), and to  set  the
  481        PCRE_EXTRA_MARK  flag  and  the  mark  field within it, every time that
  482        pcre[16|32]_exec() is called. If  the  variable  that  the  mark  field
  483        points  to  is  non-NULL  for  a  match,  non-match,  or partial match,
  484        pcretest prints the string to which it points. For  a  match,  this  is
  485        shown  on  a  line  by itself, tagged with "MK:". For a non-match it is
  486        added to the message.
  488        The /L modifier must be followed directly by the name of a locale,  for
  489        example,
  491          /pattern/Lfr_FR
  493        For this reason, it must be the last modifier. The given locale is set,
  494        pcre[16|32]_maketables() is called to build a set of  character  tables
  495        for  the  locale, and this is then passed to pcre[16|32]_compile() when
  496        compiling the regular expression. Without an /L (or /T) modifier,  NULL
  497        is  passed  as  the  tables  pointer;  that  is, /L applies only to the
  498        expression on which it appears.
  500        The /M modifier causes the size in bytes of the memory  block  used  to
  501        hold  the compiled pattern to be output. This does not include the size
  502        of the pcre[16|32] block; it is just the actual compiled data.  If  the
  503        pattern is successfully studied with the PCRE_STUDY_JIT_COMPILE option,
  504        the size of the JIT compiled code is also output.
  506        The /Q modifier is used to test the use of pcre_stack_guard. It must be
  507        followed  by '0' or '1', specifying the return code to be given from an
  508        external function that is passed to PCRE and used  for  stack  checking
  509        during compilation (see the pcreapi documentation for details).
  511        The  /S  modifier  causes  pcre[16|32]_study()  to  be called after the
  512        expression has been compiled, and the results used when the  expression
  513        is matched. There are a number of qualifying characters that may follow
  514        /S.  They may appear in any order.
  516        If /S is followed by an exclamation mark, pcre[16|32]_study() is called
  517        with  the PCRE_STUDY_EXTRA_NEEDED option, causing it always to return a
  518        pcre_extra block, even when studying discovers no useful information.
  520        If /S is followed by a second S character, it suppresses studying, even
  521        if  it  was  requested  externally  by the -s command line option. This
  522        makes it possible to specify that certain patterns are always  studied,
  523        and others are never studied, independently of -s. This feature is used
  524        in the test files in a few cases where the output is different when the
  525        pattern is studied.
  527        If  the  /S  modifier  is  followed  by  a  +  character,  the  call to
  528        pcre[16|32]_study() is made with all the JIT study options,  requesting
  529        just-in-time  optimization  support if it is available, for both normal
  530        and partial matching. If you want to restrict the JIT compiling  modes,
  531        you can follow /S+ with a digit in the range 1 to 7:
  533          1  normal match only
  534          2  soft partial match only
  535          3  normal match and soft partial match
  536          4  hard partial match only
  537          6  soft and hard partial match
  538          7  all three modes (default)
  540        If /S++ is used instead of /S+ (with or without a following digit), the
  541        text "(JIT)" is added to the first output line  after  a  match  or  no
  542        match when JIT-compiled code was actually used.
  544        Note  that  there  is  also  an independent /+ modifier; it must not be
  545        given immediately after /S or /S+ because this will be misinterpreted.
  547        If JIT studying is successful, the compiled JIT code will automatically
  548        be  used  when pcre[16|32]_exec() is run, except when incompatible run-
  549        time options are specified. For more details, see the pcrejit  documen-
  550        tation.  See also the \J escape sequence below for a way of setting the
  551        size of the JIT stack.
  553        Finally, if /S is followed by a minus  character,  JIT  compilation  is
  554        suppressed,  even if it was requested externally by the -s command line
  555        option. This makes it possible to specify that JIT is never to be  used
  556        for certain patterns.
  558        The  /T  modifier  must be followed by a single digit. It causes a spe-
  559        cific set of built-in character tables to be passed to pcre[16|32]_com-
  560        pile().  It  is used in the standard PCRE tests to check behaviour with
  561        different character tables. The digit specifies the tables as follows:
  563          0   the default ASCII tables, as distributed in
  564                pcre_chartables.c.dist
  565          1   a set of tables defining ISO 8859 characters
  567        In table 1, some characters whose codes are greater than 128 are  iden-
  568        tified as letters, digits, spaces, etc.
  570    Using the POSIX wrapper API
  572        The  /P modifier causes pcretest to call PCRE via the POSIX wrapper API
  573        rather than its native API. This supports only the 8-bit library.  When
  574        /P  is set, the following modifiers set options for the regcomp() func-
  575        tion:
  577          /i    REG_ICASE
  578          /m    REG_NEWLINE
  579          /N    REG_NOSUB
  580          /s    REG_DOTALL     )
  581          /U    REG_UNGREEDY   ) These options are not part of
  582          /W    REG_UCP        )   the POSIX standard
  583          /8    REG_UTF8       )
  585        The /+ modifier works as  described  above.  All  other  modifiers  are
  586        ignored.
  588    Locking out certain modifiers
  590        PCRE  can be compiled with or without support for certain features such
  591        as UTF-8/16/32 or Unicode properties. Accordingly, the  standard  tests
  592        are  split  up  into  a number of different files that are selected for
  593        running depending on which features are available.  When  updating  the
  594        tests, it is all too easy to put a new test into the wrong file by mis-
  595        take; for example, to put a test that requires UTF support into a  file
  596        that  is used when it is not available. To help detect such mistakes as
  597        early as possible, there is a facility for locking out  specific  modi-
  598        fiers. If an input line for pcretest starts with the string "< forbid "
  599        the following sequence of characters is taken as a  list  of  forbidden
  600        modifiers. For example, in the test files that must not use UTF or Uni-
  601        code property support, this line appears:
  603          < forbid 8W
  605        This locks out the /8 and /W modifiers. An immediate error is given  if
  606        they  are  subsequently encountered. If the character string contains <
  607        but not >, all the multi-character modifiers  that  begin  with  <  are
  608        locked  out.  Otherwise,  such modifiers must be explicitly listed, for
  609        example:
  611          < forbid <JS><cr>
  613        There must be a single space between < and "forbid" for this feature to
  614        be  recognised.  If  there  is not, the line is interpreted either as a
  615        request to re-load a pre-compiled pattern (see  "SAVING  AND  RELOADING
  616        COMPILED  PATTERNS"  below) or, if there is a another < character, as a
  617        pattern that uses < as its delimiter.
  622        Before each data line is  passed  to  pcre[16|32]_exec(),  leading  and
  623        trailing  white space is removed, and it is then scanned for \ escapes.
  624        Some of these are pretty esoteric features, intended for  checking  out
  625        some  of the more complicated features of PCRE. If you are just testing
  626        "ordinary" regular expressions, you probably don't need any  of  these.
  627        The following escapes are recognized:
  629          \a         alarm (BEL, \x07)
  630          \b         backspace (\x08)
  631          \e         escape (\x27)
  632          \f         form feed (\x0c)
  633          \n         newline (\x0a)
  634          \qdd       set the PCRE_MATCH_LIMIT limit to dd
  635                       (any number of digits)
  636          \r         carriage return (\x0d)
  637          \t         tab (\x09)
  638          \v         vertical tab (\x0b)
  639          \nnn       octal character (up to 3 octal digits); always
  640                       a byte unless > 255 in UTF-8 or 16-bit or 32-bit mode
  641          \o{dd...}  octal character (any number of octal digits}
  642          \xhh       hexadecimal byte (up to 2 hex digits)
  643          \x{hh...}  hexadecimal character (any number of hex digits)
  644          \A         pass the PCRE_ANCHORED option to pcre[16|32]_exec()
  645                       or pcre[16|32]_dfa_exec()
  646          \B         pass the PCRE_NOTBOL option to pcre[16|32]_exec()
  647                       or pcre[16|32]_dfa_exec()
  648          \Cdd       call pcre[16|32]_copy_substring() for substring dd
  649                       after a successful match (number less than 32)
  650          \Cname     call pcre[16|32]_copy_named_substring() for substring
  651                       "name" after a successful match (name termin-
  652                       ated by next non alphanumeric character)
  653          \C+        show the current captured substrings at callout
  654                       time
  655          \C-        do not supply a callout function
  656          \C!n       return 1 instead of 0 when callout number n is
  657                       reached
  658          \C!n!m     return 1 instead of 0 when callout number n is
  659                       reached for the nth time
  660          \C*n       pass the number n (may be negative) as callout
  661                       data; this is used as the callout return value
  662          \D         use the pcre[16|32]_dfa_exec() match function
  663          \F         only shortest match for pcre[16|32]_dfa_exec()
  664          \Gdd       call pcre[16|32]_get_substring() for substring dd
  665                       after a successful match (number less than 32)
  666          \Gname     call pcre[16|32]_get_named_substring() for substring
  667                       "name" after a successful match (name termin-
  668                       ated by next non-alphanumeric character)
  669          \Jdd       set up a JIT stack of dd kilobytes maximum (any
  670                       number of digits)
  671          \L         call pcre[16|32]_get_substringlist() after a
  672                       successful match
  673          \M         discover the minimum MATCH_LIMIT and
  674                       MATCH_LIMIT_RECURSION settings
  675          \N         pass the PCRE_NOTEMPTY option to pcre[16|32]_exec()
  676                       or pcre[16|32]_dfa_exec(); if used twice, pass the
  677                       PCRE_NOTEMPTY_ATSTART option
  678          \Odd       set the size of the output vector passed to
  679                       pcre[16|32]_exec() to dd (any number of digits)
  680          \P         pass the PCRE_PARTIAL_SOFT option to pcre[16|32]_exec()
  681                       or pcre[16|32]_dfa_exec(); if used twice, pass the
  682                       PCRE_PARTIAL_HARD option
  683          \Qdd       set the PCRE_MATCH_LIMIT_RECURSION limit to dd
  684                       (any number of digits)
  685          \R         pass the PCRE_DFA_RESTART option to pcre[16|32]_dfa_exec()
  686          \S         output details of memory get/free calls during matching
  687          \Y             pass     the    PCRE_NO_START_OPTIMIZE    option    to
  688        pcre[16|32]_exec()
  689                       or pcre[16|32]_dfa_exec()
  690          \Z         pass the PCRE_NOTEOL option to pcre[16|32]_exec()
  691                       or pcre[16|32]_dfa_exec()
  692          \?         pass the PCRE_NO_UTF[8|16|32]_CHECK option to
  693                       pcre[16|32]_exec() or pcre[16|32]_dfa_exec()
  694          \>dd       start the match at offset dd (optional "-"; then
  695                       any number of digits); this sets the startoffset
  696                       argument        for        pcre[16|32]_exec()         or
  697        pcre[16|32]_dfa_exec()
  698          \<cr>      pass the PCRE_NEWLINE_CR option to pcre[16|32]_exec()
  699                       or pcre[16|32]_dfa_exec()
  700          \<lf>      pass the PCRE_NEWLINE_LF option to pcre[16|32]_exec()
  701                       or pcre[16|32]_dfa_exec()
  702          \<crlf>    pass the PCRE_NEWLINE_CRLF option to pcre[16|32]_exec()
  703                       or pcre[16|32]_dfa_exec()
  704          \<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to pcre[16|32]_exec()
  705                       or pcre[16|32]_dfa_exec()
  706          \<any>     pass the PCRE_NEWLINE_ANY option to pcre[16|32]_exec()
  707                       or pcre[16|32]_dfa_exec()
  709        The  use of \x{hh...} is not dependent on the use of the /8 modifier on
  710        the pattern. It is recognized always. There may be any number of  hexa-
  711        decimal  digits  inside  the  braces; invalid values provoke error mes-
  712        sages.
  714        Note that \xhh specifies one byte rather than one  character  in  UTF-8
  715        mode;  this  makes it possible to construct invalid UTF-8 sequences for
  716        testing purposes. On the other hand, \x{hh} is interpreted as  a  UTF-8
  717        character  in UTF-8 mode, generating more than one byte if the value is
  718        greater than 127.  When testing the 8-bit library not  in  UTF-8  mode,
  719        \x{hh} generates one byte for values less than 256, and causes an error
  720        for greater values.
  722        In UTF-16 mode, all 4-digit \x{hhhh} values are accepted. This makes it
  723        possible to construct invalid UTF-16 sequences for testing purposes.
  725        In  UTF-32  mode,  all  4- to 8-digit \x{...} values are accepted. This
  726        makes it possible to construct invalid  UTF-32  sequences  for  testing
  727        purposes.
  729        The  escapes  that  specify  line ending sequences are literal strings,
  730        exactly as shown. No more than one newline setting should be present in
  731        any data line.
  733        A  backslash  followed by anything else just escapes the anything else.
  734        If the very last character is a backslash, it is ignored. This gives  a
  735        way  of  passing  an empty line as data, since a real empty line termi-
  736        nates the data input.
  738        The \J escape provides a way of setting the maximum stack size that  is
  739        used  by the just-in-time optimization code. It is ignored if JIT opti-
  740        mization is not being used. Providing a stack that is larger  than  the
  741        default 32K is necessary only for very complicated patterns.
  743        If \M is present, pcretest calls pcre[16|32]_exec() several times, with
  744        different values in the match_limit and match_limit_recursion fields of
  745        the  pcre[16|32]_extra  data structure, until it finds the minimum num-
  746        bers for each parameter that allow pcre[16|32]_exec() to complete with-
  747        out  error.  Because  this  is testing a specific feature of the normal
  748        interpretive pcre[16|32]_exec() execution, the use of any JIT optimiza-
  749        tion  that might have been set up by the /S+ qualifier of -s+ option is
  750        disabled.
  752        The match_limit number is a measure of the amount of backtracking  that
  753        takes  place,  and  checking it out can be instructive. For most simple
  754        matches, the number is quite small, but for patterns  with  very  large
  755        numbers  of  matching  possibilities,  it can become large very quickly
  756        with increasing length of  subject  string.  The  match_limit_recursion
  757        number  is  a  measure  of how much stack (or, if PCRE is compiled with
  758        NO_RECURSE, how much heap) memory  is  needed  to  complete  the  match
  759        attempt.
  761        When  \O  is  used, the value specified may be higher or lower than the
  762        size set by the -O command line option (or defaulted to 45); \O applies
  763        only  to  the  call  of  pcre[16|32]_exec()  for  the  line in which it
  764        appears.
  766        If the /P modifier was present on the pattern, causing the POSIX  wrap-
  767        per  API  to  be  used, the only option-setting sequences that have any
  768        effect are \B,  \N,  and  \Z,  causing  REG_NOTBOL,  REG_NOTEMPTY,  and
  769        REG_NOTEOL, respectively, to be passed to regexec().
  774        By   default,  pcretest  uses  the  standard  PCRE  matching  function,
  775        pcre[16|32]_exec() to match each  data  line.  PCRE  also  supports  an
  776        alternative  matching  function, pcre[16|32]_dfa_test(), which operates
  777        in a different way, and has some restrictions. The differences  between
  778        the two functions are described in the pcrematching documentation.
  780        If  a data line contains the \D escape sequence, or if the command line
  781        contains the -dfa option, the alternative matching  function  is  used.
  782        This function finds all possible matches at a given point. If, however,
  783        the \F escape sequence is present in the data line, it stops after  the
  784        first match is found. This is always the shortest possible match.
  789        This  section  describes  the output when the normal matching function,
  790        pcre[16|32]_exec(), is being used.
  792        When a match succeeds, pcretest outputs the list of captured substrings
  793        that  pcre[16|32]_exec() returns, starting with number 0 for the string
  794        that matched the whole pattern. Otherwise, it outputs "No  match"  when
  795        the  return is PCRE_ERROR_NOMATCH, and "Partial match:" followed by the
  796        partially   matching   substring   when   pcre[16|32]_exec()    returns
  797        PCRE_ERROR_PARTIAL.  (Note  that  this is the entire substring that was
  798        inspected during the partial match; it may  include  characters  before
  799        the  actual  match  start  if a lookbehind assertion, \K, \b, or \B was
  800        involved.) For any other return, pcretest  outputs  the  PCRE  negative
  801        error  number  and a short descriptive phrase. If the error is a failed
  802        UTF string check, the offset of the start of the failing character  and
  803        the  reason  code are also output, provided that the size of the output
  804        vector is at least two. Here is an example of an  interactive  pcretest
  805        run.
  807          $ pcretest
  808          PCRE version 8.13 2011-04-30
  810            re> /^abc(\d+)/
  811          data> abc123
  812           0: abc123
  813           1: 123
  814          data> xyz
  815          No match
  817        Unset capturing substrings that are not followed by one that is set are
  818        not returned by pcre[16|32]_exec(), and are not shown by  pcretest.  In
  819        the following example, there are two capturing substrings, but when the
  820        first data line is matched, the second, unset substring is  not  shown.
  821        An  "internal" unset substring is shown as "<unset>", as for the second
  822        data line.
  824            re> /(a)|(b)/
  825          data> a
  826           0: a
  827           1: a
  828          data> b
  829           0: b
  830           1: <unset>
  831           2: b
  833        If the strings contain any non-printing characters, they are output  as
  834        \xhh  escapes  if  the  value is less than 256 and UTF mode is not set.
  835        Otherwise they are output as \x{hh...} escapes. See below for the defi-
  836        nition  of non-printing characters. If the pattern has the /+ modifier,
  837        the output for substring 0 is followed by the the rest of  the  subject
  838        string, identified by "0+" like this:
  840            re> /cat/+
  841          data> cataract
  842           0: cat
  843           0+ aract
  845        If  the  pattern  has  the /g or /G modifier, the results of successive
  846        matching attempts are output in sequence, like this:
  848            re> /\Bi(\w\w)/g
  849          data> Mississippi
  850           0: iss
  851           1: ss
  852           0: iss
  853           1: ss
  854           0: ipp
  855           1: pp
  857        "No match" is output only if the first match attempt fails. Here is  an
  858        example  of a failure message (the offset 4 that is specified by \>4 is
  859        past the end of the subject string):
  861            re> /xyz/
  862          data> xyz\>4
  863          Error -24 (bad offset value)
  865        If any of the sequences \C, \G, or \L are present in a data  line  that
  866        is  successfully  matched,  the substrings extracted by the convenience
  867        functions are output with C, G, or L after the string number instead of
  868        a colon. This is in addition to the normal full list. The string length
  869        (that is, the return from the extraction function) is given  in  paren-
  870        theses after each string for \C and \G.
  872        Note that whereas patterns can be continued over several lines (a plain
  873        ">" prompt is used for continuations), data lines may not. However new-
  874        lines  can  be included in data by means of the \n escape (or \r, \r\n,
  875        etc., depending on the newline sequence setting).
  880        When the alternative matching function, pcre[16|32]_dfa_exec(), is used
  881        (by  means  of the \D escape sequence or the -dfa command line option),
  882        the output consists of a list of all the  matches  that  start  at  the
  883        first point in the subject where there is at least one match. For exam-
  884        ple:
  886            re> /(tang|tangerine|tan)/
  887          data> yellow tangerine\D
  888           0: tangerine
  889           1: tang
  890           2: tan
  892        (Using the normal matching function on this data  finds  only  "tang".)
  893        The  longest matching string is always given first (and numbered zero).
  894        After a PCRE_ERROR_PARTIAL return, the output is "Partial match:", fol-
  895        lowed  by  the  partially  matching  substring.  (Note that this is the
  896        entire substring that was inspected during the partial  match;  it  may
  897        include characters before the actual match start if a lookbehind asser-
  898        tion, \K, \b, or \B was involved.)
  900        If /g is present on the pattern, the search for further matches resumes
  901        at the end of the longest match. For example:
  903            re> /(tang|tangerine|tan)/g
  904          data> yellow tangerine and tangy sultana\D
  905           0: tangerine
  906           1: tang
  907           2: tan
  908           0: tang
  909           1: tan
  910           0: tan
  912        Since  the  matching  function  does not support substring capture, the
  913        escape sequences that are concerned with captured  substrings  are  not
  914        relevant.
  919        When the alternative matching function has given the PCRE_ERROR_PARTIAL
  920        return, indicating that the subject partially matched the pattern,  you
  921        can  restart  the match with additional subject data by means of the \R
  922        escape sequence. For example:
  924            re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
  925          data> 23ja\P\D
  926          Partial match: 23ja
  927          data> n05\R\D
  928           0: n05
  930        For further information about partial  matching,  see  the  pcrepartial
  931        documentation.
  936        If  the pattern contains any callout requests, pcretest's callout func-
  937        tion is called during matching. This works  with  both  matching  func-
  938        tions. By default, the called function displays the callout number, the
  939        start and current positions in the text at the callout  time,  and  the
  940        next pattern item to be tested. For example:
  942          --->pqrabcdef
  943            0    ^  ^     \d
  945        This  output  indicates  that  callout  number  0  occurred for a match
  946        attempt starting at the fourth character of the  subject  string,  when
  947        the pointer was at the seventh character of the data, and when the next
  948        pattern item was \d. Just one circumflex is output  if  the  start  and
  949        current positions are the same.
  951        Callouts numbered 255 are assumed to be automatic callouts, inserted as
  952        a result of the /C pattern modifier. In this case, instead  of  showing
  953        the  callout  number, the offset in the pattern, preceded by a plus, is
  954        output. For example:
  956            re> /\d?[A-E]\*/C
  957          data> E*
  958          --->E*
  959           +0 ^      \d?
  960           +3 ^      [A-E]
  961           +8 ^^     \*
  962          +10 ^ ^
  963           0: E*
  965        If a pattern contains (*MARK) items, an additional line is output when-
  966        ever  a  change  of  latest mark is passed to the callout function. For
  967        example:
  969            re> /a(*MARK:X)bc/C
  970          data> abc
  971          --->abc
  972           +0 ^       a
  973           +1 ^^      (*MARK:X)
  974          +10 ^^      b
  975          Latest Mark: X
  976          +11 ^ ^     c
  977          +12 ^  ^
  978           0: abc
  980        The mark changes between matching "a" and "b", but stays the  same  for
  981        the  rest  of  the match, so nothing more is output. If, as a result of
  982        backtracking, the mark reverts to being unset, the  text  "<unset>"  is
  983        output.
  985        The  callout  function  in pcretest returns zero (carry on matching) by
  986        default, but you can use a \C item in a data line (as described  above)
  987        to change this and other parameters of the callout.
  989        Inserting  callouts can be helpful when using pcretest to check compli-
  990        cated regular expressions. For further information about callouts,  see
  991        the pcrecallout documentation.
  996        When  pcretest is outputting text in the compiled version of a pattern,
  997        bytes other than 32-126 are always treated as  non-printing  characters
  998        are are therefore shown as hex escapes.
 1000        When  pcretest  is  outputting text that is a matched part of a subject
 1001        string, it behaves in the same way, unless a different locale has  been
 1002        set  for  the  pattern  (using  the  /L  modifier).  In  this case, the
 1003        isprint() function to distinguish printing and non-printing characters.
 1008        The facilities described in this section are  not  available  when  the
 1009        POSIX  interface  to  PCRE  is being used, that is, when the /P pattern
 1010        modifier is specified.
 1012        When the POSIX interface is not in use, you can cause pcretest to write
 1013        a  compiled  pattern to a file, by following the modifiers with > and a
 1014        file name.  For example:
 1016          /pattern/im >/some/file
 1018        See the pcreprecompile documentation for a discussion about saving  and
 1019        re-using  compiled patterns.  Note that if the pattern was successfully
 1020        studied with JIT optimization, the JIT data cannot be saved.
 1022        The data that is written is binary.  The  first  eight  bytes  are  the
 1023        length  of  the  compiled  pattern  data  followed by the length of the
 1024        optional study data, each written as four  bytes  in  big-endian  order
 1025        (most  significant  byte  first). If there is no study data (either the
 1026        pattern was not studied, or studying did not return any data), the sec-
 1027        ond  length  is  zero. The lengths are followed by an exact copy of the
 1028        compiled pattern. If there is additional study  data,  this  (excluding
 1029        any  JIT  data)  follows  immediately after the compiled pattern. After
 1030        writing the file, pcretest expects to read a new pattern.
 1032        A saved pattern can be reloaded into pcretest by  specifying  <  and  a
 1033        file  name  instead  of a pattern. There must be no space between < and
 1034        the file name, which must not  contain  a  <  character,  as  otherwise
 1035        pcretest  will  interpret  the line as a pattern delimited by < charac-
 1036        ters. For example:
 1038           re> </some/file
 1039          Compiled pattern loaded from /some/file
 1040          No study data
 1042        If the pattern was previously studied with the  JIT  optimization,  the
 1043        JIT  information cannot be saved and restored, and so is lost. When the
 1044        pattern has been loaded, pcretest proceeds to read data  lines  in  the
 1045        usual way.
 1047        You  can copy a file written by pcretest to a different host and reload
 1048        it there, even if the new host has opposite endianness to  the  one  on
 1049        which  the pattern was compiled. For example, you can compile on an i86
 1050        machine and run on a SPARC machine. When a pattern  is  reloaded  on  a
 1051        host with different endianness, the confirmation message is changed to:
 1053          Compiled pattern (byte-inverted) loaded from /some/file
 1055        The test suite contains some saved pre-compiled patterns with different
 1056        endianness. These are reloaded using "<!" instead  of  just  "<".  This
 1057        suppresses the "(byte-inverted)" text so that the output is the same on
 1058        all hosts. It also forces debugging output once the  pattern  has  been
 1059        reloaded.
 1061        File  names  for  saving and reloading can be absolute or relative, but
 1062        note that the shell facility of expanding a file name that starts  with
 1063        a tilde (~) is not available.
 1065        The  ability to save and reload files in pcretest is intended for test-
 1066        ing and experimentation. It is not intended for production use  because
 1067        only  a  single pattern can be written to a file. Furthermore, there is
 1068        no facility for supplying  custom  character  tables  for  use  with  a
 1069        reloaded  pattern.  If  the  original  pattern was compiled with custom
 1070        tables, an attempt to match a subject string using a  reloaded  pattern
 1071        is  likely to cause pcretest to crash.  Finally, if you attempt to load
 1072        a file that is not in the correct format, the result is undefined.
 1075 SEE ALSO
 1077        pcre(3), pcre16(3),  pcre32(3),  pcreapi(3),  pcrecallout(3),  pcrejit,
 1078        pcrematching(3), pcrepartial(d), pcrepattern(3), pcreprecompile(3).
 1081 AUTHOR
 1083        Philip Hazel
 1084        University Computing Service
 1085        Cambridge CB2 3QH, England.
 1090        Last updated: 23 February 2017
 1091        Copyright (c) 1997-2017 University of Cambridge.