"Fossies" - the Fresh Open Source Software Archive

Member "roundup-2.0.0/doc/rest.txt" (29 Jun 2020, 71854 Bytes) of package /linux/www/roundup-2.0.0.tar.gz:


As a special service "Fossies" has tried to format the requested text file into HTML format (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 .. index:: pair: api; Representational state transfer
    2    pair: api; rest
    3 
    4 ====================
    5 REST API for Roundup
    6 ====================
    7 
    8 .. contents::
    9    :local:
   10    :depth: 3
   11 
   12 Introduction
   13 ------------
   14 
   15 After the last 1.6.0 Release, a REST-API developed in 2015 during a
   16 Google Summer of Code (GSOC) by Chau Nguyen, supervised by Ezio
   17 Melotti was integrated. The code was updated by Ralf Schlatterbeck
   18 and John Rouillard to fix some shortcomings and provide the necessary
   19 functions for a single page web application, e.g. etag support,
   20 pagination, field embedding among others.
   21 
   22 Enabling the REST API
   23 ---------------------
   24 
   25 The REST API can be disabled in the ``[web]`` section of ``config.ini``
   26 via the variable ``enable_rest`` which is ``yes`` by default.
   27 
   28 Users have to be authorized to use the rest api. The user must have
   29 "Rest Access" permission. To add this to the "User" role change
   30 schema.py to add::
   31 
   32     db.security.addPermissionToRole('User', 'Rest Access')
   33 
   34 This is usually included near where other permissions like "Web Access"
   35 or "Email Access" are assigned.
   36 
   37 You could also create a new role "rest" and assign the "Rest Access"
   38 permission to that role and then just add the "rest" permissions to
   39 those users who should have access.
   40 
   41 The REST api is reached via the ``/rest/`` endpoint of the tracker
   42 URL. Partial URLs paths below (not starting with https) will have
   43 /rest removed for brevity.
   44 
   45 Make sure that the ``secret_key`` option is defined in the
   46 ``[web]`` section of your tracker's ``config.ini``. Following the
   47 `upgrading directions`_ using ``roundup-admin ... updateconfig
   48 ...`` will generate the ``secret_key`` comments and setting. Then
   49 you can merge this into your ``config.ini``.  If you are
   50 installing a new tracker with ``roundup-admin ... install`` the
   51 ``secret_key`` value is automatically set to some random value.
   52 
   53 If ``secret_key`` is not set, the etag value returned by a REST call
   54 will changed on every call even though the item has not changed. This
   55 means users will be unable to submit changes using the rest
   56 interface. (Note, if you run roundup in a persistent mode: server,
   57 wsgi, mod_python, the etag will change on every restart if not
   58 explicitly set.)
   59 
   60 .. _upgrading directions: upgrading.html
   61 
   62 Preventing CSRF Attacks
   63 =======================
   64 
   65 Clients should set the header X-REQUESTED-WITH to any value and the
   66 tracker's config.ini should have ``csrf_enforce_header_x-requested-with
   67 = yes`` or ``required``.
   68 
   69 Rate Limiting the API
   70 =====================
   71 
   72 This is a work in progress. This version of roundup includes Rate
   73 Limiting for the API (which is different from rate limiting login
   74 attempts on the web interface).
   75 
   76 It is enabled by setting the ``api_calls_per_interval`` and
   77 ``api_interval_in_sec`` configuration parameters in the ``[web]``
   78 section of ``config.ini``. The settings are documented in the
   79 config.ini file.
   80 
   81 If ``api_calls_per_interval = 60`` and ``api_interval_in_sec = 60``
   82 the user can make 60 calls in a minute. They can use them all up in the
   83 first second and then get one call back every second. With
   84 ``api_calls_per_interval = 60`` and ``api_interval_in_sec = 3600`` (1
   85 hour) they can use all 60 calls in the first second and they get one
   86 additional call every 10 seconds. ``api_calls_per_interval`` is the
   87 burst rate that you are willing to allow within ``api_interval_in_sec``
   88 seconds. The average rate of use is the ratio of
   89 ``api_calls_per_interval/api_interval_in_sec``. So you can have many
   90 values that permit one call per second on average: 1/1, 60/60,
   91 3600/3600, but they all have a different maximum burst rates: 1/sec,
   92 60/sec and 3600/sec.
   93 
   94 A single page app may make 20 or 30 calls to populate the page (e.g. a
   95 list of open issues). Then wait a few seconds for the user to select
   96 an issue. When displaying the issue, it needs another 20 or calls to
   97 populate status dropdowns, pull the first 10 messages in the issue
   98 etc. Controlling the burst rate as well as the average rate is a
   99 tuning exercise left for the tracker admin.
  100 
  101 Also the rate limit is a little lossy. Under heavy load, it is
  102 possible for it to miscount allowing more than burst count. Errors of
  103 up to 10% have been seen on slower hardware.
  104 
  105 Client API
  106 ----------
  107 
  108 The top-level REST url ``/rest/`` will display the current version of
  109 the REST API (Version 1 as of this writing) and some links to relevant
  110 endpoints of the API. In the following the ``/rest`` prefix is omitted
  111 from relative REST-API links for brevity.
  112 
  113 Headers
  114 =======
  115 
  116 If rate limiting is enabled there are 3 "standard" headers:
  117 
  118    **X-RateLimit-Limit**:  Calls allowed per period.
  119 
  120    **X-RateLimit-Remaining**: Calls available to be completed in this window.
  121 
  122    **X-RateLimit-Reset**: window ends in this many seconds. (Note,
  123    not an epoch timestamp). After this time, all
  124    X-RateLimit-Limit calls are available again.
  125 
  126 and one helpful header to report the period that is missing
  127 from other lists of rate limit headers:
  128 
  129    **X-RateLimit-Limit-Period**: Defines period in seconds for X-RateLimit-Limit.
  130 
  131 Also if the user has exceeded the rate limit, this header is added:
  132 
  133    **Retry-After**: The number of second to wait until 1 api call will succeed.
  134 
  135 If the client has requested a deprecated API endpoint, the header:
  136 
  137    **Sunset**: an http date after which the end point will not be
  138    available. This is not returned by current code, but can be used
  139    when `Programming the REST API`_. It should be used as a hint that
  140    the REST endpoint will be going away. See
  141    https://tools.ietf.org/html/rfc8594 for details on this header and
  142    the sunset link type.
  143 
  144 Hyperdb Stats
  145 =============
  146 
  147 Adding ``@stats=true`` as a GET query parameter or POST data item will
  148 augment the response with an ``@stats`` dictionary. Any value other
  149 than ``true`` (any case) will disable the ``@stats`` dictionary. When
  150 stats are enabled the response includes an ``@stats`` member and looks
  151 like::
  152 
  153   { "data": {
  154       ...
  155       "@stats": {
  156             "cache_hits": 3,
  157             "cache_misses": 1,
  158             "get_items": 0.0009722709655761719,
  159             "filtering": 0,
  160             "elapsed": 0.04731464385986328
  161       }
  162     }
  163   }
  164 
  165 These are the same values returned in the html interface by setting
  166 the ``CGI_SHOW_TIMING`` environment variable. By default performance
  167 stats are not shown. The fields are subject to change. An
  168 understanding of the code is recommended if you are going to use this
  169 info.
  170 
  171 Versioning
  172 ==========
  173 
  174 Currently there is only one version of the API. Versions are simple
  175 integers. The current version is ``1``. Version selection is
  176 implemented in the server using one of three methods:
  177 
  178    1. Explicit version param in accept header:
  179       ``application/json; version=1``
  180 
  181    2. Version suffix in vendor accept header:
  182       ``application/vnd.json.test-v1+json``
  183 
  184    3. Adding version specifier in query string: ``@apiver=1``
  185 
  186 If an explicit version is not provided, the server default is used.
  187 The server default is reported by querying the ``/rest/`` endpoint as
  188 described above.
  189 
  190 General Guidelines
  191 ==================
  192 
  193 Performing a ``GET`` on an item or property of an item will return an
  194 ETag header or an @etag property. This needs to be submitted with
  195 ``DELETE``, ``PUT`` and ``PATCH`` operations on the item using an
  196 ``If-Match`` header or an ``"@etag`` property in the data payload if
  197 the method supports a payload.
  198 
  199 The exact details of returned data is determined by the value of the
  200 ``@verbose`` query parameter.  The various supported values and their
  201 effects are described in the following sections.
  202 
  203 The default return format is JSON. If you add the ``dicttoxml.py``
  204 module you can request XML formatted data using the header ``Accept:
  205 application/xml`` in your request. Both output formats are similar in
  206 structure.
  207 
  208 All output is wrapped in an envelope called ``data``.
  209 
  210 When using collection endpoints (think list of issues, users ...), the
  211 ``data`` envelope contains metadata (e.g. total number of items) as
  212 well as a ``collections`` list of objects::
  213 
  214   { "data": {
  215       "meta data field1": "value",
  216       "meta data field2": "value",
  217       "collection": [
  218             { "link": "url to item",
  219               "id": "internal identifier for item" },
  220 	    { "link": "url to second item",
  221               "id": "id item 2" },
  222       ... ]
  223       "@links": {
  224          "relation": [
  225                   { "rel": "relation/subrelation",
  226                     "uri": "uri to use to implement relation" },
  227                   ...
  228                   ],
  229          "relation2": [ {...} ], ...
  230       }
  231     }
  232   }
  233 
  234 available meta data is described in the documentation for the
  235 collections endpoint. 
  236 
  237 The ``link`` fields implement `HATEOS`_ by supplying a url for the
  238 resource represented by that object. The "link" parameter with the
  239 value of a url is a special case of the @links parameter.
  240 
  241 .. _HATEOS: https://en.wikipedia.org/wiki/HATEOAS
  242 
  243 In the @links object, each relationship is a list of full link json
  244 objects.  These include rel (relationship) and uri properties. In the
  245 future this may be extended to include other data like content-type.
  246 However including a full @links object for every item includes a lot
  247 of overhead since in most cases only the self relationship needs to be
  248 represented.
  249 
  250 Because every object, link and multilink ends up getting a url, the
  251 shorter 'link' representation is used for this special case. The
  252 ``link`` property expresses the ``self`` relationship and its value is
  253 the uri property of the full link object. In collections, properties
  254 from each item can be embedded in the returned data (see ``@fields``
  255 below). This can not be done if the property is called link as that
  256 conflicts with the self url.
  257 
  258 When using an item endpoint (think an individual issue), metadata is
  259 included in the ``data`` envelope. Inside of the envelope, the
  260 ``attributes`` object contains the data for the field/properties of
  261 the issue. Example::
  262 
  263   { "data": {
  264       "meta data field1": "value",
  265       "type": "type of item, issue, user ..."
  266       "link": "link to retrieve item",
  267       "attributes": {
  268           "title": "title of issue",
  269           "nosy": [ 
  270 	            { "link": "url for user4",
  271                       "id": "4" }
  272           ],
  273 
  274       ... }
  275     }
  276   }
  277 
  278 Using a property endpoint (e.g. title or nosy list for an issue) the
  279 ``data`` wrapper has a ``data`` subfield that represents the value of
  280 the property. This ``data`` subfield may be a simple string (all types
  281 except multilink) or a list of strings (multilink
  282 properties). Example::
  283 
  284   { "data": {
  285         "type": "description of class",
  286         "@etag": "\"f15e6942f00a41960de45f9413684591\"",
  287         "link": "link to retrieve property",
  288         "id": "id for object with this property",
  289         "data": "value of property"
  290     }
  291   }
  292 
  293 
  294 Special Endpoints
  295 =================
  296 
  297 There are a few special endpoints that provide some additional data.
  298 Tracker administrators can add new endpoints. See
  299 "Programming the REST API"_ below.
  300 
  301 /summary
  302 ^^^^^^^^
  303 
  304 A Summary page can be reached via ``/summary`` via the ``GET`` method.
  305 This is currently hard-coded for the standard tracker schema shipped
  306 with roundup and will display a summary of open issues.
  307 
  308 /data
  309 ^^^^^
  310 
  311 This is the primary entry point for data from the tracker.
  312 
  313 The ``/data`` link will display a set of classes of the tracker. All
  314 classes can be reached via ``/data/<classname>`` where ``<classname>``
  315 is replace with the name of the class to query, e.g. ``/data/issue``.
  316 Individual items of a class (e.g. a single issue) can be queried by
  317 giving the issue-id, e.g., ``/data/issue/42``. Individual properties of
  318 an item can be queried by appending the property, e.g.,
  319 ``/data/issue/42/title``.
  320 
  321 
  322 All the links mentioned in the following support the http method ``GET``. 
  323 Results of a ``GET`` request will always return the results as a
  324 dictionary with the entry ``data`` referring to the returned data.
  325 
  326 Details are in the sections below.
  327 
  328 /data/\ *class* Collection
  329 ==========================
  330 
  331 When performing the ``GET`` method on a class (e.g. ``/data/issue``),
  332 the ``data`` object includes the number of items available in
  333 ``@total_size``. A a ``collection`` list follows which contains the id
  334 and link to the respective item.  For example a get on
  335 https://.../rest/data/issue returns::
  336 
  337     {
  338 	"data": {
  339 	    "collection": [
  340 		{
  341 		    "id": "1",
  342 		    "link": "https://.../rest/data/issue/1"
  343 		},
  344 		{
  345 		    "id": "100",
  346 		    "link": "https://.../rest/data/issue/100"
  347 		}
  348 	...
  349 	    ],
  350 	    "@total_size": 171
  351 	}
  352     }
  353 
  354 Collection endpoints support a number of features as seen in the next
  355 sections.
  356 
  357 A server may implement a default maximum number of items in the
  358 collection.  This can be used to prevent denial of service (DOS).  As
  359 a result all clients must be programmed to expect pagination
  360 decorations in the response. See the section on pagination below for
  361 details.
  362 
  363 Searching
  364 ^^^^^^^^^
  365 
  366 Searching is done by adding roundup field names and values as query
  367 parameters. Using: https://.../rest/data/issue you can search using:
  368 
  369 .. list-table:: Query Parameters Examples
  370   :header-rows: 1
  371   :widths: 20 20 80
  372 
  373   * - Query parameter
  374     - Field type
  375     - Explanation
  376   * - ``title=foo``
  377     - String
  378     - perform a substring search and find any issue with the word foo
  379       in the title.
  380   * - ``status=2``
  381     - Link
  382     - find any issue whose status link is set to the id 2.
  383   * - ``status=open``
  384     - Link
  385     - find any issue where the name of the status is open.
  386       Note this is not a string match so using nosy=ope will fail.
  387   * - ``nosy=1``
  388     - MultiLink
  389     - find any issue where the multilink nosy includes the id 1.
  390   * - ``nosy=admin``
  391     - MultiLink
  392     - find any issue where the multilink nosy includes the user admin.
  393       Note this is not a string match so using nosy=admi will fail.
  394   * - ``booleanfield=1`` - also values: true, TRUE, yes, YES etc. Other
  395       values match false.
  396     - Boolean
  397     - find an issue with the boolean field set to true.
  398 
  399 As seen above, Links and Multilinks can be specified numerically or
  400 symbolically, e.g., searching for issues in status ``closed`` can be
  401 achieved by searching for ``status=closed`` or ``status=3`` (provided
  402 the ``closed`` status has ID 3). Note that even though the symbolic
  403 name is a string, in this case it is also a key value. As a result it
  404 only does an exact match.
  405 
  406 Searching for strings (e.g. the issue title, or a keyword name)
  407 performs a case-insensitive substring search. Searching for
  408 ``title=Something`` (or in long form title~=Something) will find all
  409 issues with "Something" or "someThing", etc. in the title.
  410 
  411 Changing the search to ``title:=Something`` (note the `:`) performs an
  412 exact case-sensitive string match for exactly one word ``Something``
  413 with a capital ``S``. Another example is:
  414 ``title:=test+that+nosy+actually+works.`` where the + signs are spaces
  415 in the string. Replacing ``+`` with the `URL encoding`_ for space
  416 ``%20`` will also work. Note that you must match the spaces when
  417 performing exact matches. So `title:=test++that+nosy+actually+works.``
  418 matches the word ``test`` with two spaces bewteen ``test`` and
  419 ``that`` in the title.
  420 
  421 To make this clear, searching
  422 ``https://.../rest/data/issue?keyword=Foo`` will not work unless there
  423 is a keyword with a (case sensitive) name field of ``Foo`` which is
  424 the key field of the keyword. However searching the text property
  425 ``name`` using ``https://.../rest/data/keyword?name=Foo`` (note
  426 searching keyword class not issue class) will return matches for
  427 ``Foo``, ``foobar``, ``foo taz`` etc.
  428 
  429 In all cases the field ``@total_size`` is reported which is the total
  430 number of items available if you were to retrieve all of them.
  431 
  432 Other data types: Date, Interval Integer, Number need examples and may
  433 need work to allow range searches. Full text search (e.g. over the
  434 body of a msg) is a work in progress.
  435 
  436 .. _URL Encoding: https://en.wikipedia.org/wiki/Percent-encoding
  437 
  438 Transitive Searching
  439 ~~~~~~~~~~~~~~~~~~~~
  440 
  441 In addition to searching an issue by its properties, you can search
  442 for issues where linked items have a certain property. For example
  443 using ``/issues?messages.author=1`` will find all issues that include
  444 (link to) a message created by the admin user. This can also be done
  445 using: ``/issues?messages.author=admin``. Note that this requires
  446 search permission for messages.author, user.id, and users.username (to
  447 perform search with ``admin``. If these search permissions are not
  448 present, the search will silently drop the attribute.
  449 
  450 Similarly you can find all issues where the nosy list includes James
  451 Bond with: ``issue?nosy.realname=james+bond``. The alternate way to
  452 perform this is to query the user class for the realname:
  453 ``user?realname=james+bond`` and retrieve the id. Then you can execute
  454 a second rest call ``issue?nosy=7`` to retrieve issues with id 7.
  455 
  456 Make sure that search access to the class/properties are granted to the
  457 user. Note that users can search a field even if they can't view
  458 it. However they may be able to use searches to discover the value of
  459 the field even if they can't view it.
  460 
  461 Sorting
  462 ^^^^^^^
  463 
  464 Collection endpoints support sorting. This is controlled by specifying a
  465 ``@sort`` parameter with a list of properties of the searched class.
  466 Optionally properties can include a sign ('+' or '-') to specify
  467 ascending or descending sort, respectively. If no sign is given,
  468 ascending sort is selected for this property. The following example
  469 would sort by status (in ascending order of the status.order property)
  470 and then by id of an issue::
  471 
  472     @sort=status,-id
  473 
  474 
  475 Pagination
  476 ^^^^^^^^^^
  477 
  478 Collection endpoints support pagination. This is controlled by query
  479 parameters ``@page_size`` and ``@page_index`` (Note the use of the
  480 leading `@` to make the parameters distinguishable from field names.)
  481 
  482 .. list-table:: Query Parameters Examples
  483   :header-rows: 1
  484   :widths: 20 80
  485 
  486   * - Query parameter
  487     - Explanation
  488   * -  ``@page_size``
  489     - specifies how many items are displayed at once. If no
  490       ``@page_size`` is specified, all matching items are returned.
  491   * - ``@page_index``
  492     - (which defaults to 1 if not given) specifies which page number
  493       of ``@page_size`` items is displayed.
  494 
  495 Also when pagination is enabled the returned data include pagination
  496 links along side the collection data. This looks like::
  497 
  498   { "data":
  499     { 
  500        "collection": { ... }, 
  501        "@total_size": 222,
  502        "@links": {
  503 	   "self": [
  504 	       {
  505 		   "uri":
  506 	   "https://.../rest/data/issue?@page_index=1&@fields=status&@page_size=5",
  507 		   "rel": "self"
  508 	       }
  509 	   ],
  510 	   "next": [
  511 	       {
  512 		   "uri":
  513 	   "https://.../rest/data/issue?@page_index=2&@fields=status&@page_size=5",
  514 		   "rel": "next"
  515 	       }
  516 	   ]
  517 	 }
  518       }
  519   }
  520 
  521 The ``@links`` parameter is a dictionary indexed by
  522 relationships. Each relationship is a list of one or more full link
  523 json objects. Above we have link relations to move to the next
  524 page. If we weren't at the first page, there would be a ``prev``
  525 relation to move to the previous page. Also we have a self relation
  526 (which is missing the @page_index, hence we are at page 1) that can be
  527 used to get the same page again.
  528 
  529 Note that the server may choose to limit the number of returned
  530 entries in the collection as a DOS prevention measure. As a result
  531 clients must be prepared to handle the incomplete response and request
  532 the next URL to retrieve all of the entries.
  533 
  534 Field embedding and verbose output
  535 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  536 
  537 In collections, you can specify what fields should be embedded in the
  538 returned data. There are some shortcuts provided using the
  539 ``@verbose`` parameter. All the examples in this section are for a GET
  540 operation on ``https://.../rest/data/issue``.
  541 
  542 .. list-table:: Query Parameters Examples
  543   :header-rows: 1
  544   :widths: 20 80
  545 
  546   * - Query parameter
  547     - Explanation
  548   * - ``@verbose=0``
  549     - each item in the collection has its "id" property displayed
  550       and a link with the URL to retrieve the item.
  551   * - ``@verbose=1``
  552     - for collections this output is the same as ``@verbose=0``. This
  553       is the default.
  554   * - ``@verbose=2``
  555     - each item in the collection includes the "label" property in
  556       addition to "id" property and a link for the item.
  557       This is useful as documented below in "Searches and selection"_.
  558   * - ``@verbose=3``
  559     - will display the content property of messages and files. Note
  560       warnings about this below. Using this for collections is
  561       discouraged as it is slow and produces a lot of data.
  562   * - ``@fields=status,title``
  563     - will return the ``status``  and ``title`` fields for the
  564       displayed issues. It is added to the fields returned by the
  565       @verbose parameter. Protected properties
  566       can be included in the list and will be returned.
  567 
  568 In addition collections support the ``@fields`` parameter which is a
  569 colon or comma separated list of fields to embed in the response. For
  570 example ``https://.../rest/data/issue?@verbose=2`` is the same as:
  571 ``https://.../rest/data/issue?@fields=title`` since the label property
  572 for an issue is its title.
  573 The @fields option supports transitive properties, e.g.
  574 ``status.name``. The transitive property may not include multilinks in
  575 the path except for the last component. So ``messages.author`` is not
  576 allowed because ``messages`` is a multilink while ``messages`` alone
  577 would be allowed.  You can use both ``@verbose`` and
  578 ``@fields`` to get additional info. For example
  579 ``https://.../rest/data/issue?@verbose=2&@fields=status`` returns::
  580 
  581 
  582   {
  583       "data": {
  584 	  "collection": [
  585 	      {
  586 		  "link": "https://.../rest/data/issue/1",
  587 		  "title": "Welcome to the tracker START HERE",
  588 		  "id": "1",
  589 		  "status": {
  590 		      "link": "https://.../rest/data/status/1",
  591 		      "id": "1",
  592 		      "name": "new"
  593 		  }
  594 	      },
  595     ...
  596   }
  597 
  598 the format of the status field (included because of
  599 ``@fields=status``) includes the label for the status. This is due to
  600 inclusion of ``@verbose=2``. Without verbose you would see::
  601 
  602   {
  603       "data": {
  604 	  "collection": [
  605 	      {
  606 		  "link":
  607       "https://.../rest/data/issue/1",
  608 		  "id": "1",
  609 		  "status": {
  610 		      "link":
  611       "https://.../rest/data/status/1",
  612 		      "id": "1"
  613 		  }
  614 	      },
  615      ...
  616   }
  617 
  618 Note that the ``link`` field that is returned doesn't exist in the
  619 database. It is a construct of the rest interface. This means that you
  620 can not set ``@fields=link`` and get the link property included in the
  621 output.
  622 
  623 Also using ``@fields=@etag`` will not work to retrieve the etag for
  624 items in the collection.
  625 
  626 See the `Searches and selection`_ section for the use cases supported
  627 by these features.
  628 
  629 Getting Message and Files Content
  630 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  631 
  632 You can retreive a message with a url like
  633 ``https://.../demo/rest/data/msg/11``. This returns something like::
  634 
  635   {
  636      "data": {
  637         "id": "11",
  638         "type": "msg",
  639         "link": "https://.../demo/rest/data/msg/11",
  640 	"attributes": {
  641             "author": {
  642                 "id": "5",
  643                 "link": "https://.../demo/rest/data/user/5"
  644             },
  645             "content": {
  646                 "link": "https://.../demo/msg11/"
  647             },
  648             "date": "2017-10-30.00:53:15",
  649             "files": [],
  650             "inreplyto": null,
  651             "messageid": "<1509324807.14.0.296813919751.issue3@localhost>",
  652             "messagetype": {
  653                 "id": "1",
  654                 "link": "https://.../demo/rest/data/msgtype/1"
  655             },
  656             "recipients": [
  657                 {
  658                     "id": "1",
  659                     "link": "https://.../demo/rest/data/user/1"
  660                 },
  661                 {
  662                     "id": "3",
  663                     "link": "https://.../demo/rest/data/user/3"
  664                 },
  665                 {
  666                     "id": "4",
  667                     "link": "https://.../demo/rest/data/user/4"
  668                 }
  669             ],
  670             "subject": null,
  671             "summary": "of has to who. or of account give because the",
  672         },
  673         "@etag": "\"584f82231079e349031bbb853747df1c\""
  674      }
  675   }
  676 
  677 To retreive the content, you can use the content link property:
  678 ``https://.../demo/msg11/``. The trailing / is required. Without the
  679 /, you get a web page that includes metadata about the message. With
  680 the slash you get a text/plain (in most cases) data stream.
  681 
  682 Also you can use the url:
  683 
  684 and the content property (if the data is utf-8 compatible) now looks
  685 like::
  686 
  687    ...
  688    "author": {
  689                "id": "5",
  690                "link": "https://.../demo/rest/data/user/5"
  691              },
  692    "content": "of has to who pleasure. or of account give because the
  693        reprehenderit\neu to quisquam velit, passage,
  694        was or toil BC quis denouncing quia\nexercise,
  695        veritatis et used voluptas I elit, a The...",
  696    "date": "2017-10-30.00:53:15",
  697    ...
  698    
  699 Lines are wrapped for display, content value is one really long
  700 line. If the data is not utf-8 compatible, you will get a link.
  701 
  702 Retrieving the contents of a file is similar. Performing a
  703 get on ``https://.../demo/rest/data/file/11`` returns::
  704 
  705   {
  706      "data": {
  707         "id": "11",
  708         "type": "file",
  709         "link": "https://.../demo/rest/data/file/11",
  710         "attributes": {
  711             "acl": null,
  712             "content": {
  713                 "link": "https://.../demo/file11/"
  714             },
  715             "name": "afile",
  716             "status": {
  717                 "id": "1",
  718                 "link": "https://.../demo/rest/data/filestatus/1"
  719             },
  720             "type": "image/vnd.microsoft.icon"
  721         },
  722         "@etag": "\"74276f75ef71a30a0cce62dc6a8aa1bb\""
  723      }
  724   }
  725 
  726 To download the file contents for this example you would
  727 perform an http GET using: ``https://.../demo/file11/``. The trailing
  728 / is required. You will receive a response of type
  729 application/octet-stream.
  730 
  731 If you perform a get on
  732 ``https://.../demo/rest/data/file/11?@verbose=3`` the content field
  733 above is displayed as (wrapped for display)::
  734 
  735   "content": "file11 is not text, retrieve using binary_content
  736               property. mdsum: bd990c0f8833dd991daf610b81b62316",
  737 	       
  738 
  739 You can use the `binary_content property`_ described below to
  740 retrieve an encoded copy of the data.
  741 
  742 Other query params
  743 ^^^^^^^^^^^^^^^^^^
  744 
  745 This table lists other supported parameters:
  746 
  747 .. list-table:: Query Parameters Examples
  748   :header-rows: 1
  749   :widths: 20 80
  750 
  751   * - Query parameter
  752     - Explanation
  753   * - ``@pretty=false``
  754     - by default json data is pretty printed to make it readable to
  755       humans. This eases testing and with compression enabled the
  756       extra whitespace doesn't bloat the returned payload excessively.
  757       You can disable pretty printing by using this query parameter.
  758       Note the default is true, so @pretty=true is not supported at
  759       this time.
  760 
  761 Using the POST method
  762 ^^^^^^^^^^^^^^^^^^^^^
  763 
  764 Only class links support the ``POST`` method for creation of new items
  765 of a class, e.g., a new issue via the ``/data/issue`` link. The post
  766 gets a dictionary of keys/values for the new item. It returns the same
  767 parameters as the GET method after successful creation.
  768 
  769 If you perform a get on an item with ``@verbose=0``, it is in the
  770 correct form to use as a the payload of a post.
  771 
  772 
  773 Safely Re-sending POST
  774 ^^^^^^^^^^^^^^^^^^^^^^
  775 
  776 POST is used to create new object in a class. E.G. a new issue.  One
  777 problem is that a POST may time out. Because it is not idempotent like
  778 a PUT or DELETE, retrying the interrupted POST may result in the
  779 creation of a duplicate issue.
  780 
  781 To solve this problem, a two step process inspired by the POE - Post
  782 Once Exactly spec:
  783 https://tools.ietf.org/html/draft-nottingham-http-poe-00 is provided.
  784 
  785 This mechanism returns a single use URL. POSTing to the URL creates
  786 a new object in the class.
  787 
  788 First we get the URL. Here is an example using curl::
  789 
  790   curl -u demo:demo -s -X POST -H "Referer: https://.../demo/" \
  791       -H "X-requested-with: rest" \
  792       -H "Content-Type: application/json" \
  793       --data '' \
  794       https://.../demo/rest/data/issue/@poe
  795 
  796 This will return a json payload like::
  797 
  798   {
  799     "data": {
  800         "expires": 1555266310.4457426,
  801         "link": "https://.../demo/rest/data/issue/@poe/vizl713xHtIzANRW9jPb3bWXePRzmehdmSXzEta1"
  802     }
  803   }
  804 
  805 The value of expires is a Unix timestamp in seconds. In this case it
  806 has the default lifetime of 30 minutes after the current time. Using
  807 the link more than 30 minutes into the future will cause a 400 error.
  808 
  809 Within 30 minutes, the link can be used to post an issue with the same
  810 payload that would normally be sent to:
  811 ``https://.../demo/rest/data/issue``.
  812 
  813 For example::
  814 
  815    curl -u demo:demo -s -X POST \
  816      -H "Referer: https://.../demo/" \
  817      -H "X-requested-with: rest" \
  818      -H "Content-Type: application/json"  \
  819      --data-binary '{ "title": "a problem" }' \
  820      https://.../demo/rest/data/issue/@poe/vizl713xHtIzANRW9jPb3bWXePRzmehdmSXzEta1
  821 
  822 returns::
  823 
  824   {
  825     "data": {
  826         "link": "https://.../demo/rest/data/issue/2280",
  827         "id": "2280"
  828     }
  829   }
  830 
  831 Once the @poe link is used and creates an issue, it becomes invalid
  832 and can't be used again.  Posting to it after the issue, or other
  833 object, is created, results in a 400 error [#poe_retry]_.
  834 
  835 Note that POE links are by restricted to the class that was used to
  836 get the link. So you can only create an issue using the link returned
  837 from ``rest/data/issue/@poe``. You can create a generic POE link by adding
  838 the "generic" field to the post payload::
  839 
  840   curl -u demo:demo -s -X POST -H "Referer: https://.../demo/" \
  841       -H "X-requested-with: rest" \
  842       --data 'lifetime=900&generic=1' \
  843       https://.../demo/rest/data/issue/@poe
  844 
  845 This will return a link under: ``https://.../demo/rest/data/issue/@poe``::
  846 
  847   {
  848     "data": {
  849         "expires": 1555268640.9606116,
  850         "link":
  851     "https://.../demo/rest/data/issue/@poe/slPrzmEq6Q9BTjvcKhfxMNZL4uHXjbHCidY1ludZ"
  852     }
  853   }
  854 
  855 You could use the link and change 'issue' to 'user' and it would work
  856 to create a user. Creating generic POE tokens is *not* recommended,
  857 but is available if a use case requires it.
  858 
  859 This example also changes the lifetime of the POE url.  This link has
  860 a lifetime of 15 minutes (900 seconds). Using it after 16 minutes will
  861 result in a 400 error. A lifetime up to 1 hour can be specified.
  862 
  863 POE url's are an optional mechanism. If:
  864 
  865 * you do not expect your client to retry a failed post,
  866 * a failed post is unlikely (e.g. you are running over a local lan),
  867 * there is a human using the client and who can intervene if a post
  868   fails
  869 
  870 you can use the url ``https://.../demo/data/<class>``. However if you
  871 are using this mechanism to automate creation of objects and will
  872 automatically retry a post until it succeeds, please use the POE
  873 mechanism.
  874 
  875 .. [#poe_retry] As a future enhancement, performing a POST to the POE
  876         link soon after it has been used to create an object will
  877         change. It will not return a 400 error. It will will trigger a
  878         301 redirect to the url for the created object. After some
  879         period of time (maybe a week) the POE link will be removed and
  880         return a 400 error. This is meant to allow the client (a time
  881         limited way) to retrieve the created resource if the
  882         response was lost.
  883 
  884 Other Supported Methods for Collections
  885 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  886 
  887 Supports the ``OPTIONS`` method for determining which methods are
  888 allowed on a given endpoint.
  889 
  890 Does not support PUT, DELETE or PATCH.
  891 
  892 /data/\ *class*/\ *id* item
  893 ===========================
  894 
  895 When performing the ``GET`` method on an item
  896 (e.g. ``/data/issue/42``), a ``link`` attribute contains the link to
  897 the item, ``id`` contains the id, ``type`` contains the class name
  898 (e.g. ``issue`` in the example) and an ``etag`` property can be used
  899 to detect modifications since the last query.
  900 
  901 Individual properties of the item are returned in an ``attributes``
  902 dictionary. The properties returned depend on the permissions of the
  903 account used for the query.
  904 
  905 By default all (visible to the current user) attributes/properties are
  906 returned. You can limit this by using the ``@fields`` query parameter
  907 similar to how it is used in collections. This way you can only return
  908 the fields you are interested in reducing network load as well as
  909 memory and parsing time on the client side. Or you can add additional
  910 transitive properties.  By default protected
  911 properties (read only in the database) are not listed. This
  912 makes it easier to submit the attributes from a
  913 ``@verbose=0`` query using PUT. To include protected properties
  914 in the output of a GET add the query parameter
  915 ``@protected=true`` to the query and attributes like: actor,
  916 created, creator and activity will be include in the result.
  917 
  918 Link and Multilink properties are displayed as a dictionary with a
  919 ``link`` and an ``id`` property by default. This is controlled by the
  920 ``@verbose`` attribute which is set to 1 by default. If set to 0, only
  921 the id is shown for Link and Multilink attributes. In this form, the
  922 data can be modified and sent back using ``PUT`` to change the item.
  923 If set to 2, the label property (usually ``name`` e.g. for status) is
  924 also put into the dictionary.  Content properties of message and file
  925 object are by default also shown as a dictionary with a sole link
  926 attribute. The link is the download link for the file or message. If
  927 @verbose is >= 3, the content property is shown in json as a (possibly
  928 very long) string. Currently the json serializer cannot handle files
  929 not properly utf-8 encoded, so specifying @verbose=3 for files is
  930 currently discouraged.
  931 
  932 An example of returned values::
  933 
  934   {
  935       "data": {
  936 	  "type": "issue",
  937 	  "@etag": "\"f15e6942f00a41960de45f9413684591\"",
  938 	  "link": "https://.../rest/data/issue/23",
  939 	  "attributes": {
  940 	      "keyword": [],
  941 	      "messages": [
  942 		  {
  943 		      "link": "https://.../rest/data/msg/375",
  944 		      "id": "375"
  945 		  },
  946 		  {
  947 		      "link": "https://.../rest/data/msg/376",
  948 		      "id": "376"
  949 		  },
  950 		  ...
  951 	      ],
  952 	      "files": [],
  953 	      "status": {
  954 		  "link": "https://.../rest/data/status/2",
  955 		  "id": "2"
  956 	      },
  957 	      "title": "This is a title title",
  958 	      "superseder": [],
  959 	      "nosy": [
  960 		  {
  961 		      "link": "https://.../rest/data/user/4",
  962 		      "id": "4"
  963 		  },
  964 		  {
  965 		      "link": "https://.../rest/data/user/5",
  966 		      "id": "5"
  967 		  }
  968 	      ],
  969 	      "assignedto": null,
  970 	  },
  971 	  "id": "23"
  972       }
  973   }
  974 
  975 Retrieve item using key value
  976 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  977 
  978 If the class has a key attribute, e.g. the 'status' class in the
  979 classic tracker, it can be used to retrieve the item.
  980 
  981 You can get an individual status by specifying the key-attribute value
  982 e.g. ``/data/status/name=closed``. Note that ``name`` in this example
  983 must be the key-attribute of the class.  A short-form (which might not
  984 be supported in future version of the API) is to specify only the
  985 value, e.g. ``/data/status/closed``.  This short-form only works when
  986 you're sure that the key of the class is not numeric. E.G. if the name
  987 was "7", /data/status/7 would return the status with id 7 not the
  988 status with name "7". To get the status with name 7, you must use
  989 the long form /data/status/name=7
  990 
  991 The long-form (with ``=``) is different from a query-parameter like
  992 ``/data/status?name=closed`` which would find all stati (statuses)
  993 that have ``closed`` as a substring.
  994 
  995 Dealing with Messages and Files
  996 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  997 
  998 Using the requests library you can upload a file using::
  999 
 1000      d = dict (name = filename, content = content, type = content_type)
 1001      j = self.post ('file', data = d)
 1002 
 1003 Instead of specifying json = dictionary we specify data = dictionary
 1004 as shown above. (We believe) this encodes the contents using
 1005 application/x-www-form-urlencoded which is not optimal for large files
 1006 due to the encoding overhead.
 1007 
 1008 The requests library can use multipart/form-data which is more
 1009 efficient for large files. To do this specify both, files= *and* data=
 1010 parameters, e.g.::
 1011 
 1012   # A binary string that can't be decoded as unicode
 1013   url = 'https://.../demo/rest/data/'
 1014   content = open ('random-junk', 'rb').read ()
 1015   fname   = 'a-bigger-testfile'
 1016   d = dict(name = fname, type='application/octet-stream')
 1017   c = dict (content = content)
 1018   r = session.post (url + 'file', files = c, data = d)
 1019 
 1020 Curl can be used to post a file using multipart/form-data with::
 1021 
 1022    curl -u demo:demo -s -X POST -H "Referer: https://.../demo/" \
 1023       -H "X-requested-with: rest" \
 1024       -F "name=afile" -F "type=image/vnd.microsoft.icon" \
 1025       -F "content=@doc/roundup-favicon.ico" \
 1026       https://.../demo/rest/data/file
 1027 
 1028 the file is located at doc/roundup-favicon.ico. These calls will
 1029 return something like::
 1030 
 1031   {
 1032       "data": {
 1033           "id": "12",
 1034           "link": "https://.../demo/rest/data/file/12"
 1035        }
 1036   }
 1037  
 1038 
 1039 Other Supported Methods for Items
 1040 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 1041 
 1042 The method ``PUT`` is allowed on individual items, e.g.
 1043 ``/data/issue/42`` On success it returns the same parameters as the
 1044 respective ``GET`` method. Note that for ``PUT`` an Etag has to be
 1045 supplied, either in the request header or as an @etag parameter. An
 1046 example::
 1047 
 1048   curl -u admin:admin -X PUT \
 1049      --header 'Referer: https://example.com/demo/' \
 1050      --header 'X-Requested-With: rest' \
 1051      --header "Content-Type: application/json" \
 1052      --header "Accept: application/json" \
 1053      --header 'If-Match: "dd41f02d6f8b4c34b439fc712b522fb3"' \
 1054      --data '{ "nosy": [ "1", "5" ] }' \
 1055      "https://example.com/demo/rest/data/issue/23"
 1056 
 1057   {
 1058       "data": {
 1059 	  "attribute": {
 1060 	      "nosy": [
 1061 		  "1",
 1062 		  "5"
 1063 	      ]
 1064 	  },
 1065 	  "type": "issue",
 1066 	  "link":
 1067 	  "https://example.com/demo/rest/data/issue/23",
 1068 	  "id": "23"
 1069       }
 1070   }
 1071 
 1072 If the above command is repeated with the data attribute::
 1073 
 1074      --data '{ "nosy": [ "1", "5" ], "title": "This is now my title" }'
 1075 
 1076 this is returned::
 1077 
 1078   {
 1079       "data": {
 1080 	  "attribute": {
 1081 	      "title": "This is now my title"
 1082 	  },
 1083 	  "type": "issue",
 1084 	  "link":
 1085       "https://.../demo/rest/data/issue/23",
 1086 	  "id": "23"
 1087       }
 1088   }
 1089 
 1090 Note that nosy is not in the attributes returned. It is the same as
 1091 before, so no change has happened and it is not reported.
 1092 Changing both nosy and title::
 1093 
 1094   curl -u admin:admin -X PUT \
 1095     --header 'Referer: https://.../' \
 1096     --header 'X-Requested-With: rest' \
 1097     --header "Content-Type: application/json" \
 1098     --header "Accept: application/json" \
 1099     --header 'If-Match: "8209add59a79713d64f4d1a072aef740"' \
 1100     --data '{ "nosy": [ "4", "5" ], "title": "This is now my new title"  }' \
 1101    "https://.../demo/rest/data/issue/23"
 1102 
 1103 which returns both title and nosy attributes::
 1104 
 1105     {
 1106 	"data": {
 1107 	    "attribute": {
 1108 		"title": "This is now my new title",
 1109 		"nosy": [
 1110 		    "4",
 1111 		    "5"
 1112 		]
 1113 	    },
 1114 	    "type": "issue",
 1115 	    "link":
 1116 	    "https://.../demo/rest/data/issue/23",
 1117 	    "id": "23"
 1118 	}
 1119     }
 1120 
 1121 Note that mixing url query parameters with payload submission doesn't
 1122 work. So using::
 1123 
 1124   https://.../rest/data/issue/23?@pretty=false
 1125 
 1126 doesn't have the desired effect. However it can be put in the data
 1127 payload::
 1128 
 1129  curl -u admin:admin ...
 1130    --data '{ "nosy": [ "4", "5" ], "title": "...", "@pretty": "false"  }'
 1131 
 1132 produces::
 1133 
 1134    {"data": {"attribute": {...}, "type": "issue",
 1135      "link": "https://...", "id": "23"}}
 1136 
 1137 the lines are wrapped for display purposes, in real life it's one long
 1138 line.
 1139 
 1140 The method ``DELETE`` is allowed on items, e.g., ``/data/issue/42``
 1141 and will retire (mark as deleted) the respective item. On success it
 1142 will only return a status code. The item is still available if
 1143 accessed directly by its item url. The item will not show up in
 1144 searches where it would have been matched if not retired.
 1145 
 1146 Finally the ``PATCH`` method can be applied to individual items, e.g.,
 1147 ``/data/issue/42``. This method gets an operator ``@op=<method>``
 1148 where ``<method>`` is one of ``add``, ``replace``, ``remove``. For
 1149 items, an additional operator ``action`` is supported. If no operator
 1150 is specified, the default is ``replace``. The first three operators
 1151 are self explanatory. For an ``action`` operator an ``@action_name``
 1152 and optional ``@action_argsXXX`` parameters have to be
 1153 supplied. Currently there are only two actions, neither has args,
 1154 namely ``retire`` and ``restore``. The ``retire`` action on an item is
 1155 the same as a ``DELETE`` method, it retires the item. The ``restore``
 1156 action is the inverse of ``retire``, the item is again visible.  On
 1157 success the returned value is the same as the respective ``GET``
 1158 method. An example to add a user to the nosy list of an item is::
 1159 
 1160   curl -u admin:admin -p -X PATCH \
 1161      --header "Content-Type: application/x-www-form-urlencoded" \
 1162      --header "Accept: application/json" \
 1163      --header 'If-Match: "c6e2d81019acff1da7a2da45f93939bd"' \
 1164      --data-urlencode '@op=add' \
 1165      --data 'nosy=3' \
 1166      "https://.../rest/data/issue/23"
 1167 
 1168 which returns::
 1169 
 1170   {
 1171       "data": {
 1172 	  "attribute": {
 1173 	      "nosy": [
 1174 		  "3",
 1175 		  "4"
 1176 	      ]
 1177 	  },
 1178 	  "type": "issue",
 1179 	  "link": "https://.../rest/data/issue/23",
 1180 	  "id": "23"
 1181       }
 1182   }
 1183 
 1184 Note that the changed values are returned so you can update
 1185 internal state in your app with the new data.
 1186 
 1187 The ``GET`` method on an item (e.g. ``/data/issue/43``) returns an
 1188 ETag in the http header *and* the ``@etag`` value in the json
 1189 payload. When modifying a property via ``PUT`` or ``PATCH`` or
 1190 ``DELETE`` the etag value for the item must be supplied using an
 1191 ``If-Match`` header. If you are using ``PUT`` or ``PATCH`` an
 1192 ``@etag`` value can be supplied in the payload in place of the
 1193 ``If-Match`` header.
 1194 
 1195 /data/\ *class*/\ *id*/\ *property* field
 1196 =========================================
 1197 
 1198 A ``GET`` method on a property (e.g. ``/data/issue/42/title``) returns the
 1199 link, an ``@etag``, the type of the property (e.g. "<type str>") the id
 1200 of the item and the content of the property in ``data``.
 1201 
 1202 For example::
 1203 
 1204   {
 1205       "data": {
 1206 	  "link": "https://.../rest/data/issue/22/title",
 1207 	  "data": "I need Broken PC",
 1208 	  "type": "<class 'str'>",
 1209 	  "id": "22",
 1210 	  "@etag": "\"370510512b2d8fc3f98aac3d762cc7b1\""
 1211       }
 1212   }
 1213 
 1214 
 1215 All endpoints support an ``OPTIONS`` method for determining which
 1216 methods are allowed on a given endpoint.
 1217 
 1218 Message and File Content
 1219 ^^^^^^^^^^^^^^^^^^^^^^^^
 1220 
 1221 Messages and files have content properties. If the data is utf-8
 1222 compatible (e.g. an email message) you can retrieve it with
 1223 rest/data/msg/11/content to obtain::
 1224 
 1225   {
 1226     "data": {
 1227     "id": "11",
 1228     "type": "<class 'str'>",
 1229     "link": "https://.../demo/rest/data/msg/11/content",
 1230     "data": "of has to who pleasure. or of account give because the reprehenderit\neu to quisquam velit, passage, was or...",
 1231             "@etag": "\"584f82231079e349031bbb853747df1c\""
 1232     }
 1233   }		
 1234 
 1235 (the content property is wrapped for display, it is one long line.)
 1236 
 1237 .. _binary_content property:
 1238 
 1239 If the data is not representable in utf-8, you need to use the
 1240 binary_content
 1241 property. E.G. ``https://.../demo/rest/data/file/11/binary_content``
 1242 returns::
 1243 
 1244   {
 1245      "data": {
 1246         "id": "11",
 1247         "type": "<class 'bytes'>",
 1248         "link": "https://.../demo/rest/data/file/11/binary_content",
 1249         "data": "b'\\x00\\x00\\x01\\x00\\x01...\\xec?\\x00\\x00'",
 1250         "@etag": "\"74276f75ef71a30a0cce62dc6a8aa1bb\""
 1251      }
 1252   }
 1253 
 1254 (data field elided for display). You can also receive the file content
 1255 as a data stream rather than encoded. See `Getting Message and Files
 1256 Content`_.
 1257 
 1258 The data is a json encoded hexidecimal representation of the data.
 1259 
 1260 
 1261 Other Supported Methods for fields
 1262 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 1263 
 1264 The method ``PUT`` is allowed on a property e.g.,
 1265 ``/data/issue/42/title``. On success it returns the same parameters as
 1266 the respective ``GET`` method. Note that for ``PUT`` an Etag has to be
 1267 supplied, either in the request header or as an @etag parameter.
 1268 Example using multipart/form-data rather than json::
 1269 
 1270   curl -vs -u provisional:provisional -X PUT \
 1271    --header "Accept: application/json" \
 1272    --data "data=Provisional" \
 1273    --header "If-Match: 079eba599152f3eed00567e23258fecf" \
 1274    --data-urlencode "@etag=079eba599152f3eed00567e23258fecf" \
 1275     "https://.../rest/data/user/5/realname"
 1276 
 1277 This example updates a leadtime field that is declared as an interval
 1278 type::
 1279 
 1280   curl -vs -u demo:demo -X PUT \
 1281     --header "Accept: application/json" \
 1282     --header 'Content-Type: application/json' \
 1283     --header "Referer: https://.../" \
 1284     --header "x-requested-with: rest" \
 1285     --header 'If-Match: "e2e6cc43c3475a4a3d9e5343617c11c3"' \
 1286     --data '{"leadtime": "2d" }'  \
 1287     "https://.../rest/data/issue/10"
 1288 
 1289 It is also possible to call ``DELETE`` on a
 1290 property of an item, e.g., ``/data/issue/42/nosy`` to delete the nosy
 1291 list. The same effect can be achieved with a ``PUT`` request and an
 1292 empty new value. This may fail if the property is required.
 1293 
 1294 The ``PATCH`` method can be applied to properties, e.g.,
 1295 ``/data/issue/42/title``.  This method gets an operator
 1296 ``@op=<method>`` where ``<method>`` is one of ``add``, ``replace``,
 1297 ``remove``. If no operator is specified, the default is ``replace``
 1298 which is the same as performing a PUT on the field url. ``add`` and
 1299 ``remove`` allow adding and removing values from MultiLink
 1300 properties. This is easier than having to rewrite the entire value for
 1301 the field using the ``replace`` operator or doing a PUT to the field.
 1302 On success the returned value is the same as the respective ``GET``
 1303 method.
 1304 
 1305 The ``GET`` method on an item (e.g. ``/data/issue/43``) returns an
 1306 ETag in the http header *and* the ``@etag`` value in the json
 1307 payload. When modifying a property via ``PUT`` or ``PATCH`` or
 1308 ``DELETE`` the etag value for the item must be supplied using an
 1309 ``If-Match`` header. If you are using ``PUT`` or ``PATCH`` an
 1310 ``@etag`` value can be supplied in the payload in place of the
 1311 ``If-Match`` header.
 1312 
 1313 Tunneling Methods via POST
 1314 ==========================
 1315 
 1316 If you are working through a proxy and unable to use http method like
 1317 PUT, PATCH or DELETE you can use POST to perform the action. To tunnel
 1318 an action through POST, send the ``X-HTTP-METHOD-OVERRIDE`` header
 1319 with a value of DELETE or other capitalized HTTP verb. The body of the
 1320 POST should be what you would send if you were using the method
 1321 without tunneling.
 1322 
 1323 Examples and Use Cases
 1324 ----------------------
 1325 
 1326 sample python client
 1327 ====================
 1328 
 1329 The client uses the python ``requests`` library for easier interaction
 1330 with a REST API supporting JSON encoding::
 1331 
 1332 
 1333         >>> import requests
 1334         >>> u = 'http://user:password@tracker.example.com/demo/rest/data/'
 1335         >>> s = requests.session()
 1336 	>>> session.auth = ('admin', 'admin')
 1337         >>> r = s.get(u + 'issue/42/title')
 1338         >>> if r.status_code != 200:
 1339         ...     print("Failed: %s: %s" % (r.status_code, r.reason))
 1340         ...     exit(1)
 1341         >>> print (r.json() ['data']['data']
 1342         TEST Title
 1343         >>> h = {'X-Requested-With': 'rest', 'Referer': 'http://tracker.example.com/demo/'}
 1344         >>> r = s.post (u + 'issue', data = dict (title = 'TEST Issue'), headers=h)
 1345         >>> if not 200 <= r.status_code <= 201:
 1346         ...     print("Failed: %s: %s" % (r.status_code, r.reason))
 1347         ...     exit(1)
 1348         >>> print(r.json())
 1349 
 1350 Retire/Restore::
 1351         >>> r = s.delete (u + 'issue/42')
 1352         >>> print (r.json())
 1353         >>> r = s.get (u + 'issue/42')
 1354         >>> etag = r.headers['ETag']
 1355         >>> print("ETag: %s" % etag)
 1356         >>> etag = r.json()['data']['@etag']
 1357         >>> print("@etag: %s" % etag)
 1358         >>> h = {'If-Match': etag, 
 1359         ...   'X-Requested-With': 'rest',
 1360         ...   'Referer': 'http://tracker.example.com/demo/'}
 1361         >>> d = {'@op:'action', '@action_name':'retire'}
 1362         >>> r = s.patch(u + 'issue/42', data = d, headers = h)
 1363         >>> print(r.json())
 1364         >>> d = {'@op:'action', '@action_name':'restore'}
 1365         >>> r = s.patch(u + 'issue/42', data = d, headers = h)
 1366         >>> print(r.json())
 1367 
 1368 Note the addition of headers for: x-requested-with and referer. This
 1369 allows the request to pass the CSRF protection mechanism. You may need
 1370 to add an Origin header if this check is enabled in your tracker's
 1371 config.ini (look for csrf_enforce_header_origin).
 1372 
 1373 A similar curl based retire example is to use::
 1374 
 1375        curl -s -u admin:admin \
 1376         -H "Referer: https://tracker.example.com/demo/" \
 1377 	-H "X-requested-with: rest"  \
 1378 	-H "Content-Type: application/json" \
 1379 	https://tracker.example.com/demo/rest/data/status/1
 1380 
 1381 to get the etag manually. Then insert the etag in the If-Match header
 1382 for this retire example::
 1383 
 1384      curl -s -u admin:admin \
 1385         -H "Referer: https://tracker.example.com/demo/" \
 1386 	-H "X-requested-with: rest"  \
 1387 	-H "Content-Type: application/json" \
 1388 	-H 'If-Match: "a502faf4d6b8e3897c4ecd66b5597571"' \
 1389 	--data-raw '{ "@op":"action", "@action_name": "retire" }'\
 1390 	-X PATCH \
 1391 	https://tracker.example.com/demo/rest/data/status/1
 1392 
 1393 and restore::
 1394 
 1395      curl -s -u admin:admin \
 1396         -H "Referer: https://tracker.example.com/demo/" \
 1397 	-H "X-requested-with: rest"  \
 1398 	-H "Content-Type: application/json" \
 1399 	-H 'If-Match: "a502faf4d6b8e3897c4ecd66b5597571"' \
 1400 	--data-raw '{ "@op":"action", "@action_name": "restore" }'\
 1401 	-X PATCH \
 1402 	https://tracker.example.com/demo/rest/data/status/1
 1403 
 1404 
 1405 Searches and selection
 1406 ======================
 1407 
 1408 One difficult interface issue is selection of items from a long list.
 1409 Using multi-item selects requires loading a lot of data (e.g. consider
 1410 a selection tool to select one or more issues as in the classic
 1411 superseder field).
 1412 
 1413 This can be made easier using javascript selection tools like select2,
 1414 selectize.js, chosen etc. These tools can query a remote data provider
 1415 to get a list of items for the user to select from.
 1416 
 1417 Consider a multi-select box for the superseder property.  Using
 1418 selectize.js (and jquery) code similar to::
 1419 
 1420     $('#superseder').selectize({
 1421 	valueField: 'id',
 1422 	labelField: 'title',
 1423 	searchField: 'title', ...
 1424 	load: function(query, callback) {
 1425 		if (!query.length) return callback();
 1426 		$.ajax({
 1427 			url: '.../rest/data/issue?@verbose=2&title='
 1428 			    + encodeURIComponent(query),
 1429 			type: 'GET',
 1430 			error: function() {callback();},
 1431 			success: function(res) {
 1432 			  callback(res.data.collection);}
 1433 
 1434 Sets up a box that a user can type the word "request" into. Then
 1435 selectize.js will use that word to generate an ajax request with the
 1436 url: ``.../rest/data/issue?@verbose=2&title=request``
 1437 
 1438 This will return data like::
 1439 
 1440   {
 1441     "data": {
 1442     "@total_size": 440,
 1443     "collection": [
 1444       {
 1445 	  "link": ".../rest/data/issue/8",
 1446 	  "id": "8",
 1447 	  "title": "Request for Power plugs"
 1448       },
 1449       {
 1450 	  "link": ".../rest/data/issue/27",
 1451 	  "id": "27",
 1452 	  "title": "Request for foo"
 1453       },
 1454   ...
 1455 
 1456 selectize.js will look at these objects (as passed to
 1457 callback(res.data.collection)) and create a select list from the each
 1458 object showing the user the labelField (title) for each object and
 1459 associating each title with the corresponding valueField (id). The
 1460 example above has 440 issues returned from a total of 2000
 1461 issues. Only 440 had the word "request" somewhere in the title greatly
 1462 reducing the amount of data that needed to be transferred.
 1463 
 1464 Similar code can be set up to search a large list of keywords using::
 1465 
 1466   .../rest/data/keyword?@verbose=2&name=some
 1467 
 1468 which would return: "some keyword" "awesome" "somebody" making
 1469 selections for links and multilinks much easier.
 1470 
 1471 A get on a collection endpoint can include other properties. Why do we
 1472 want this?  Selectize.js can set up option groups (optgroups) in the
 1473 select pulldown. So by including status in the returned data using
 1474 a url like ``https://.../rest/data/issue?@verbose=2&@fields=status``
 1475 we get::
 1476 
 1477    {
 1478      "link": "https://.../rest/data/issue/1001",
 1479      "title": "Request for Broken PC",
 1480      "id": "1001",
 1481      "status": {
 1482         "link": "https://.../rest/data/status/6",
 1483         "id": "6",
 1484         "name": "resolved"
 1485       }
 1486    }
 1487 
 1488 a select widget like::
 1489 
 1490   === New ===
 1491   A request
 1492   === Open ===
 1493   Request for bar
 1494   Request for foo
 1495 
 1496 etc. can be generated. Also depending on the javascript library, other
 1497 fields can be used for subsearch and sorting.
 1498 
 1499 
 1500 Programming the REST API
 1501 ------------------------
 1502 
 1503 You can extend the rest api for a tracker. This describes how to add
 1504 new rest end points. At some point it will also describe the rest.py
 1505 structure and implementation.
 1506 
 1507 Adding new rest endpoints
 1508 =========================
 1509 
 1510 Add or edit the file interfaces.py at the root of the tracker
 1511 directory.
 1512 
 1513 In that file add::
 1514 
 1515     from roundup.rest import Routing, RestfulInstance, _data_decorator
 1516     from roundup.exceptions import Unauthorised
 1517 
 1518     class RestfulInstance:
 1519 
 1520 	@Routing.route("/summary2")
 1521 	@_data_decorator
 1522 	def summary2(self, input):
 1523 	    result = { "hello": "world" }
 1524 	    return 200, result
 1525 
 1526 will make a new endpoint .../rest/summary2 that you can test with::
 1527 
 1528     $ curl -X GET .../rest/summary2
 1529     {
 1530         "data": {
 1531             "hello": "world"
 1532         }
 1533     }
 1534 
 1535 Similarly appending this to interfaces.py after summary2::
 1536 
 1537     # handle more endpoints
 1538         @Routing.route("/data/<:class_name>/@schema", 'GET')
 1539         def get_element_schema(self, class_name, input):
 1540 	    result = { "schema": {} }
 1541 	    uid = self.db.getuid ()
 1542 	    if not self.db.security.hasPermission('View', uid, class_name) :
 1543 		raise Unauthorised('Permission to view %s denied' % class_name)
 1544 
 1545 	    class_obj = self.db.getclass(class_name)
 1546 	    props = class_obj.getprops(protected=False)
 1547 	    schema = result['schema']
 1548 
 1549 	    for prop in props:
 1550 		schema[prop] = { "type": repr(class_obj.properties[prop]) }
 1551 
 1552 	    return result
 1553 
 1554 ..
 1555   the # comment in the example is needed to preserve indention under Class.
 1556 
 1557 returns some data about the class::
 1558 
 1559     $ curl -X GET .../rest/data/issue/@schema
 1560     {   
 1561 	"schema": {
 1562 	    "keyword": {
 1563 		"type": "<roundup.hyperdb.Multilink to \"keyword\">"
 1564 	    },
 1565 	    "title": {
 1566 		"type": "<roundup.hyperdb.String>"
 1567 	    },
 1568 	    "files": {
 1569 		"type": "<roundup.hyperdb.Multilink to \"file\">"
 1570 	    },
 1571 	    "status": {
 1572 		"type": "<roundup.hyperdb.Link to \"status\">"
 1573 	    }, ...
 1574 	}
 1575     }
 1576 
 1577 
 1578 Adding other endpoints (e.g. to allow an OPTIONS query against
 1579 ``/data/issue/@schema``) is left as an exercise for the reader.
 1580 
 1581 Redefine/move rest endpoints
 1582 ============================
 1583 
 1584 In addition to adding new endpoints, you can redefine existing
 1585 endpoints. Adding this as described above::
 1586 
 1587     @Routing.route("/summary")
 1588     @_data_decorator
 1589     def summary2(self, input):
 1590         result = { "hello": "world" }
 1591         return 200, result
 1592 
 1593 will return::
 1594 
 1595   {
 1596     "data": {
 1597             "hello": "world"
 1598             }
 1599   }
 1600 
 1601 
 1602 In addition to overriding existing endpoints, you can move existing
 1603 endpoints to new locations. Adding::
 1604 
 1605     @Routing.route("/data2/<:classname>")
 1606     def get_collection2(self, classname, input):
 1607         """ Remap existing function in rest.py to a new endpoint
 1608 
 1609             Existing function is decorated with:
 1610 
 1611                   @Routing.route("/data/<:classname>")
 1612                   @_data_decorator
 1613 
 1614             so we need to drop @_data_decorator from this function since
 1615             we can't apply @_data_decorator twice.
 1616         """
 1617         return self.get_collection(classname, input)
 1618 
 1619 will make the response at /rest/data2/<class> be the same as what is
 1620 normally at /rest/data/<class>.
 1621 
 1622 
 1623 Controlling Access to Backend Data
 1624 ==================================
 1625 
 1626 Roundup's schema is the primary access control mechanism. Roles and
 1627 Permissions provide the ability to carefully control what data can be
 1628 seen.
 1629 
 1630 However the templating system can access the hyperdb directly which
 1631 allows filtering to happen with admin privs escaping the standard
 1632 permissions scheme. For example access to a user's roles should be
 1633 limited to the user (read only) and an admin.  If you have customized
 1634 your schema to implement `Restricting the list of
 1635 users that are assignable to a task <customizing.html#restricting-the-list-of-users-that-are-assignable-to-a-task>`__
 1636 so that only users with a
 1637 Developer role are allowed to be assigned to an issue, a rest end
 1638 point must be added to provide a view that exposes users with this
 1639 permission.
 1640 
 1641 Using the normal ``/data/user?roles=Developer`` will return all the
 1642 users in the system unless you are an admin user because most users
 1643 can't see the roles. Building on the `Adding new rest endpoints`_
 1644 section this code adds a new endpoint `/data/@permission/Developer`
 1645 that returns a list of users with the developer role::
 1646 
 1647     from roundup.rest import Routing, RestfulInstance
 1648     from cgi import MiniFieldStorage
 1649 
 1650     class RestfulInstance(object):
 1651 
 1652 	@Routing.route("/data/@permission/Developer")
 1653 	def get_role_Developer(self, input):
 1654 	    '''An endpoint to return a list of users with Developer
 1655 	       role who can be assigned to an issue.
 1656 
 1657 	       It ignores attempt to search by any property except
 1658 	       username and realname. It also ignores the whole @fields
 1659 	       specification if it specifies a property the user
 1660 	       can't view. Other @ query params (e.g. @page... and
 1661 	       @verbose) are supported.
 1662 
 1663 	       It assumes admin access rights so that the roles property
 1664 	       of the user can be searched. This is needed if the roles
 1665 	       property is not searchable/viewable by normal users. A user
 1666 	       who can search roles can identify users with the admin
 1667 	       role. So it does not respond the same as a rest/data/users
 1668 	       search by a non-admin user.
 1669 	    '''
 1670 	    # get real user id
 1671 	    realuid=self.db.getuid()
 1672 
 1673 	    def allowed_field(fs):
 1674 		if fs.name in ['username', 'realname' ]:
 1675 		    # only allow search matches for these fields
 1676 		    return True
 1677 		elif fs.name in [ '@fields' ]:
 1678 		    for prop in fs.value.split(','):
 1679 			# if any property is unviewable to user, remove
 1680 			# @field entry. If they can't see it for the admin
 1681 			# user, don't let them see it for any user.
 1682 			if not self.db.security.hasPermission(
 1683 				'View', realuid, 'user', property=prop,
 1684 				itemid='1'):
 1685 			    return False
 1686 		    return True
 1687 		elif fs.name.startswith("@"):
 1688 		    # allow @page..., @verbose etc. 
 1689 		    return True
 1690 
 1691 		# deny all other url parmeters
 1692 		return False
 1693 
 1694 	    # Cleanup input.list to prevent user from probing roles
 1695 	    # or viewing things the user should not be able to view.
 1696 	    input.list[:] = [ fs for fs in input.list 
 1697 			      if allowed_field(fs) ]
 1698 
 1699 	    # Add the role filter required to implement the permission
 1700 	    # search
 1701 	    input.list.append(MiniFieldStorage("roles", "Developer"))
 1702 
 1703 	    # change user to acquire permission to search roles
 1704 	    self.db.setCurrentUser('admin') 
 1705 
 1706 	    # Once we have cleaned up the request, pass it to
 1707 	    # get_collection as though /rest/data/users?... has been called
 1708 	    # to get @verbose and other args supported.
 1709 	    return self.get_collection('user', input)
 1710 
 1711 Calling this with::
 1712 
 1713    curl 'http://example.com/demo/rest/data/@permission/Developer?@fields=realname&roles=Users&@verbose=2'
 1714    
 1715 produces output similar to::
 1716 
 1717     {
 1718 	"data": {
 1719 	    "collection": [
 1720 		{
 1721 		    "username": "agent",
 1722 		    "link": http://example.com/demo/rest/data/user/4",
 1723                     "realname": "James Bond",
 1724 		    "id": "4"
 1725 		}
 1726 	    ],
 1727 	    "@total_size": 1
 1728 	}
 1729     }
 1730 
 1731 assuming user 4 is the only user with the Developer role. Note that
 1732 the url passes the ``roles=User`` filter option which is silently
 1733 ignored.
 1734 
 1735 Changing Access Roles with JSON Web Tokens
 1736 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 1737 
 1738 As discussed above Roundup's schema is the access control mechanism.
 1739 However you may want to integrate a third party system with roundup.
 1740 E.G. suppose you use a time tracking service that takes an issue id
 1741 and keeps a running count of how much time was spent on it. Then with
 1742 a single button push it can add the recorded time to the roundup
 1743 issue.
 1744 
 1745 You probably don't want to give this third party service your roundup
 1746 username and credentials. Especially if your roundup instance is under
 1747 your company's single sign on infrastructure.
 1748 
 1749 So what we need is a way for this third party service to impersonate
 1750 you and have access to create a roundup timelog entry (see
 1751 `<customizing.html#adding-a-time-log-to-your-issues>`__ ). Then add it
 1752 to the associated issue. This should happen without sharing passwords
 1753 and without allowing the third party service to see the issue (except the
 1754 ``times`` property), user, or other information in the tracker.
 1755 
 1756 Enter the use of a JSON web token. Roundup has rudimentary ability to
 1757 manage JWTs and use them for authentication and authorization.
 1758 
 1759 There are 5 steps to set this up:
 1760 
 1761 1. install pyjwt library using pip or pip3. If roundup can't find the
 1762    jwt module you will see the error ``Support for jwt disabled.``
 1763 2. create a new role that allows Create access to timelog and edit/view
 1764    access to an issues' ``times`` property.
 1765 3. add support for issuing (and validating) jwts to the rest interface.
 1766    This uses the `Adding new rest endpoints`_ mechanism.
 1767 4. configure roundup's config.ini [web] jwt_secret with at least 32
 1768    random characters of data. (You will get a message
 1769    ``Support for jwt disabled by admin.`` if it's not long enough.)
 1770 5. add an auditor to make sure that users with this role are appending
 1771    timelog links to the ``times`` property of the issue.
 1772 
 1773 Create role
 1774 ~~~~~~~~~~~
 1775 
 1776 Adding this snippet of code to the tracker's ``schema.py`` should create a role with the
 1777 proper authorization::
 1778 
 1779    db.security.addRole(name="User:timelog",
 1780          description="allow a user to create and append timelogs")
 1781 
 1782    db.security.addPermissionToRole('User:timelog', 'Rest Access')
 1783 
 1784    perm = db.security.addPermission(name='Create', klass='timelog',
 1785             description="Allow timelog creation", props_only=False)
 1786    db.security.addPermissionToRole("User:timelog", perm)
 1787 
 1788    perm = db.security.addPermission(name='View', klass='issue',
 1789             properties=('id', 'times'),
 1790             description="Allow retrieving issue etag or timelog issue",
 1791             props_only=False)
 1792    db.security.addPermissionToRole("User:timelog", perm)
 1793 
 1794    perm = db.security.addPermission(name='Edit', klass='issue',
 1795             properties=('id', 'times'),
 1796             description="Allow editing timelog for issue",
 1797             props_only=False)
 1798    db.security.addPermissionToRole("User:timelog", perm)
 1799 
 1800 The role is named to work with the /rest/jwt/issue rest endpoint
 1801 defined below. Starting the role name with ``User:`` allows the jwt
 1802 issue code to create a token with this role if the user requesting the
 1803 role has the User role.
 1804 
 1805 The role *must* have access to the issue ``id`` to retrieve the etag for
 1806 the issue.  The etag is passed in the ``If-Match`` HTTP header when you
 1807 make a call to patch or update the ``times`` property of the issue.
 1808 
 1809 If you use a PATCH rest call with "@op=add" to append the new timelog,
 1810 you don't need View access to the ``times`` property. If you replace the
 1811 ``times`` value, you need to read the current value of ``times`` (using
 1812 View permission), append the newly created timelog id to the (array)
 1813 value, and replace the ``times`` value.
 1814 
 1815 Note that the json returned after the operation will include the new
 1816 value of the ``times`` value so your code can verify that it worked.
 1817 This does potentially leak info about the previous id's in the field.
 1818 
 1819 Create rest endpoints
 1820 ~~~~~~~~~~~~~~~~~~~~~
 1821 
 1822 Here is code to add to your tracker's ``interfaces.py`` (note code has
 1823 only been tested with python3)::
 1824 
 1825     from roundup.rest import Routing, RestfulInstance, _data_decorator
 1826 
 1827     class RestfulInstance(object):
 1828         @Routing.route("/jwt/issue", 'POST')
 1829         @_data_decorator
 1830         def generate_jwt(self, input):
 1831             import jwt
 1832             import datetime
 1833             from roundup.anypy.strings import b2s
 1834 
 1835             # require basic auth to generate a token
 1836             # At some point we can support a refresh token.
 1837             # maybe a jwt with the "refresh": True claim generated
 1838             # using: "refresh": True in the json request payload.
 1839 
 1840             denialmsg='Token creation requires login with basic auth.'
 1841             if 'HTTP_AUTHORIZATION' in self.client.env:
 1842                 try:
 1843                     auth = self.client.env['HTTP_AUTHORIZATION']
 1844                     scheme, challenge = auth.split(' ', 1)
 1845                 except (ValueError, AttributeError):
 1846                     # bad format for header
 1847                     raise Unauthorised(denialmsg)
 1848                 if scheme.lower() != 'basic':
 1849                     raise Unauthorised(denialmsg)
 1850             else:
 1851                 raise Unauthorised(denialmsg)
 1852 
 1853             # If we reach this point we have validated that the user has
 1854             # logged in with a password using basic auth.
 1855             all_roles = list(self.db.security.role.items())
 1856             rolenames = []
 1857             for role in all_roles:
 1858                 rolenames.append(role[0])
 1859 
 1860             user_roles = list(self.db.user.get_roles(self.db.getuid()))
 1861 
 1862             claim= { 'sub': self.db.getuid(),
 1863                      'iss': self.db.config.TRACKER_WEB,
 1864                      'aud': self.db.config.TRACKER_WEB,
 1865                      'iat': datetime.datetime.utcnow(),
 1866                    }
 1867 
 1868             lifetime = 0
 1869             if 'lifetime' in input:
 1870                 if input['lifetime'].value != 'unlimited':
 1871                     try:
 1872                         lifetime = datetime.timedelta(seconds=int(input['lifetime'].value))
 1873                     except ValueError:
 1874                         raise UsageError("Value 'lifetime' must be 'unlimited' or an integer to specify" +
 1875                                          " lifetime in seconds. Got %s."%input['lifetime'].value)
 1876             else:
 1877                 lifetime = datetime.timedelta(seconds=86400) # 1 day by default
 1878 
 1879             if lifetime: # if lifetime = 0 make unlimited by omitting exp claim
 1880                 claim['exp'] = datetime.datetime.utcnow() + lifetime
 1881 
 1882             newroles = []
 1883             if 'roles' in input:
 1884                 for role in input['roles'].value:
 1885                     if role not in rolenames:
 1886                         raise UsageError("Role %s is not valid."%role)
 1887                     if role in user_roles:
 1888                         newroles.append(role)
 1889                         continue
 1890                     parentrole = role.split(':', 1)[0]
 1891                     if parentrole in user_roles:
 1892                         newroles.append(role)
 1893                         continue
 1894 
 1895                     raise UsageError("Role %s is not permitted."%role)
 1896 
 1897                 claim['roles'] = newroles
 1898             else:
 1899                 claim['roles'] = user_roles
 1900             secret = self.db.config.WEB_JWT_SECRET
 1901             myjwt = jwt.encode(claim, secret, algorithm='HS256')
 1902 
 1903             result = {"jwt": b2s(myjwt),
 1904                      }
 1905 
 1906             return 200, result
 1907 
 1908         @Routing.route("/jwt/validate", 'GET')
 1909         @_data_decorator
 1910         def validate_jwt(self,input):
 1911             import jwt
 1912             if not 'jwt' in input:
 1913                 raise UsageError("jwt key must be specified")
 1914 
 1915             myjwt = input['jwt'].value
 1916 
 1917             secret = self.db.config.WEB_JWT_SECRET
 1918             try:
 1919                 result = jwt.decode(myjwt, secret,
 1920                                     algorithms=['HS256'],
 1921                                     audience=self.db.config.TRACKER_WEB,
 1922                                     issuer=self.db.config.TRACKER_WEB,
 1923                 )
 1924             except jwt.exceptions.InvalidTokenError as err:
 1925                 return 401, str(err)
 1926 
 1927             return 200, result
 1928 
 1929 **Note this is sample code. Use at your own risk.** It breaks a few
 1930 rules about jwts (e.g. it allows you to make unlimited lifetime
 1931 jwts). If you subscribe to the concept of jwt refresh tokens, this code
 1932 will have to be changed as it will only generate jwts with
 1933 username/password authentication.
 1934 
 1935 Currently use of jwts an experiment. If this appeals to you consider
 1936 providing patches to existing code to:
 1937 
 1938 1. record all jwts created by a user
 1939 2. using the record to allow jwts to be revoked and ignored by the
 1940    roundup core
 1941 3. provide a UI page for managing/revoking jwts
 1942 4. provide a rest api for revoking jwts
 1943 
 1944 These end points can be used like::
 1945 
 1946    curl -u demo -s -X POST -H "Referer: https://.../demo/" \
 1947       -H "X-requested-with: rest" \
 1948       -H "Content-Type: application/json" \
 1949       --data '{"lifetime": "3600", "roles": [ "user:timelog" ] }' \
 1950    https://.../demo/rest/jwt/issue
 1951 
 1952 (note roles is a json array/list of strings not a string) to get::
 1953 
 1954   {
 1955     "data": {
 1956             "jwt":  "eyJ0eXAiOiJK......XxMDb-Q3oCnMpyhxPXMAk"
 1957         }
 1958   }
 1959 
 1960 The jwt is shortened in the example since it's large. You can validate
 1961 a jwt to see if it's still valid using::
 1962 
 1963 
 1964   curl -s -H "Referer: https://.../demo/" \
 1965   -H "X-requested-with: rest" \
 1966       https://.../demo/rest/jwt/validate?jwt=eyJ0eXAiOiJK...XxMDb-Q3oCnMpyhxPXMAk
 1967 
 1968 (note no login is required) which returns::
 1969   
 1970   {
 1971     "data": {
 1972        "user": "3",
 1973        "roles": [
 1974             "user:timelog"
 1975         ],
 1976        "iss": "https://.../demo/",
 1977        "aud": "https://.../demo/",
 1978        "iat": 1569542404,
 1979        "exp": 1569546004
 1980      }
 1981   }				
 1982 
 1983 Final steps
 1984 ^^^^^^^^^^^
 1985 
 1986 See the `upgrading directions`_ on how to use the ``updateconfig``
 1987 command to generate an updated copy of config.ini using
 1988 roundup-admin. Then set the ``jwt_secret`` to at least 32 characters
 1989 (more is better up to 512 bits).
 1990 
 1991 Writing an auditor that uses "db.user.get_roles" to see if the user
 1992 making the change has the ``user:timelog`` role, and then comparing
 1993 the original ``times`` list to the new list to verify that it is being
 1994 added to and not changed otherwise is left as an exercise for the
 1995 reader. (If you develop one, please contribute via the tracker:
 1996 https://issues.roundup-tracker.org/.)
 1997 
 1998 Lastly you can create a JWT using the end point above and make a rest
 1999 call to create a new timelog entry and another call to update the
 2000 issues times property.  If you have other ideas on how jwts can be
 2001 used, please share on the roundup mailing lists. See:
 2002 https://sourceforge.net/p/roundup/mailman/ for directions on
 2003 subscribing and for archives of the lists.
 2004 
 2005 
 2006 Creating Custom Rate Limits
 2007 ===========================
 2008 
 2009 You can replace the default rate limiter that is configured using
 2010 the tracker's ``config.ini``. You can return different rate
 2011 limits based on the user, time of day, phase of moon etc.
 2012 
 2013 Assume you add two integer valued properties to the user
 2014 object. Let's call them ``rate_limit_interval`` and
 2015 ``rate_limit_calls``. Add code similar to this to interfaces.py
 2016 to override the default rate limiter code::
 2017 
 2018     from roundup.rest import RestfulInstance, RateLimit
 2019     from datetime import timedelta
 2020 
 2021     def grl(self):
 2022         calls = self.db.config.WEB_API_CALLS_PER_INTERVAL
 2023         interval = self.db.config.WEB_API_INTERVAL_IN_SEC
 2024 
 2025         if calls and interval: # use to disable all rate limits
 2026 
 2027             uid = self.db.getuid()
 2028             class_obj = self.db.getclass('user')
 2029             node = class_obj.getnode(uid)
 2030 
 2031 	    # set value to 0 to use WEB_API_CALLS_PER_INTERVAL
 2032 	    user_calls = node.__getattr__('rate_limit_calls')
 2033 	    # set to 0 to use WEB_API_INTERVAL_IN_SEC
 2034 	    user_interval = node.__getattr__('rate_limit_interval')
 2035 	    
 2036             return RateLimit(user_calls or calls,
 2037 	       	   timedelta(seconds=(user_interval or interval)))
 2038         else:
 2039             # disable rate limiting if either parameter is 0
 2040             return None
 2041 
 2042     RestfulInstance.getRateLimit = grl
 2043 
 2044 this should replace the default getRateLimit with the new grl
 2045 function.  This new function uses values for the number of calls
 2046 and period that are specific to a user.  If either is set to 0,
 2047 the defaults from ``config.ini`` file are used.
 2048 
 2049 Test Examples
 2050 ^^^^^^^^^^^^^
 2051 
 2052 Rate limit tests::
 2053 
 2054    seq 1 300 | xargs -P 20 -n 1 curl --head -si \
 2055         https://.../rest/data/status/new \# | grep Remaining