"Fossies" - the Fresh Open Source Software Archive

Member "SAOImageDS9/tcllib/devdoc/indexing.txt" (13 Nov 2019, 15778 Bytes) of package /linux/misc/ds9.8.1.tar.gz:


As a special service "Fossies" has tried to format the requested text file into HTML format (style: standard) with prefixed line numbers. Alternatively you can here view or download the uninterpreted source code file.

    1 Tcllib package indexing
    2 =======================
    3 
    4 This document describes the possibilities for using one or more
    5 pkgIndex.tcl files in an installation of tcllib to provide the
    6 information about all of its packages to a tcl interpreter, discusses
    7 their pro and contra and makes a choice for Tcllib 1.4. A roadmap of
    8 changes in the future is made available as an appendix.
    9 
   10 Background under which to see the solutions:
   11 
   12         There are three level of groupings:
   13 
   14         -       The tcllib project itself
   15         -       Modules in the project (== subdirectory of 'modules')
   16         -       Packages in a module.
   17 
   18         Each module currently contains one package index file.
   19 
   20         Some modules contain more than one package. They share the index.
   21 
   22         Most packages require specific versions of the Tcl
   23         interpreter. They perform the checks in their package index
   24         file and do not register if the pre-requisites are not
   25         fulfilled.
   26 
   27         Other checks are possible, but currently not in use.
   28 
   29 Note I:
   30 	Whether a solution is actually applicable depends on external
   31 	factors, like the chosen directory layout of an installed
   32 	tcllib.
   33 
   34 Note II:
   35 	All solutions currently depend on the specific implementation
   36 	of [tclPkgUnknown] coming with the basic core, simply by the
   37 	fact that the files looked at are called 'pkgIndex.tcl'. This
   38 	is therefore no contra argument against any specific solution,
   39 	but against all. We ignore this as currently there is no
   40 	better replacement in existence.
   41 
   42 Note III:
   43 	We have to support Tcl before 8.3. as some packages in tcllib
   44 	allow this.
   45 
   46 
   47 [i1/ng] No global package index
   48 -------------------------------
   49 
   50         In this solution the module package indices are the only index
   51         files present in an installation.
   52 
   53         This solution is applicable if and only if one of the flat
   54         directory layouts (L2/Fa or L2/Fb) has been chosen.
   55 
   56         Pro:
   57                 Simple. No need for complex management.
   58 
   59 
   60 [i2/ad] Global package index, auto_path extension, direct
   61 ---------------------------------------------------------
   62 
   63         A single global package index is present in the toplevel
   64         directory of the installation.
   65 
   66         This solution is applicable if and only if the deep directory
   67         layout (L2/D) has been chosen.
   68 
   69         The package index contains a series of statements extending
   70         the auto_path variable with all module directories. The list
   71         of names of the module directories is hardcoded. In other
   72         words, it is _not_ determined via [glob].
   73 
   74         Example:
   75                 lappend auto_path [file join $dir md4]
   76                 lappend auto_path [file join $dir md5]
   77                 lappend auto_path [file join $dir sha1]
   78                 ...
   79 
   80 
   81         Pro:
   82                 [[0]]   Compared to [i3/ag] this should be bit faster
   83                         as glob'ing the directory tree of tcllib is
   84                         avoided. This performance-boost is not a big
   85                         pro according to the opinions below.
   86 
   87                 [[1]]   Relies on the module package index files for
   88                         the actual registration of packages, thus
   89                         automatically inherits the correct constraints
   90                         on the registration of packages. No additional
   91                         complexities.
   92 
   93                 [[2]]   Easier to generate than [i6/dr].
   94 
   95         Contra:
   96                 [[3]]   Hard coding the directory names implies that
   97                         adding modules to the installed tcllib is not
   98                         as easy as just creating a new directory for
   99                         the module/package. The global index has to be
  100                         updated too.
  101 
  102                         Contra-Contra:
  103                                 <<Don: New, updated packages should be
  104                                 installed separately, outside of
  105                                 tcllib. The ticked version number
  106                                 ensures that it is prefered over the
  107                                 package in tcllib.>>
  108 
  109                                 <<AK: Agree fully>>
  110         
  111                 [[4]]   Extending the 'auto_path' list causes the
  112                         package management of the tcl core to re-read
  113                         the list and glob through all of them for new
  114                         package indices. This has a high cost in terms
  115                         of filesystem access, i.e. is an issue of
  116                         performance.
  117 
  118                         Contra-Contra:
  119                                 <<Don: IMHO, it's not really tcllib's
  120                                 job to fix [tclPkgUnknown]'s
  121                                 performance problems. If performance
  122                                 is a problem, users could try the
  123                                 patch with Tcl Feature Request 680169
  124                                 and see if it helps.>>
  125 
  126                                 <<AK: Not sure yet about this>>
  127 
  128 
  129                 [[5]]   This enables auto-loading in each module
  130                         (according to any tclIndex file installed).
  131                         This should not be done by the package
  132                         indexer, but by the package itself.  See
  133                         control for an example.
  134 
  135 	  	[[10]]	Will not work with Tcl releases prior to
  136 			8.3.1.  Only then was [tclPkgUnknown]
  137 			"enhanced" to deal with changing ::auto_path
  138 			values.  If tcllib 1.4 wishes to continue
  139 			supporting pre-8.3.1 Tcl, then this option has
  140 			to be supplemented with a fallback.
  141 
  142 
  143 [i3/ag] Global package index, auto_path extension, glob
  144 -------------------------------------------------------
  145 
  146         This is like [i2/ad], except that the list of sub directories
  147         is not hardcoded into the index, but determined through glob.
  148 
  149         Example:
  150                 foreach subdir [glob -nocomplain -type d $dir/*] {
  151                     lappend auto_path $subdir
  152                 }
  153 
  154         Pro:
  155                 Anti-[[3]]
  156                 [[1]]
  157 
  158         Contra:
  159                 All the contras of [i2/ad] and Anti-[[0]].
  160 
  161 
  162 [i4/sd] Global package index, sourcing module indices, direct
  163 -------------------------------------------------------------
  164 
  165         A single global package index is present in the toplevel
  166         directory of the installation.
  167 
  168         This solution is applicable if and only if the deep directory
  169         layout (L2/D) has been chosen.
  170 
  171         The package index contains a series of statements source'ing
  172         the package index files of the modules in tcllib. The list
  173         of names of the module directories is hardcoded. In other
  174         words, it is _not_ determined via [glob].
  175 
  176         Example:
  177                 set main $dir
  178                 set dir [file join $main md4]  ; source [file join $dir pkgIndex.tcl]
  179                 set dir [file join $main md5]  ; source [file join $dir pkgIndex.tcl]
  180                 set dir [file join $main sha1] ; source [file join $dir pkgIndex.tcl]
  181                 ...
  182 
  183         Pro:
  184                 [[0]], but compared to [i5/sg].
  185                 [[1]]
  186                 [[2]]
  187                 [[6]]   In contrast to [i2/ad] and [i3/ag] repeated
  188                         glob'ing for package index files is
  189                         avoided. This cuts down on costly FS accesses.
  190                         I.e. another perf. boost.
  191 
  192         Contra:
  193                 [[3]]
  194 
  195 [i5/sg] Global package index, sourcing module indices, glob
  196 -----------------------------------------------------------
  197 
  198         This is like [i4/sd], except that the list of package indices
  199         to source is not hardcoded into the index, but determined
  200         through glob.
  201 
  202         Example:
  203                 foreach subdir [glob -nocomplain -type d $dir/*] {
  204                         set dir $subdir
  205                         source [file join $dir pkgIndex.tcl]
  206                 }
  207 
  208         Pro:
  209                 Anti-[[3]]
  210                 [[1]]
  211                 [[2]]
  212 
  213         Contra:
  214                 All the contras of [i2/sd], and Anti-[[0]]
  215 
  216 
  217 [i6/dr] Global package index, direct registration
  218 -------------------------------------------------
  219 
  220         A single global package index is present in the toplevel
  221         directory of the installation.
  222 
  223         This solution is applicable if and only if the deep directory
  224         layout (L2/D) has been chosen.
  225 
  226         The package index contains a series of statements which
  227         directly register all the tcllib packages.
  228 
  229         Example:
  230                 if {[constraint]} {return}
  231                 package ifneeded md4  [list source [file join $dir md4 md4.tcl]]
  232                 package ifneeded md5  [list source [file join $dir md4 md4.tcl]]
  233                 package ifneeded sha1 [list source [file join $dir md4 md4.tcl]]
  234                 ... more constraints ... package ifneeded
  235 
  236         Pro:
  237                 [[7]]   This is the fasted solution as the number of
  238                         accesses to the filesystem is minimal.
  239 
  240         Contra:
  241                 [[[3]]
  242                 Anti-[[1]]	Care has to be taken to ensure that
  243                                 the constraints the module indices
  244                                 place on the registration of packages
  245                                 are replicated in the global
  246                                 index. All other solutions simply used
  247                                 the module indices and thus got it
  248                                 right automatically. Now supporting
  249                                 code is required to detect such
  250                                 constraints and then to properly
  251                                 recreate them globally.
  252 
  253                                 = High complexity for the maintainer.
  254 
  255 [i7/ad] Global package index, auto_path extension, direct
  256 ---------------------------------------------------------
  257 
  258         A single global package index is present in the toplevel
  259         directory of the installation.
  260 
  261         This solution is applicable if and only if the deep directory
  262         layout (L2/D) has been chosen.
  263 
  264         The package index contains a single statement extending the
  265         auto_path variable with the tcllib main directory. The
  266         standard package management will then find all module sub
  267         directories and the package indices in them.
  268 
  269         Example:
  270                 lappend auto_path $dir
  271 
  272         Pro:
  273                 [[1]]
  274                 [[8]]   This is the easiest solution by far in terms
  275                         of code to write, and complexities to solve
  276                         (none).
  277 
  278 		[[9]]	<<Don: I believe this is the only proposal listed
  279 			that actually fixes tcllib Bug 720318
  280 			(successful [package require] of packages
  281 			within a SafeBase) because it is the only one
  282 			that changes the value of ::auto_path.>>
  283 
  284 			<<AK: This is true, yet brittle. It depends on
  285 			when the SafeBase sees the auto_path. If it
  286 			happens to be before a [package require
  287 			something] forced the reading of all package
  288 			indices (and thus the extension of
  289 			'auto_path') we are still SOL.>>
  290 
  291 	Contra: [[4]]
  292 	  	[[10]]
  293 
  294 
  295 [i8/pm] Global package index, pkg_mkIndex
  296 -----------------------------------------
  297 
  298 Just use [pkg_mkIndex modules */*.tcl] to generate the master index.
  299 
  300 	Pro:
  301 		Easy to do.
  302 
  303 	Contra:
  304 		Does not handle constraints in subordinate package
  305 		indices, simply because they are actually ignored
  306 		during processing.
  307 
  308 		Adding code to handle constraints evolves this into
  309 		[i6/dr].
  310 
  311 	Note: The contra is hard enough IMHO to make this solution not
  312 	applicable for 1.4, which does have constraints, and handling
  313 	them wrong (not at all) is a bug.
  314 
  315 
  316 General discussion
  317 ------------------
  318 
  319 Given that a deep directory layout was chosen [i1/ng] is not
  320 applicable and therefore dropped from the discussion.
  321 
  322 In the pro and contra arguments listed above three independent axes of
  323 reasoning emerged:
  324 
  325 a)        Performance of the solution, with the number of accesses to
  326           filesystem the main factor determining it.
  327 
  328 b)        Complexity/difficulty of the solution with regard to
  329           adding/updating packages.
  330 
  331 c)        Complexity of generating the master index.
  332 
  333 Axis (b) has essentially been thrown out. Trying to modify the
  334 installation of tcllib itself is bad practice. Install new/updated
  335 packages separately. The version numbering takes care of the rest,
  336 i.e. usage of the new over the older version found in tcllib.
  337 
  338 With respect to axis (c), complexity of generation, [i7/ad] is the
  339 definite winner, with the other *d solutions close behind (all use
  340 fixed scripts, I7/ad wins on size). This is followed by the *g
  341 solutions as they require actual dynamic generation of code. And at
  342 the bottom of the ladder is [i6/dr] with its need for close inspection
  343 of the sub-ordinate indices to get everything right.
  344 
  345 Now axis (a), performance, [i6/dr] is most likely the winner as it
  346 causes only one index to be read and nothing else. This is followed by
  347 the all *d solutions, which read the subordinate indices, but do not
  348 need much glob'ing. The actual order in this group is difficult to
  349 determine. I guess that the auto_path extending methods are slower
  350 than the sourcing methods, and the adding of one directory faster than
  351 the adding of all, as the latter looks for much more subdirectories.
  352 The next group are the *g solutions as they perform their own glob'ing too
  353 beyond that done by the package mgmt.
  354 
  355 Two final rankings
  356 
  357         (c), then (a)           (a), then (c)
  358         -------------           -------------
  359         [i7/ad]                 [i6/dr]
  360         [i4/sd]                 [i4/sd]
  361         [i2/ad]                 [i7/ad]
  362         [i5/sg]                 [i2/ad]
  363         [i3/ag]                 [i5/sg]
  364         [i6/dr]                 [i3/ag]
  365         -------------           -------------
  366 
  367 [i4/sd] seems to be a good compromise solution between performance and
  368 complexity of generation, but [i7/ad] is not too bad either.
  369 
  370 [i4/sd] reminder:
  371         set main $dir
  372         set dir [file join $main md4]  ; source [file join $dir pkgIndex.tcl]
  373         set dir [file join $main md5]  ; source [file join $dir pkgIndex.tcl]
  374         set dir [file join $main sha1] ; source [file join $dir pkgIndex.tcl]
  375         ...
  376 
  377 [i7/ad] reminder:
  378         lappend auto_path $dir
  379 
  380 Other opinions:
  381 
  382       	Don Porter prefers [i7/ad], and [i6/dr] as second choice.  Also
  383 	as [i7/ad] fallback for older Tcl before 8.3.1
  384 
  385 	Joe English strictly opposes any solution modifying the
  386 	auto_path, violating his opinion that index scripts should
  387 	have no side-effects beyond registering a package.
  388 
  389 
  390 Chosen solution for Tcllib 1.4
  391 ------------------------------
  392 
  393 After comparing the code for the combination of [i7/ad] and [i6/dr] as
  394 submitted by Don Porter, and for [i4/sd] as submitted by myself
  395 (Andreas), and a small discussion on the Tcl'ers chat between Don and
  396 me, we took [i4/sd] for the main body of the index, and the header of
  397 Don's code. Basically the chosen package index is a combination of
  398 [i7/id] and of [i4/sd] as fallback.
  399 
  400 This is still as easy to generate as [4/sd], the index is also only a
  401 bit more complex, and speed should be okay too.
  402 
  403 Don convinced me that while extending auto_path is definitely bad in
  404 the long-term it is still okay for the short-term and release 1.4.
  405 
  406 
  407 Roadmap
  408 -------
  409 
  410 After Tcllib has been driven into the state of one package per module
  411 directory, and switched to a flat directory layout for its
  412 installation we switch to [i1/ng] for the indexing structure.
  413 
  414 
  415 -----------------------------------
  416 This document is in the public domain.
  417 
  418                         Andreas Kupries <andreas_kupries@users.sf.net>