"Fossies" - the Fresh Open Source Software Archive  

Source code changes of the file "doc/source/design/source-shell-script-in-modulefile.rst" between
modules-5.1.1.tar.bz2 and modules-5.2.0.tar.bz2

About: The Environment Modules package provides for the dynamic modification of a user’s environment via modulefiles.

source-shell-script-in-modulefile.rst  (modules-5.1.1.tar.bz2)	:	source-shell-script-in-modulefile.rst  (modules-5.2.0.tar.bz2)
skipping to change at line 16		skipping to change at line 16
Configuration		Configuration
-------------		-------------
- No specific configuration		- No specific configuration
Specification		Specification
-------------		-------------
- Execute a shell script and include the environment changes it performs in modu lefile		- Execute a shell script and include the environment changes it performs in modu lefile
- Environment changes done by shell script are gathered and evaluated in mod		- Environment changes done by shell script are gathered and evaluated in modul
ulefile context through corresponding modulefile commands		efile context through corresponding modulefile commands
- Goal is to get the same environment after loading a modulefile using a :mf		- Goal is to get the same environment after loading a modulefile using a :mfcm
cmd:`source-sh` than if shell script targeted by the ``source-sh`` where directl		d:`source-sh` than if shell script targeted by the ``source-sh`` where directly
y sourced in shell session		sourced in shell session
- Environment changes tracked are (all environment elements ``module`` can handl e):		- Environment changes tracked are (all environment elements ``module`` can handl e):
- Value set to variable: transformed into ``setenv`` modulefile command		- Value set to variable: transformed into ``setenv`` modulefile command
- Variable unset: transformed into ``unsetenv`` modulefile command		- Variable unset: transformed into ``unsetenv`` modulefile command
- Path element added to variable: transformed into ``append-path`` or ``prep		- Path element added to variable: transformed into ``append-path`` or ``prepen
end-path`` modulefile command		d-path`` modulefile command
- Path element removed from variable: transformed into ``remove-path`` modul		- Path element removed from variable: transformed into ``remove-path`` modulef
efile command		ile command
- Current working directory change: transformed into ``cd`` modulefile comma		- Current working directory change: transformed into ``cd`` modulefile command
nd		- Shell alias definition: transformed into ``set-alias``
- Shell alias definition: transformed into ``set-alias``		- Shell alias unset: transformed into ``unset-alias``
- Shell alias unset: transformed into ``unset-alias``		- Shell function definition: transformed into ``set-function``
- Shell function definition: transformed into ``set-function``		- Shell function unset: transformed into ``unset-function``
- Shell function unset: transformed into ``unset-function``		- Shell completion definition: transformed into ``complete``
- Shell completion definition: transformed into ``complete``		- Shell completion unset: transformed into ``uncomplete``
- Shell completion unset: transformed into ``uncomplete``
- Depending on modulefile evaluation mode, ``source-sh`` has different behaviors :		- Depending on modulefile evaluation mode, ``source-sh`` has different behaviors :
- ``load``: execute script to gather its environment changes, transform them		- ``load``: execute script to gather its environment changes, transform them i
in modulefile commands and evaluate them		n modulefile commands and evaluate them
- ``unload``: undo environment changes made on load mode		- ``unload``: undo environment changes made on load mode
- ``display``: execute script to gather its environment changes and report r		- ``display``: execute script to gather its environment changes and report res
esulting command name and arguments for display		ulting command name and arguments for display
- ``help``, ``test`` and ``whatis``: no operation		- ``help``, ``test`` and ``whatis``: no operation
- **Limitation**: code in modulefile cannot rely on the environment changes done in script targeted by a ``source-sh`` command		- **Limitation**: code in modulefile cannot rely on the environment changes done in script targeted by a ``source-sh`` command
- For instance an environment variable set in shell script cannot be used to		- For instance an environment variable set in shell script cannot be used to d
define another variable later in modulefile		efine another variable later in modulefile
- This will work on ``load``, ``unload`` and ``display`` modes, as script is		- This will work on ``load``, ``unload`` and ``display`` modes, as script is r
run and/or analyzed and corresponding modulefile commands are evaluated in modu		un and/or analyzed and corresponding modulefile commands are evaluated in module
lefile context		file context
- But it will not work on the other modes, as script is not run and analyzed		- But it will not work on the other modes, as script is not run and analyzed t
there		here
- To simplify processing as script need to be run and analyzed if not ye		- To simplify processing as script need to be run and analyzed if not yet lo
t loaded, but if already loaded changes recorded in environment for tracking sho		aded, but if already loaded changes recorded in environment for tracking should
uld be used instead		be used instead
- To avoid a negative impact on performances on the ``whatis`` global ev		- To avoid a negative impact on performances on the ``whatis`` global evalua
aluation		tion
- Result of ``source-sh`` command evaluation is stored into the environment when modulefile is loaded		- Result of ``source-sh`` command evaluation is stored into the environment when modulefile is loaded
- To keep track of environment changes made by ``source-sh`` script evaluati		- To keep track of environment changes made by ``source-sh`` script evaluation
on		- In order to undo these changes when unloading modulefile and report correspo
- In order to undo these changes when unloading modulefile and report corres		nding modulefile commands when displaying modulefile
ponding modulefile commands when displaying modulefile		- Environment variable :envvar:`__MODULES_LMSOURCESH` is used for this need
- Environment variable :envvar:`__MODULES_LMSOURCESH` is used for this need
- Using following syntax: ``mod&shell script arg|cmd|cmd|...&shell scrip		- Using following syntax: ``mod&shell script arg|cmd|cmd|...&shell script|cm
t|cmd:mod&shell script arg|cmd``		d:mod&shell script arg|cmd``
- Example value: ``foo/1&sh /tmp/source.sh|append-path PATH /tmp|cd /tmp		- Example value: ``foo/1&sh /tmp/source.sh|append-path PATH /tmp|cd /tmp``
``		- Characters used to structure information in ``__MODULES_LMSOURCESH`` (``:`
- Characters used to structure information in ``__MODULES_LMSOURCESH`` (		`, ``&`` and ``|``) are escaped
``:``, ``&`` and ``|``) are escaped
- Respectively to ``<EnvModEscPS>``, ``<EnvModEscS1>`` and ``<EnvMod		- Respectively to ``<EnvModEscPS>``, ``<EnvModEscS1>`` and ``<EnvModEscS2>
EscS2>``		``
- If found in environment changes to record		- If found in environment changes to record
- Actual bodies of shell alias, shell functions and shell completions ar e not recorded in ``__MODULES_LMSOURCESH``, an empty body is recorded instead		- Actual bodies of shell alias, shell functions and shell completions are no t recorded in ``__MODULES_LMSOURCESH``, an empty body is recorded instead
- Example value: ``foo/1&sh /tmp/source.sh|set-alias alfoo {}|set-fu nction funcfoo {}|complete bash foocmd {}``		- Example value: ``foo/1&sh /tmp/source.sh|set-alias alfoo {}|set-function funcfoo {}|complete bash foocmd {}``
- When unloading modulefile, content found for this modulefile in ``__MODULE S_LMSOURCESH`` variable is evaluated to reverse environment changes		- When unloading modulefile, content found for this modulefile in ``__MODULES_ LMSOURCESH`` variable is evaluated to reverse environment changes
- When reaching a ``source-sh`` modulefile command, recorded content is evaluated through a modulefile Tcl interpreter in unload mode, to get the revers ed effect		- When reaching a ``source-sh`` modulefile command, recorded content is eval uated through a modulefile Tcl interpreter in unload mode, to get the reversed e ffect
- When displaying modulefile		- When displaying modulefile
- If it is loaded		- If it is loaded
- the content found for this modulefile in ``__MODULES_LMSOURCESH``		- the content found for this modulefile in ``__MODULES_LMSOURCESH`` variab
variable is evaluated in display mode to report each resulting modulefile comman		le is evaluated in display mode to report each resulting modulefile command
d		- script is evaluated to fetch shell alias, function and completion defini
- script is evaluated to fetch shell alias, function and completion		tions which are not recorded in ``__MODULES_LMSOURCESH``
definitions which are not recorded in ``__MODULES_LMSOURCESH``
- If not loaded, script is evaluated to gather environment changes and r eport each resulting modulefile command		- If not loaded, script is evaluated to gather environment changes and repor t each resulting modulefile command
- Script targeted by a ``source-sh`` command has to be run and environment prior this run and after this run have to be compared to determine the environment ch anges the script performs		- Script targeted by a ``source-sh`` command has to be run and environment prior this run and after this run have to be compared to determine the environment ch anges the script performs
- The shell to use to run script has to be specified to ``source-sh``		- The shell to use to run script has to be specified to ``source-sh``
- This shell will be run to execute the following sequence:		- This shell will be run to execute the following sequence:
1. output current environment state		1. output current environment state
2. source the script with its args		2. source the script with its args
3. output environment state after script source		3. output environment state after script source
- Script output is kept to return it in an error message in case its executi		- Script output is kept to return it in an error message in case its execution
on fails		fails
- This comparison determines the environment changes performed by script whi		- This comparison determines the environment changes performed by script which
ch are then translated into corresponding modulefile commands		are then translated into corresponding modulefile commands
- Shell is run in a mode where neither its user nor its system-wide configur		- Shell is run in a mode where neither its user nor its system-wide configurat
ation is evaluated to avoid side effect		ion is evaluated to avoid side effect
- Shell needs current environment to correctly evaluate script		- Shell needs current environment to correctly evaluate script
- it seems desirable to run shell in an empty environment to get the ful		- it seems desirable to run shell in an empty environment to get the full li
l list of changes it does over the environment		st of changes it does over the environment
- but the script may need the environment to be defined to correctly ope		- but the script may need the environment to be defined to correctly operate
rate
- for instance it needs the ``PATH`` to be set to execute external c ommands		- for instance it needs the ``PATH`` to be set to execute external command s
- so shell run inherits current environment to ensure script will be pro		- so shell run inherits current environment to ensure script will be properl
perly evaluated		y evaluated
- but as a consequence if an environment change performed by script is a		- but as a consequence if an environment change performed by script is alrea
lready done prior script run, this environment change will not be *seen*		dy done prior script run, this environment change will not be *seen*
- **Limitation**: a variable already set by another module will not be seen set sourced script		- **Limitation**: a variable already set by another module will not be seen set sourced script
- if this module is unloaded and if it does not trigger the unload of the mo dulefile using ``source-sh``, variable will be unset as source-sh script has not increased variable reference counter		- if this module is unloaded and if it does not trigger the unload of the modu lefile using ``source-sh``, variable will be unset as source-sh script has not i ncreased variable reference counter
- **Note**: environment change done through ``source-sh`` will not preserve chan ge order occurring in sourced script, as all changes are analyzed after sourcing script		- **Note**: environment change done through ``source-sh`` will not preserve chan ge order occurring in sourced script, as all changes are analyzed after sourcing script
- Most shell will not get the full environment content when spawned as a sub-pro cess		- Most shell will not get the full environment content when spawned as a sub-pro cess
- For instance shell aliases are not transmitted into sub-shells		- For instance shell aliases are not transmitted into sub-shells
- As a result the environment prior script source will most of the time have		- As a result the environment prior script source will most of the time have n
no shell alias defined		o shell alias defined
- **Note**: shells may have specific behaviors		- **Note**: shells may have specific behaviors
- ``dash`` cannot pass arguments to the sourced script		- ``dash`` cannot pass arguments to the sourced script
- ``ksh93`` reads the full script prior sourcing it which may cause issue if		- ``ksh93`` reads the full script prior sourcing it which may cause issue if a
a shell alias is defined in script then used in script		shell alias is defined in script then used in script
- **Limitation**: implementation does not currently support:		- **Limitation**: implementation does not currently support:
- the ``zsh`` shell when used as ``sh`` or as ``ksh``		- the ``zsh`` shell when used as ``sh`` or as ``ksh``
- the ``mksh`` shell		- the ``mksh`` shell
- the BSD variant of the ``sh`` shell		- the BSD variant of the ``sh`` shell
- the Windows ``cmd`` shell		- the Windows ``cmd`` shell
- **Note**: if sourced script produces shell alias or function, these alias or f unction may not be compatible with the current shell of the user		- **Note**: if sourced script produces shell alias or function, these alias or f unction may not be compatible with the current shell of the user
- Shell completion is defined for the shell used to evaluate script		- Shell completion is defined for the shell used to evaluate script
- :mfcmd:`complete` commands are generated even if current user shell is not		- :mfcmd:`complete` commands are generated even if current user shell is not t
the same as the shell used to evaluate script		he same as the shell used to evaluate script
- but the completion definition will produce a no-operation on current user		- but the completion definition will produce a no-operation on current user sh
shell when :mfcmd:`complete` command is evaluated		ell when :mfcmd:`complete` command is evaluated
- **Note**: the mechanism described here only applies for shell script as to und erstand the impact the script execution over the user environment, this environm ent need to be compared prior and after execution		- **Note**: the mechanism described here only applies for shell script as to und erstand the impact the script execution over the user environment, this environm ent need to be compared prior and after execution
- ``source-sh`` modulefile command relies of the ``sh-to-mod`` procedure of ``mo dulecmd.tcl``		- ``source-sh`` modulefile command relies of the ``sh-to-mod`` procedure of ``mo dulecmd.tcl``
- ``sh-to-mod`` procedure handles the evaluation of the targeted script and		- ``sh-to-mod`` procedure handles the evaluation of the targeted script and co
comparison of the environment prior and after this comparison		mparison of the environment prior and after this comparison
- ``sh-to-mod`` returns as a result the list of modulefile commands describi		- ``sh-to-mod`` returns as a result the list of modulefile commands describing
ng the environment changes made by the analyzed script		the environment changes made by the analyzed script
- :subcmd:`sh-to-mod` is a module sub-command, calling the ``sh-to-mod`` procedu re and outputting resulting modulefile commands		- :subcmd:`sh-to-mod` is a module sub-command, calling the ``sh-to-mod`` procedu re and outputting resulting modulefile commands
- with a ``#%Module`` header		- with a ``#%Module`` header
- to enable users to convert the environment changes made by script in modul		- to enable users to convert the environment changes made by script in modulef
efiles		iles
- There is no need to also make ``sh-to-mod`` a modulefile command and ``source- sh`` a module sub-command		- There is no need to also make ``sh-to-mod`` a modulefile command and ``source- sh`` a module sub-command
- Shell to use to source targeted script may be specified as a command name or a s a path name		- Shell to use to source targeted script may be specified as a command name or a s a path name
- When specified as a command name (e.g., ``bash``), command location will b		- When specified as a command name (e.g., ``bash``), command location will be
e determined based on currently set ``PATH``		determined based on currently set ``PATH``
- When specified as a path name (e.g., ``/bin/bash``), this path name determ		- When specified as a path name (e.g., ``/bin/bash``), this path name determin
ines command location		es command location
- Prior executing shell and making it sourcing script:		- Prior executing shell and making it sourcing script:
- Shell is checked to ensure an executable command matches it		- Shell is checked to ensure an executable command matches it
- Script is checked to ensure it exists		- Script is checked to ensure it exists
- Those tests are done prior executing to avoid it if one check fails and pr		- Those tests are done prior executing to avoid it if one check fails and prov
ovide a consistent error message whatever the shell used		ide a consistent error message whatever the shell used
- Environment changes to output as result should be enclosed and escaped		- Environment changes to output as result should be enclosed and escaped
- Enclosed if they contains space character (like white-space, tab and newli		- Enclosed if they contains space character (like white-space, tab and newline
ne characters)		characters)
- Escaped if they contains curly braces, as output is formatted as Tcl code,		- Escaped if they contains curly braces, as output is formatted as Tcl code, t
to avoid parsing issue		o avoid parsing issue
- If an error occurs during targeted script evaluation		- If an error occurs during targeted script evaluation
- Error is thrown which leads to either a modulefile evaluation error or a m		- Error is thrown which leads to either a modulefile evaluation error or a mod
odule sub-command error		ule sub-command error
- Error and output messages reported by evaluated script will be reported al		- Error and output messages reported by evaluated script will be reported alon
ong error by modulefile command or module sub-command		g error by modulefile command or module sub-command
- To get prior and resulting environment state		- To get prior and resulting environment state
- ``env`` command is not used to gather environment state through exec sourc e execution		- ``env`` command is not used to gather environment state through exec source execution
- it would simplify environment state parsing, as same command would be		- it would simplify environment state parsing, as same command would be used
used for every shell		for every shell
- but it is an external command, so requires extra processing and an add		- but it is an external command, so requires extra processing and an additio
itional requirement		nal requirement
- moreover it does not return shell functions in general, only exported		- moreover it does not return shell functions in general, only exported Bash
Bash functions		functions
- Shell builtin commands are used to query existing environment variables, a liases, functions, completions and current working directory		- Shell builtin commands are used to query existing environment variables, ali ases, functions, completions and current working directory
- which provides best processing efficiency		- which provides best processing efficiency
- but leads to specific output parsing for each shell		- but leads to specific output parsing for each shell
- A separator string ``%ModulesShToMod%`` is printed between each kind of en vironment item (variable, alias, etc) and also prior and after script evaluation		- A separator string ``%ModulesShToMod%`` is printed between each kind of envi ronment item (variable, alias, etc) and also prior and after script evaluation
- to separate each output kind and then be able to split them for separa te analysis		- to separate each output kind and then be able to split them for separate a nalysis
- De-duplication of path entries is applied for changes on path-like environment variables		- De-duplication of path entries is applied for changes on path-like environment variables
- If the same path entry appears several times in the newly prepended entrie		- If the same path entry appears several times in the newly prepended entries
s for a variable, the first occurrence of this entry is kept others are dropped		for a variable, the first occurrence of this entry is kept others are dropped
- If the same path entry appears several times in the newly appended entries		- If the same path entry appears several times in the newly appended entries f
for a variable, the first occurrence of this entry is kept others are dropped		or a variable, the first occurrence of this entry is kept others are dropped
- De-duplication is not applied for path entries:		- De-duplication is not applied for path entries:
- appearing in both the new prepended entries and newly appended entries		- appearing in both the new prepended entries and newly appended entries
- appearing in newly prepended entries or newly appended entries and in		- appearing in newly prepended entries or newly appended entries and in entr
entries defined prior script evaluation		ies defined prior script evaluation
- An environment variable equaling to the path separator character (``:``) prior script evaluation is considered as undefined prior script evaluation to avoid m isleading analysis		- An environment variable equaling to the path separator character (``:``) prior script evaluation is considered as undefined prior script evaluation to avoid m isleading analysis
- Environment variables made for Modules private use are filtered-out from the e nvironment changes produced		- Environment variables made for Modules private use are filtered-out from the e nvironment changes produced
- ``LOADEDMODULES``, ``_LMFILES_`` and any variable prefixed by ``__MODULES_		- ``LOADEDMODULES``, ``_LMFILES_`` and any variable prefixed by ``__MODULES_``
`` are concerned		are concerned
- Changes relative to Modules state are ignored this way		- Changes relative to Modules state are ignored this way
- If script loads in turn a modulefile, environment changes reported will no		- If script loads in turn a modulefile, environment changes reported will not
t report the loaded module but only the environment changes it does		report the loaded module but only the environment changes it does
- Modules configuration variable (prefixed by ``MODULES_``) are still taken		- Modules configuration variable (prefixed by ``MODULES_``) are still taken in
into account		to account
- There may be several shell completion changes found for the same command on pa rticular shell like fish		- There may be several shell completion changes found for the same command on pa rticular shell like fish
- It produces multiple ``complete`` modulefile commands		- It produces multiple ``complete`` modulefile commands
- But in ``__MODULES_LMSOURCESH`` variable, a single entry is recorded as a		- But in ``__MODULES_LMSOURCESH`` variable, a single entry is recorded as a si
single ``uncomplete`` command should be generated when unloading modulefile		ngle ``uncomplete`` command should be generated when unloading modulefile
- If completion definition is partial for a command when loading modulefile,		- If completion definition is partial for a command when loading modulefile, o
on fish shell previous completion definition is cleared right before setting th		n fish shell previous completion definition is cleared right before setting thos
ose defined in modulefile		e defined in modulefile
- When a completion definition difference is spotted after script evaluation		- When a completion definition difference is spotted after script evaluation,
, new completion definition is set which means previous definition is cleared (e		new completion definition is set which means previous definition is cleared (eit
ither intrinsically by shell or for fish due to the definition clear specificall		her intrinsically by shell or for fish due to the definition clear specifically
y added)		added)
- Fish shell allows nested definition of shell functions		- Fish shell allows nested definition of shell functions
- A separator string ``%ModulesSubShToMod%`` is printed between each functio		- A separator string ``%ModulesSubShToMod%`` is printed between each function
n definition not to get main and nested functions mixed up		definition not to get main and nested functions mixed up
- Such sub-separator is only applied on fish shell		- Such sub-separator is only applied on fish shell
- **FUTURE**: this feature may be extended to translate environment changes made by tools like Spack, Lmod or pkg-config. It may provide this way bridges betwee n different realms.		- **FUTURE**: this feature may be extended to translate environment changes made by tools like Spack, Lmod or pkg-config. It may provide this way bridges betwee n different realms.
		.. vim:set tabstop=2 shiftwidth=2 expandtab autoindent:
End of changes. 45 change blocks.
162 lines changed or deleted		157 lines changed or added

---