"Fossies" - the Fresh Open Source Software Archive 
Member "salt-2018.3.2/salt/output/highstate.py" (18 Jun 2018, 21722 Bytes) of package /linux/misc/salt-2018.3.2.tar.gz:
As a special service "Fossies" has tried to format the requested source page into HTML format using (guessed) Python source code syntax highlighting (style:
standard) with prefixed line numbers.
Alternatively you can here
view or
download the uninterpreted source code file.
For more information about "highstate.py" see the
Fossies "Dox" file reference documentation and the last
Fossies "Diffs" side-by-side code changes report:
2017.7.5_vs_2018.3.0.
1 # -*- coding: utf-8 -*-
2 '''
3 Outputter for displaying results of state runs
4 ==============================================
5
6 The return data from the Highstate command is a standard data structure
7 which is parsed by the highstate outputter to deliver a clean and readable
8 set of information about the HighState run on minions.
9
10 Two configurations can be set to modify the highstate outputter. These values
11 can be set in the master config to change the output of the ``salt`` command or
12 set in the minion config to change the output of the ``salt-call`` command.
13
14 state_verbose:
15 By default `state_verbose` is set to `True`, setting this to `False` will
16 instruct the highstate outputter to omit displaying anything in green, this
17 means that nothing with a result of True and no changes will not be printed
18 state_output:
19 The highstate outputter has six output modes,
20 ``full``, ``terse``, ``mixed``, ``changes`` and ``filter``
21 * The default is set to ``full``, which will display many lines of detailed
22 information for each executed chunk.
23 * If ``terse`` is used, then the output is greatly simplified and shown in
24 only one line.
25 * If ``mixed`` is used, then terse output will be used unless a state
26 failed, in which case full output will be used.
27 * If ``changes`` is used, then terse output will be used if there was no
28 error and no changes, otherwise full output will be used.
29 * If ``filter`` is used, then either or both of two different filters can be
30 used: ``exclude`` or ``terse``.
31 * for ``exclude``, state.highstate expects a list of states to be excluded (or ``None``)
32 followed by ``True`` for terse output or ``False`` for regular output.
33 Because of parsing nuances, if only one of these is used, it must still
34 contain a comma. For instance: `exclude=True,`.
35 * for ``terse``, state.highstate expects simply ``True`` or ``False``.
36 These can be set as such from the command line, or in the Salt config as
37 `state_output_exclude` or `state_output_terse`, respectively.
38 The output modes have one modifier:
39 ``full_id``, ``terse_id``, ``mixed_id``, ``changes_id`` and ``filter_id``
40 If ``_id`` is used, then the corresponding form will be used, but the value for ``name``
41 will be drawn from the state ID. This is useful for cases where the name
42 value might be very long and hard to read.
43 state_tabular:
44 If `state_output` uses the terse output, set this to `True` for an aligned
45 output format. If you wish to use a custom format, this can be set to a
46 string.
47
48 Example usage:
49
50 If ``state_output: filter`` is set in the configuration file:
51
52 .. code-block:: bash
53
54 salt '*' state.highstate exclude=None,True
55
56
57 means to exclude no states from the highstate and turn on terse output.
58
59 .. code-block:: bash
60
61 salt twd state.highstate exclude=problemstate1,problemstate2,False
62
63
64 means to exclude states ``problemstate1`` and ``problemstate2``
65 from the highstate, and use regular output.
66
67 Example output for the above highstate call when ``top.sls`` defines only
68 one other state to apply to minion ``twd``:
69
70 .. code-block:: text
71
72 twd:
73
74 Summary for twd
75 ------------
76 Succeeded: 1 (changed=1)
77 Failed: 0
78 ------------
79 Total states run: 1
80
81
82 Example output with no special settings in configuration files:
83
84 .. code-block:: text
85
86 myminion:
87 ----------
88 ID: test.ping
89 Function: module.run
90 Result: True
91 Comment: Module function test.ping executed
92 Changes:
93 ----------
94 ret:
95 True
96
97 Summary for myminion
98 ------------
99 Succeeded: 1
100 Failed: 0
101 ------------
102 Total: 0
103 '''
104
105 # Import python libs
106 from __future__ import absolute_import, print_function, unicode_literals
107 import pprint
108 import textwrap
109
110 # Import salt libs
111 import salt.utils.color
112 import salt.utils.data
113 import salt.utils.stringutils
114 import salt.output
115
116 # Import 3rd-party libs
117 from salt.ext import six
118
119 import logging
120
121 log = logging.getLogger(__name__)
122
123
124 def output(data, **kwargs): # pylint: disable=unused-argument
125 '''
126 The HighState Outputter is only meant to be used with the state.highstate
127 function, or a function that returns highstate return data.
128 '''
129 # Discard retcode in dictionary as present in orchestrate data
130 local_masters = [key for key in data.keys() if key.endswith('.local_master')]
131 orchestrator_output = 'retcode' in data.keys() and len(local_masters) == 1
132
133 if orchestrator_output:
134 del data['retcode']
135
136 # If additional information is passed through via the "data" dictionary to
137 # the highstate outputter, such as "outputter" or "retcode", discard it.
138 # We only want the state data that was passed through, if it is wrapped up
139 # in the "data" key, as the orchestrate runner does. See Issue #31330,
140 # pull request #27838, and pull request #27175 for more information.
141 if 'data' in data:
142 data = data.pop('data')
143
144 ret = [
145 _format_host(host, hostdata)[0]
146 for host, hostdata in six.iteritems(data)
147 ]
148 if ret:
149 return "\n".join(ret)
150 log.error(
151 'Data passed to highstate outputter is not a valid highstate return: %s',
152 data
153 )
154 # We should not reach here, but if we do return empty string
155 return ''
156
157
158 def _format_host(host, data):
159 host = salt.utils.data.decode(host)
160
161 colors = salt.utils.color.get_colors(
162 __opts__.get('color'),
163 __opts__.get('color_theme'))
164 tabular = __opts__.get('state_tabular', False)
165 rcounts = {}
166 rdurations = []
167 hcolor = colors['GREEN']
168 hstrs = []
169 nchanges = 0
170 strip_colors = __opts__.get('strip_colors', True)
171
172 if isinstance(data, int) or isinstance(data, six.string_types):
173 # Data in this format is from saltmod.function,
174 # so it is always a 'change'
175 nchanges = 1
176 hstrs.append(('{0} {1}{2[ENDC]}'
177 .format(hcolor, data, colors)))
178 hcolor = colors['CYAN'] # Print the minion name in cyan
179 if isinstance(data, list):
180 # Errors have been detected, list them in RED!
181 hcolor = colors['LIGHT_RED']
182 hstrs.append((' {0}Data failed to compile:{1[ENDC]}'
183 .format(hcolor, colors)))
184 for err in data:
185 if strip_colors:
186 err = salt.output.strip_esc_sequence(
187 salt.utils.data.decode(err)
188 )
189 hstrs.append(('{0}----------\n {1}{2[ENDC]}'
190 .format(hcolor, err, colors)))
191 if isinstance(data, dict):
192 # Verify that the needed data is present
193 data_tmp = {}
194 for tname, info in six.iteritems(data):
195 if isinstance(info, dict) and tname is not 'changes' and info and '__run_num__' not in info:
196 err = ('The State execution failed to record the order '
197 'in which all states were executed. The state '
198 'return missing data is:')
199 hstrs.insert(0, pprint.pformat(info))
200 hstrs.insert(0, err)
201 if isinstance(info, dict) and 'result' in info:
202 data_tmp[tname] = info
203 data = data_tmp
204 # Everything rendered as it should display the output
205 for tname in sorted(
206 data,
207 key=lambda k: data[k].get('__run_num__', 0)):
208 ret = data[tname]
209 # Increment result counts
210 rcounts.setdefault(ret['result'], 0)
211 rcounts[ret['result']] += 1
212 rduration = ret.get('duration', 0)
213 try:
214 rdurations.append(float(rduration))
215 except ValueError:
216 rduration, _, _ = rduration.partition(' ms')
217 try:
218 rdurations.append(float(rduration))
219 except ValueError:
220 log.error('Cannot parse a float from duration {0}'
221 .format(ret.get('duration', 0)))
222
223 tcolor = colors['GREEN']
224 orchestration = ret.get('__orchestration__', False)
225 schanged, ctext = _format_changes(ret['changes'], orchestration)
226 if not ctext and 'pchanges' in ret:
227 schanged, ctext = _format_changes(ret['pchanges'], orchestration)
228 nchanges += 1 if schanged else 0
229
230 # Skip this state if it was successful & diff output was requested
231 if __opts__.get('state_output_diff', False) and \
232 ret['result'] and not schanged:
233 continue
234
235 # Skip this state if state_verbose is False, the result is True and
236 # there were no changes made
237 if not __opts__.get('state_verbose', False) and \
238 ret['result'] and not schanged:
239 continue
240
241 if schanged:
242 tcolor = colors['CYAN']
243 if ret['result'] is False:
244 hcolor = colors['RED']
245 tcolor = colors['RED']
246 if ret['result'] is None:
247 hcolor = colors['LIGHT_YELLOW']
248 tcolor = colors['LIGHT_YELLOW']
249
250 state_output = __opts__.get('state_output', 'full').lower()
251 comps = tname.split('_|-')
252
253 if state_output.endswith('_id'):
254 # Swap in the ID for the name. Refs #35137
255 comps[2] = comps[1]
256
257 if state_output.startswith('filter'):
258 # By default, full data is shown for all types. However, return
259 # data may be excluded by setting state_output_exclude to a
260 # comma-separated list of True, False or None, or including the
261 # same list with the exclude option on the command line. For
262 # now, this option must include a comma. For example:
263 # exclude=True,
264 # The same functionality is also available for making return
265 # data terse, instead of excluding it.
266 cliargs = __opts__.get('arg', [])
267 clikwargs = {}
268 for item in cliargs:
269 if isinstance(item, dict) and '__kwarg__' in item:
270 clikwargs = item.copy()
271
272 exclude = clikwargs.get(
273 'exclude', __opts__.get('state_output_exclude', [])
274 )
275 if isinstance(exclude, six.string_types):
276 exclude = six.text_type(exclude).split(',')
277
278 terse = clikwargs.get(
279 'terse', __opts__.get('state_output_terse', [])
280 )
281 if isinstance(terse, six.string_types):
282 terse = six.text_type(terse).split(',')
283
284 if six.text_type(ret['result']) in terse:
285 msg = _format_terse(tcolor, comps, ret, colors, tabular)
286 hstrs.append(msg)
287 continue
288 if six.text_type(ret['result']) in exclude:
289 continue
290
291 elif any((
292 state_output.startswith('terse'),
293 state_output.startswith('mixed') and ret['result'] is not False, # only non-error'd
294 state_output.startswith('changes') and ret['result'] and not schanged # non-error'd non-changed
295 )):
296 # Print this chunk in a terse way and continue in the loop
297 msg = _format_terse(tcolor, comps, ret, colors, tabular)
298 hstrs.append(msg)
299 continue
300
301 state_lines = [
302 '{tcolor}----------{colors[ENDC]}',
303 ' {tcolor} ID: {comps[1]}{colors[ENDC]}',
304 ' {tcolor}Function: {comps[0]}.{comps[3]}{colors[ENDC]}',
305 ' {tcolor} Result: {ret[result]!s}{colors[ENDC]}',
306 ' {tcolor} Comment: {comment}{colors[ENDC]}',
307 ]
308 if __opts__.get('state_output_profile', True) and 'start_time' in ret:
309 state_lines.extend([
310 ' {tcolor} Started: {ret[start_time]!s}{colors[ENDC]}',
311 ' {tcolor}Duration: {ret[duration]!s}{colors[ENDC]}',
312 ])
313 # This isn't the prettiest way of doing this, but it's readable.
314 if comps[1] != comps[2]:
315 state_lines.insert(
316 3, ' {tcolor} Name: {comps[2]}{colors[ENDC]}')
317 # be sure that ret['comment'] is utf-8 friendly
318 try:
319 if not isinstance(ret['comment'], six.text_type):
320 ret['comment'] = six.text_type(ret['comment'])
321 except UnicodeDecodeError:
322 # If we got here, we're on Python 2 and ret['comment'] somehow
323 # contained a str type with unicode content.
324 ret['comment'] = salt.utils.stringutils.to_unicode(ret['comment'])
325 try:
326 comment = salt.utils.data.decode(ret['comment'])
327 comment = comment.strip().replace(
328 '\n',
329 '\n' + ' ' * 14)
330 except AttributeError: # Assume comment is a list
331 try:
332 comment = ret['comment'].join(' ').replace(
333 '\n',
334 '\n' + ' ' * 13)
335 except AttributeError:
336 # Comment isn't a list either, just convert to string
337 comment = six.text_type(ret['comment'])
338 comment = comment.strip().replace(
339 '\n',
340 '\n' + ' ' * 14)
341 # If there is a data attribute, append it to the comment
342 if 'data' in ret:
343 if isinstance(ret['data'], list):
344 for item in ret['data']:
345 comment = '{0} {1}'.format(comment, item)
346 elif isinstance(ret['data'], dict):
347 for key, value in ret['data'].items():
348 comment = '{0}\n\t\t{1}: {2}'.format(comment, key, value)
349 else:
350 comment = '{0} {1}'.format(comment, ret['data'])
351 for detail in ['start_time', 'duration']:
352 ret.setdefault(detail, '')
353 if ret['duration'] != '':
354 ret['duration'] = '{0} ms'.format(ret['duration'])
355 svars = {
356 'tcolor': tcolor,
357 'comps': comps,
358 'ret': ret,
359 'comment': salt.utils.data.decode(comment),
360 # This nukes any trailing \n and indents the others.
361 'colors': colors
362 }
363 hstrs.extend([sline.format(**svars) for sline in state_lines])
364 changes = ' Changes: ' + ctext
365 hstrs.append(('{0}{1}{2[ENDC]}'
366 .format(tcolor, changes, colors)))
367
368 if 'warnings' in ret:
369 rcounts.setdefault('warnings', 0)
370 rcounts['warnings'] += 1
371 wrapper = textwrap.TextWrapper(
372 width=80,
373 initial_indent=' ' * 14,
374 subsequent_indent=' ' * 14
375 )
376 hstrs.append(
377 ' {colors[LIGHT_RED]} Warnings: {0}{colors[ENDC]}'.format(
378 wrapper.fill('\n'.join(ret['warnings'])).lstrip(),
379 colors=colors
380 )
381 )
382
383 # Append result counts to end of output
384 colorfmt = '{0}{1}{2[ENDC]}'
385 rlabel = {True: 'Succeeded', False: 'Failed', None: 'Not Run', 'warnings': 'Warnings'}
386 count_max_len = max([len(six.text_type(x)) for x in six.itervalues(rcounts)] or [0])
387 label_max_len = max([len(x) for x in six.itervalues(rlabel)] or [0])
388 line_max_len = label_max_len + count_max_len + 2 # +2 for ': '
389 hstrs.append(
390 colorfmt.format(
391 colors['CYAN'],
392 '\nSummary for {0}\n{1}'.format(host, '-' * line_max_len),
393 colors
394 )
395 )
396
397 def _counts(label, count):
398 return '{0}: {1:>{2}}'.format(
399 label,
400 count,
401 line_max_len - (len(label) + 2)
402 )
403
404 # Successful states
405 changestats = []
406 if None in rcounts and rcounts.get(None, 0) > 0:
407 # test=True states
408 changestats.append(
409 colorfmt.format(
410 colors['LIGHT_YELLOW'],
411 'unchanged={0}'.format(rcounts.get(None, 0)),
412 colors
413 )
414 )
415 if nchanges > 0:
416 changestats.append(
417 colorfmt.format(
418 colors['GREEN'],
419 'changed={0}'.format(nchanges),
420 colors
421 )
422 )
423 if changestats:
424 changestats = ' ({0})'.format(', '.join(changestats))
425 else:
426 changestats = ''
427 hstrs.append(
428 colorfmt.format(
429 colors['GREEN'],
430 _counts(
431 rlabel[True],
432 rcounts.get(True, 0) + rcounts.get(None, 0)
433 ),
434 colors
435 ) + changestats
436 )
437
438 # Failed states
439 num_failed = rcounts.get(False, 0)
440 hstrs.append(
441 colorfmt.format(
442 colors['RED'] if num_failed else colors['CYAN'],
443 _counts(rlabel[False], num_failed),
444 colors
445 )
446 )
447
448 num_warnings = rcounts.get('warnings', 0)
449 if num_warnings:
450 hstrs.append(
451 colorfmt.format(
452 colors['LIGHT_RED'],
453 _counts(rlabel['warnings'], num_warnings),
454 colors
455 )
456 )
457 totals = '{0}\nTotal states run: {1:>{2}}'.format('-' * line_max_len,
458 sum(six.itervalues(rcounts)) - rcounts.get('warnings', 0),
459 line_max_len - 7)
460 hstrs.append(colorfmt.format(colors['CYAN'], totals, colors))
461
462 if __opts__.get('state_output_profile', True):
463 sum_duration = sum(rdurations)
464 duration_unit = 'ms'
465 # convert to seconds if duration is 1000ms or more
466 if sum_duration > 999:
467 sum_duration /= 1000
468 duration_unit = 's'
469 total_duration = 'Total run time: {0} {1}'.format(
470 '{0:.3f}'.format(sum_duration).rjust(line_max_len - 5),
471 duration_unit)
472 hstrs.append(colorfmt.format(colors['CYAN'], total_duration, colors))
473
474 if strip_colors:
475 host = salt.output.strip_esc_sequence(host)
476 hstrs.insert(0, ('{0}{1}:{2[ENDC]}'.format(hcolor, host, colors)))
477 return '\n'.join(hstrs), nchanges > 0
478
479
480 def _nested_changes(changes):
481 '''
482 Print the changes data using the nested outputter
483 '''
484 ret = '\n'
485 ret += salt.output.out_format(
486 changes,
487 'nested',
488 __opts__,
489 nested_indent=14)
490 return ret
491
492
493 def _format_changes(changes, orchestration=False):
494 '''
495 Format the changes dict based on what the data is
496 '''
497 if not changes:
498 return False, ''
499
500 if orchestration:
501 return True, _nested_changes(changes)
502
503 if not isinstance(changes, dict):
504 return True, 'Invalid Changes data: {0}'.format(changes)
505
506 ret = changes.get('ret')
507 if ret is not None and changes.get('out') == 'highstate':
508 ctext = ''
509 changed = False
510 for host, hostdata in six.iteritems(ret):
511 s, c = _format_host(host, hostdata)
512 ctext += '\n' + '\n'.join((' ' * 14 + l) for l in s.splitlines())
513 changed = changed or c
514 else:
515 changed = True
516 ctext = _nested_changes(changes)
517 return changed, ctext
518
519
520 def _format_terse(tcolor, comps, ret, colors, tabular):
521 '''
522 Terse formatting of a message.
523 '''
524 result = 'Clean'
525 if ret['changes']:
526 result = 'Changed'
527 if ret['result'] is False:
528 result = 'Failed'
529 elif ret['result'] is None:
530 result = 'Differs'
531 if tabular is True:
532 fmt_string = ''
533 if 'warnings' in ret:
534 fmt_string += '{c[LIGHT_RED]}Warnings:\n{w}{c[ENDC]}\n'.format(
535 c=colors, w='\n'.join(ret['warnings'])
536 )
537 fmt_string += '{0}'
538 if __opts__.get('state_output_profile', True) and 'start_time' in ret:
539 fmt_string += '{6[start_time]!s} [{6[duration]!s:>7} ms] '
540 fmt_string += '{2:>10}.{3:<10} {4:7} Name: {1}{5}'
541 elif isinstance(tabular, six.string_types):
542 fmt_string = tabular
543 else:
544 fmt_string = ''
545 if 'warnings' in ret:
546 fmt_string += '{c[LIGHT_RED]}Warnings:\n{w}{c[ENDC]}'.format(
547 c=colors, w='\n'.join(ret['warnings'])
548 )
549 fmt_string += ' {0} Name: {1} - Function: {2}.{3} - Result: {4}'
550 if __opts__.get('state_output_profile', True) and 'start_time' in ret:
551 fmt_string += ' Started: - {6[start_time]!s} Duration: {6[duration]!s} ms'
552 fmt_string += '{5}'
553
554 msg = fmt_string.format(tcolor,
555 comps[2],
556 comps[0],
557 comps[-1],
558 result,
559 colors['ENDC'],
560 ret)
561 return msg