"Fossies" - the Fresh Open Source Software Archive  

Source code changes of the file "docs/manual/installation/worker.rst" between
buildbot-3.0.2.tar.gz and buildbot-3.1.0.tar.gz

About: Buildbot is a continuous integration testing framework (Python-based). It supports also automation of complex build systems, application deployment, and management of sophisticated software-release processes.

worker.rst  (buildbot-3.0.2):worker.rst  (buildbot-3.1.0)
Worker Setup Worker Setup
============ ============
.. _Creating-a-worker: .. _Creating-a-worker:
Creating a worker Creating a worker
----------------- -----------------
Typically, you will be adding a worker to an existing buildmaster, to provide ad ditional architecture coverage. Typically, you will be adding a worker to an existing buildmaster, to provide ad ditional architecture coverage.
The Buildbot administrator will give you several pieces of information necessary to connect to the buildmaster. The Buildbot administrator will give you several pieces of information necessary to connect to the buildmaster.
You should also be somewhat familiar with the project being tested, so you can t roubleshoot build problems locally. You should also be somewhat familiar with the project being tested so that you c an troubleshoot build problems locally.
The Buildbot exists to make sure that the project's stated ``how to build it`` p rocess actually works. Buildbot exists to make sure that the project's stated ``how to build it`` proce ss actually works.
To this end, the worker should run in an environment just like that of your regu lar developers. To this end, the worker should run in an environment just like that of your regu lar developers.
Typically the project build process is documented somewhere (:file:`README`, :fi Typically the project's build process is documented somewhere (:file:`README`, :
le:`INSTALL`, etc), in a document that should mention all library dependencies a file:`INSTALL`, etc), in a document that should mention all library dependencies
nd contain a basic set of build instructions. and contain a basic set of build instructions.
This document will be useful as you configure the host and account in which the This document will be useful as you configure the host and account in which work
worker runs. er runs.
Here's a good checklist for setting up a worker: Here's a good checklist for setting up a worker:
1. Set up the account 1. Set up the account
It is recommended (although not mandatory) to set up a separate user account f or the worker. It is recommended (although not mandatory) to set up a separate user account f or the worker.
This account is frequently named ``buildbot`` or ``worker``. This account is frequently named ``buildbot`` or ``worker``.
This serves to isolate your personal working environment from that of the work er's, and helps to minimize the security threat posed by letting possibly-unknow n contributors run arbitrary code on your system. This serves to isolate your personal working environment from that of the work er's, and helps to minimize the security threat posed by letting possibly-unknow n contributors run arbitrary code on your system.
The account should have a minimum of fancy init scripts. The account should have a minimum of fancy init scripts.
skipping to change at line 47 skipping to change at line 47
(or not: if your worker is supposed to make sure that building without optiona l libraries still works, then don't install those libraries.) (or not: if your worker is supposed to make sure that building without optiona l libraries still works, then don't install those libraries.)
Again, these libraries don't necessarily have to be installed to a site-wide s hared location, but they must be available to your build process. Again, these libraries don't necessarily have to be installed to a site-wide s hared location, but they must be available to your build process.
Accomplishing this is usually very specific to the build process, so installin g them to :file:`/usr` or :file:`/usr/local` is usually the best approach. Accomplishing this is usually very specific to the build process, so installin g them to :file:`/usr` or :file:`/usr/local` is usually the best approach.
4. Test the build process 4. Test the build process
Follow the instructions in the :file:`INSTALL` document, in the worker's accou nt. Follow the instructions in the :file:`INSTALL` document, in the worker's accou nt.
Perform a full CVS (or whatever) checkout, configure, make, run tests, etc. Perform a full CVS (or whatever) checkout, configure, make, run tests, etc.
Confirm that the build works without manual fussing. Confirm that the build works without manual fussing.
If it doesn't work when you do it by hand, it will be unlikely to work when th e Buildbot attempts to do it in an automated fashion. If it doesn't work when you do it manually, it will be unlikely to work when B uildbot attempts to do it in an automated fashion.
5. Choose a base directory 5. Choose a base directory
This should be somewhere in the worker's account, typically named after the pr oject which is being tested. This should be somewhere in the worker's account, typically named after the pr oject which is being tested.
The worker will not touch any file outside of this directory. The worker will not touch any file outside of this directory.
Something like :file:`~/Buildbot` or :file:`~/Workers/fooproject` is appropria te. Something like :file:`~/Buildbot` or :file:`~/Workers/fooproject` is appropria te.
6. Get the buildmaster host/port, botname, and password 6. Get the buildmaster host/port, workername, and password
When the Buildbot admin configures the buildmaster to accept and use your work er, they will provide you with the following pieces of information: When the Buildbot admin configures the buildmaster to accept and use your work er, they will provide you with the following pieces of information:
* your worker's name * your worker's name
* the password assigned to your worker * the password assigned to your worker
* the hostname and port number of the buildmaster, i.e. `<http://buildbot.exam ple.org:8007>`__ * the hostname and port number of the buildmaster
7. Create the worker 7. Create the worker
Now run the 'worker' command as follows: Now run the 'worker' command as follows:
:samp:`buildbot-worker create-worker {BASEDIR} {MASTERHOST}:{PORT} {WORKER NAME} {PASSWORD}` :samp:`buildbot-worker create-worker {BASEDIR} {MASTERHOST}:{PORT} {WORKER NAME} {PASSWORD}`
This will create the base directory and a collection of files inside, includin g the :file:`buildbot.tac` file that contains all the information you passed to the :command:`buildbot` command. This will create the base directory and a collection of files inside, includin g the :file:`buildbot.tac` file that contains all the information you passed to the :command:`buildbot-worker` command.
8. Fill in the hostinfo files 8. Fill in the hostinfo files
When it first connects, the worker will send a few files up to the buildmaster which describe the host that it is running on. When it first connects, the worker will send a few files up to the buildmaster which describe the host that it is running on.
These files are presented on the web status display so that developers have mo re information to reproduce any test failures that are witnessed by the Buildbot . These files are presented on the web status display so that developers have mo re information to reproduce any test failures that are witnessed by the Buildbot .
There are sample files in the :file:`info` subdirectory of the Buildbot's base directory. There are sample files in the :file:`info` subdirectory of the Buildbot's base directory.
You should edit these to correctly describe you and your host. You should edit these to correctly describe you and your host.
:file:`{BASEDIR}/info/admin` should contain your name and email address. :file:`{BASEDIR}/info/admin` should contain your name and email address.
This is the ``worker admin address``, and will be visible from the build statu s page (so you may wish to munge it a bit if address-harvesting spambots are a c oncern). This is the ``worker admin address``, and will be visible from the build statu s page (so you may wish to munge it a bit if address-harvesting spambots are a c oncern).
skipping to change at line 124 skipping to change at line 124
If you want build products to be *writable* by other accounts too, use ``--u mask=0o000``, but this is likely to be a security problem. If you want build products to be *writable* by other accounts too, use ``--u mask=0o000``, but this is likely to be a security problem.
.. option:: --keepalive .. option:: --keepalive
This is a number that indicates how frequently ``keepalive`` messages should be sent from the worker to the buildmaster, expressed in seconds. This is a number that indicates how frequently ``keepalive`` messages should be sent from the worker to the buildmaster, expressed in seconds.
The default (600) causes a message to be sent to the buildmaster at least on ce every 10 minutes. The default (600) causes a message to be sent to the buildmaster at least on ce every 10 minutes.
To set this to a lower value, use e.g. ``--keepalive=120``. To set this to a lower value, use e.g. ``--keepalive=120``.
If the worker is behind a NAT box or stateful firewall, these messages may h elp to keep the connection alive: some NAT boxes tend to forget about a connecti on if it has not been used in a while. If the worker is behind a NAT box or stateful firewall, these messages may h elp to keep the connection alive: some NAT boxes tend to forget about a connecti on if it has not been used in a while.
When this happens, the buildmaster will think that the worker has disappeare d, and builds will time out. When this happens, the buildmaster will think that the worker has disappeare d, and builds will time out.
Meanwhile the worker will not realize than anything is wrong. Meanwhile the worker will not realize that anything is wrong.
.. option:: --maxdelay .. option:: --maxdelay
This is a number that indicates the maximum amount of time the worker will w ait between connection attempts, expressed in seconds. This is a number that indicates the maximum amount of time the worker will w ait between connection attempts, expressed in seconds.
The default (300) causes the worker to wait at most 5 minutes before trying to connect to the buildmaster again. The default (300) causes the worker to wait at most 5 minutes before trying to connect to the buildmaster again.
.. option:: --maxretries .. option:: --maxretries
This is a number that indicates the maximum number of time the worker will m ake connection attempts. This is a number that indicates the maximum number of times the worker will make connection attempts.
After that amount, the worker process will stop. After that amount, the worker process will stop.
This option is useful for :ref:`Latent-Workers` to avoid consuming resources in case of misconfiguration or master failure. This option is useful for :ref:`Latent-Workers` to avoid consuming resources in case of misconfiguration or master failure.
For VM based latent workers, the user is responsible for halting the system For VM based latent workers, the user is responsible for halting the system
when Buildbot worker has exited. when the Buildbot worker has exited.
This feature is heavily OS dependent, and cannot be managed by Buildbot work This feature is heavily OS dependent, and cannot be managed by the Buildbot
er. worker.
For example with systemd_, one can add ``ExecStopPost=shutdown now`` to the For example, with systemd_, one can add ``ExecStopPost=shutdown now`` to the
Buildbot worker service unit configuration. Buildbot worker service unit configuration.
.. _systemd: https://www.freedesktop.org/software/systemd/man/systemd.servic e.html .. _systemd: https://www.freedesktop.org/software/systemd/man/systemd.servic e.html
.. option:: --log-size .. option:: --log-size
This is the size in bytes when to rotate the Twisted log files. This is the size in bytes when exceeded to rotate the Twisted log files.
.. option:: --log-count .. option:: --log-count
This is the number of log rotations to keep around. This is the number of log rotations to keep around.
You can either specify a number or ``None`` to keep all :file:`twistd.log` f iles around. You can either specify a number or ``None`` to keep all :file:`twistd.log` f iles around.
The default is 10. The default is 10.
.. option:: --allow-shutdown .. option:: --allow-shutdown
Can also be passed directly to the Worker constructor in :file:`buildbot.tac `. Can also be passed directly to the worker constructor in :file:`buildbot.tac `.
If set, it allows the worker to initiate a graceful shutdown, meaning that i t will ask the master to shut down the worker when the current build, if any, is complete. If set, it allows the worker to initiate a graceful shutdown, meaning that i t will ask the master to shut down the worker when the current build, if any, is complete.
Setting allow_shutdown to ``file`` will cause the worker to watch :file:`shu tdown.stamp` in basedir for updates to its mtime. Setting allow_shutdown to ``file`` will cause the worker to watch :file:`shu tdown.stamp` in basedir for updates to its mtime.
When the mtime changes, the worker will request a graceful shutdown from the master. When the mtime changes, the worker will request a graceful shutdown from the master.
The file does not need to exist prior to starting the worker. The file does not need to exist prior to starting the worker.
Setting allow_shutdown to ``signal`` will set up a SIGHUP handler to start a graceful shutdown. Setting allow_shutdown to ``signal`` will set up a SIGHUP handler to start a graceful shutdown.
When the signal is received, the worker will request a graceful shutdown fro m the master. When the signal is received, the worker will request a graceful shutdown fro m the master.
The default value is ``None``, in which case this feature will be disabled. The default value is ``None``, in which case this feature will be disabled.
skipping to change at line 208 skipping to change at line 208
.. _Worker-TLS-Config: .. _Worker-TLS-Config:
Worker TLS Configuration Worker TLS Configuration
~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~
``tls`` ``tls``
See ``--useTls`` option above as an alternative to setting the ``conneciton_ string`` manually. See ``--useTls`` option above as an alternative to setting the ``conneciton_ string`` manually.
``connection_string`` ``connection_string``
For TLS connections to the master the ``connection_string``-argument must be used to ``Worker.__init__`` function. ``buildmaster_host`` and ``port`` must th en be ``None``. For TLS connections to the master, the ``connection_string``-argument must b e passed to the worker constructor. ``buildmaster_host`` and ``port`` must then be ``None``.
``connection_string`` will be used to create a client endpoint with clientFr omString_. An example of ``connection_string`` is ``"TLS:buildbot-master.com:998 9"``. ``connection_string`` will be used to create a client endpoint with clientFr omString_. An example of ``connection_string`` is ``"TLS:buildbot-master.com:998 9"``.
See more about how to formulate the connection string in ConnectionStrings_. See more about how to formulate the connection string in ConnectionStrings_.
Example TLS connection string: Example TLS connection string:
.. code-block:: python .. code-block:: python
s = Worker(None, None, workername, passwd, basedir, keepalive, s = Worker(None, None, workername, passwd, basedir, keepalive,
connection_string='TLS:buildbot-master.com:9989') connection_string='TLS:buildbot-master.com:9989')
Make sure the worker trusts the masters certificate. If you have an non-auth Make sure the worker trusts the certificate of the master. If you have a non
oritative certificate -authoritative certificate
(CA is self-signed) the trustRoot parameter can be used. (CA is self-signed), the trustRoots parameter can be used.
.. code-block:: python .. code-block:: python
s = Worker(None, None, workername, passwd, basedir, keepalive, s = Worker(None, None, workername, passwd, basedir, keepalive,
connection_string= connection_string=
'TLS:buildbot-master.com:9989:trustRoots=/dir-with-ca-certs') 'TLS:buildbot-master.com:9989:trustRoots=/dir-with-ca-certs')
It must point to a directory with PEM-encoded certificates in files with fil e ending .pem. For example: It must point to a directory with PEM-encoded certificates. For example:
.. code-block:: bash .. code-block:: bash
$ cat /dir-with-ca-certs/ca.pem $ cat /dir-with-ca-certs/ca.pem
-----BEGIN CERTIFICATE----- -----BEGIN CERTIFICATE-----
MIIE9DCCA9ygAwIBAgIJALEqLrC/m1w3MA0GCSqGSIb3DQEBCwUAMIGsMQswCQYD MIIE9DCCA9ygAwIBAgIJALEqLrC/m1w3MA0GCSqGSIb3DQEBCwUAMIGsMQswCQYD
VQQGEwJaWjELMAkGA1UECBMCUUExEDAOBgNVBAcTB05vd2hlcmUxETAPBgNVBAoT VQQGEwJaWjELMAkGA1UECBMCUUExEDAOBgNVBAcTB05vd2hlcmUxETAPBgNVBAoT
CEJ1aWxkYm90MRkwFwYDVQQLExBEZXZlbG9wbWVudCBUZWFtMRQwEgYDVQQDEwtC CEJ1aWxkYm90MRkwFwYDVQQLExBEZXZlbG9wbWVudCBUZWFtMRQwEgYDVQQDEwtC
dWlsZGJvdCBDQTEQMA4GA1UEKRMHRWFzeVJTQTEoMCYGCSqGSIb3DQEJARYZYnVp dWlsZGJvdCBDQTEQMA4GA1UEKRMHRWFzeVJTQTEoMCYGCSqGSIb3DQEJARYZYnVp
bGRib3RAaW50ZWdyYXRpb24udGVzdDAeFw0xNjA5MDIxMjA5NTJaFw0yNjA4MzEx bGRib3RAaW50ZWdyYXRpb24udGVzdDAeFw0xNjA5MDIxMjA5NTJaFw0yNjA4MzEx
skipping to change at line 265 skipping to change at line 265
EDAOBgNVBCkTB0Vhc3lSU0ExKDAmBgkqhkiG9w0BCQEWGWJ1aWxkYm90QGludGVn EDAOBgNVBCkTB0Vhc3lSU0ExKDAmBgkqhkiG9w0BCQEWGWJ1aWxkYm90QGludGVn
cmF0aW9uLnRlc3SCCQCxKi6wv5tcNzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEB cmF0aW9uLnRlc3SCCQCxKi6wv5tcNzAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEB
CwUAA4IBAQCJGJVMAmwZRK/mRqm9E0e3s4YGmYT2jwX5IX17XljEy+1cS4huuZW2 CwUAA4IBAQCJGJVMAmwZRK/mRqm9E0e3s4YGmYT2jwX5IX17XljEy+1cS4huuZW2
33CFpslkT1MN/r8IIZWilxT/lTujHyt4eERGjE1oRVKU8rlTH8WUjFzPIVu7nkte 33CFpslkT1MN/r8IIZWilxT/lTujHyt4eERGjE1oRVKU8rlTH8WUjFzPIVu7nkte
09abqynAoec8aQukg79NRCY1l/E2/WzfnUt3yTgKPfZmzoiN0K+hH4gVlWtrizPA 09abqynAoec8aQukg79NRCY1l/E2/WzfnUt3yTgKPfZmzoiN0K+hH4gVlWtrizPA
LaGwoslYYTA6jHNEeMm8OQLNf17OTmAa7EpeIgVpLRCieI9S3JIG4WYU8fVkeuiU LaGwoslYYTA6jHNEeMm8OQLNf17OTmAa7EpeIgVpLRCieI9S3JIG4WYU8fVkeuiU
cB439SdixU4cecVjNfFDpq6JM8N6+DQoYOSNRt9Dy0ioGyx5D4lWoIQ+BmXQENal cB439SdixU4cecVjNfFDpq6JM8N6+DQoYOSNRt9Dy0ioGyx5D4lWoIQ+BmXQENal
gw+XLyejeNTNgLOxf9pbNYMJqxhkTkoE gw+XLyejeNTNgLOxf9pbNYMJqxhkTkoE
-----END CERTIFICATE----- -----END CERTIFICATE-----
Using TCP in ``connection_string`` is the equivalent as using the ``buildmas ter_host`` and ``port`` arguments. Using TCP in ``connection_string`` is the equivalent to using the ``buildmas ter_host`` and ``port`` arguments.
.. code-block:: python .. code-block:: python
s = Worker(None, None, workername, passwd, basedir, keepalive s = Worker(None, None, workername, passwd, basedir, keepalive
connection_string='TCP:buildbot-master.com:9989') connection_string='TCP:buildbot-master.com:9989')
is equivalent to is equivalent to
.. code-block:: python .. code-block:: python
 End of changes. 16 change blocks. 
27 lines changed or deleted 27 lines changed or added

Home  |  About  |  Features  |  All  |  Newest  |  Dox  |  Diffs  |  RSS Feeds  |  Screenshots  |  Comments  |  Imprint  |  Privacy  |  HTTP(S)