repositories.md (poetry-1.1.15) | : | repositories.md (poetry-1.2.0) | ||
---|---|---|---|---|
skipping to change at line 14 | skipping to change at line 14 | |||
type: docs | type: docs | |||
layout: "docs" | layout: "docs" | |||
menu: | menu: | |||
docs: | docs: | |||
weight: 50 | weight: 50 | |||
--- | --- | |||
# Repositories | # Repositories | |||
## Using the PyPI repository | Poetry supports the use of [PyPI](https://pypi.org) and private repositories for | |||
discovery of | ||||
packages as well as for publishing your projects. | ||||
By default, Poetry is configured to use the [PyPI](https://pypi.org) repository, | By default, Poetry is configured to use the [PyPI](https://pypi.org) repository, | |||
for package installation and publishing. | for package installation and publishing. | |||
So, when you add dependencies to your project, Poetry will assume they are avail able | So, when you add dependencies to your project, Poetry will assume they are avail able | |||
on PyPI. | on PyPI. | |||
This represents most cases and will likely be enough for most users. | This represents most cases and will likely be enough for most users. | |||
## Using a private repository | ### Private Repository Example | |||
However, at times, you may need to keep your package private while still being | #### Installing from private package sources | |||
able to share it with your teammates. In this case, you will need to use a priva | By default, Poetry discovers and installs packages from [PyPI](https://pypi.org) | |||
te | . But, you want to | |||
repository. | install a dependency to your project for a [simple API repository](#simple-api-r | |||
epository)? Let's | ||||
do it. | ||||
### Adding a repository | First, [configure](#project-configuration) the [package source](#package-source) | |||
repository to your | ||||
project. | ||||
Adding a new repository is easy with the `config` command. | ```bash | |||
poetry source add foo https://pypi.example.org/simple/ | ||||
``` | ||||
Then, assuming the repository requires authentication, configure credentials for | ||||
it. | ||||
```bash | ||||
poetry config http-basic.foo <username> <password> | ||||
``` | ||||
{{% warning %}} | ||||
Depending on your system configuration, credentials might be saved in your comma | ||||
nd line history. | ||||
Many shells do not save commands to history when they are prefixed by a space ch | ||||
aracter. For more information, please refer to your shell's documentation. | ||||
{{% /warning %}} | ||||
Once this is done, you can add dependencies to your project from this source. | ||||
```bash | ||||
poetry add --source foo private-package | ||||
``` | ||||
#### Publishing to a private repository | ||||
Great, now all that is left is to publish your package. Assuming you'd want to s | ||||
hare it privately | ||||
with your team, you can configure the | ||||
[Upload API](https://warehouse.pypa.io/api-reference/legacy.html#upload-api) end | ||||
point for your | ||||
[publishable repository](#publishable-repository). | ||||
```bash | ||||
poetry config repositories.foo https://pypi.example.org/legacy/ | ||||
``` | ||||
{{% note %}} | ||||
If you need to use a different credential for your [package source](#package-sou | ||||
rces), then it is | ||||
recommended to use a different name for your publishing repository. | ||||
```bash | ||||
poetry config repositories.foo-pub https://pypi.example.org/legacy/ | ||||
poetry config http-basic.foo-pub <username> <password> | ||||
``` | ||||
{{% /note %}} | ||||
Now, all the is left is to build and publish your project using the | ||||
[`publish`]({{< relref "cli#publish" >}}). | ||||
```bash | ||||
poetry publish --build --repository foo-pub | ||||
``` | ||||
## Package Sources | ||||
By default, Poetry is configured to use the Python ecosystem's canonical package | ||||
index | ||||
[PyPI](https://pypi.org). | ||||
{{% note %}} | ||||
With the exception of the implicitly configured source for [PyPI](https://pypi.o | ||||
rg) named `pypi`, | ||||
package sources are local to a project and must be configured within the project | ||||
's | ||||
[`pyproject.toml`]({{< relref "pyproject" >}}) file. This is **not** the same co | ||||
nfiguration used | ||||
when publishing a package. | ||||
{{% /note %}} | ||||
### Project Configuration | ||||
These package sources maybe managed using the [`source`]({{< relref "cli#source" | ||||
>}}) command for | ||||
your project. | ||||
```bash | ||||
poetry source add foo https://foo.bar/simple/ | ||||
``` | ||||
This will generate the following configuration snippet in your | ||||
[`pyproject.toml`]({{< relref "pyproject" >}}) file. | ||||
```toml | ||||
[[tool.poetry.source]] | ||||
name = "foo" | ||||
url = "https://foo.bar/simple/" | ||||
default = false | ||||
secondary = false | ||||
``` | ||||
{{% warning %}} | ||||
If package sources are defined for a project, these will take precedence over | ||||
[PyPI](https://pypi.org). If you do not want this to be the case, you should dec | ||||
lare **all** package | ||||
sources to be [secondary](#secondary-package-sources). | ||||
{{% /warning %}} | ||||
See [Supported Package Sources](#supported-package-sources) for source type spec | ||||
ific information. | ||||
{{% note %}} | ||||
If your package source requires [credentials](#configuring-credentials) or | ||||
[certificates](#certificates), please refer to the relevant sections below. | ||||
{{% /note %}} | ||||
#### Default Package Source | ||||
By default, Poetry configures [PyPI](https://pypi.org) as the default package so | ||||
urce for your | ||||
project. You can alter this behaviour and exclusively look up packages only from | ||||
the configured | ||||
package sources by adding a **single** source with `default = true`. | ||||
```bash | ||||
poetry source add --default foo https://foo.bar/simple/ | ||||
``` | ||||
{{% warning %}} | ||||
Configuring a custom package source as default, will effectively disable [PyPI]( | ||||
https://pypi.org) | ||||
as a package source for your project. | ||||
{{% /warning %}} | ||||
#### Secondary Package Sources | ||||
If package sources are configured as secondary, all it means is that these will | ||||
be given a lower | ||||
priority when selecting compatible package distribution that also exists in your | ||||
default package | ||||
source. | ||||
You can configure a package source as a secondary source with `secondary = true` | ||||
in your package | ||||
source configuration. | ||||
```bash | ||||
poetry source add --secondary foo https://foo.bar/simple/ | ||||
``` | ||||
There can be more than one secondary package source. | ||||
{{% note %}} | ||||
All package sources (including secondary sources) will be searched during the pa | ||||
ckage lookup | ||||
process. These network requests will occur for all sources, regardless of if the | ||||
package is | ||||
found at one or more sources. | ||||
If you wish to avoid this, you may explicitly specify which source to search in | ||||
for a particular | ||||
package. | ||||
```bash | ||||
poetry add --source pypi httpx | ||||
``` | ||||
{{% /note %}} | ||||
### Supported Package Sources | ||||
#### Python Package Index (PyPI) | ||||
Poetry interacts with [PyPI](https://pypi.org) via its | ||||
[JSON API](https://warehouse.pypa.io/api-reference/json.html). This is used to r | ||||
etrieve a requested | ||||
package's versions, metadata, files, etc. | ||||
{{% note %}} | ||||
If the the package's published metadata is invalid, Poetry will download the ava | ||||
ilable bdist/sdist to | ||||
inspect it locally to identify the relevant metadata. | ||||
{{% /note %}} | ||||
If you want to explicitly select a package from [PyPI](https://pypi.org) you can | ||||
use the `--source` | ||||
option with the [`add`]({{< relref "cli#add" >}}) command, like shown below. | ||||
```bash | ||||
poetry add --source pypi httpx@^0.22.0 | ||||
``` | ||||
This will generate the following configuration snippet in your `pyproject.toml` | ||||
file. | ||||
```toml | ||||
httpx = {version = "^0.22.0", source = "pypi"} | ||||
``` | ||||
{{% warning %}} | ||||
If any source within a project is configured with `default = true`, The implicit | ||||
`pypi` source will | ||||
be disabled and not used for any packages. | ||||
{{% /warning %}} | ||||
#### Simple API Repository | ||||
Poetry can fetch and install package dependencies from public or private custom | ||||
repositories that | ||||
implement the simple repository API as described in [PEP 503](https://peps.pytho | ||||
n.org/pep-0503/). | ||||
{{% warning %}} | ||||
When using sources that distributes large wheels without providing file checksum | ||||
in file URLs, | ||||
Poetry will download each candidate wheel at least once in order to generate the | ||||
checksum. This can | ||||
manifest as long dependency resolution times when adding packages from this sour | ||||
ce. | ||||
{{% /warning %}} | ||||
These package sources maybe configured via the following command in your project | ||||
. | ||||
```bash | ||||
poetry source add testpypi https://test.pypi.org/simple/ | ||||
``` | ||||
{{% note %}} | ||||
Note the trailing `/simple/`. This is important when configuring | ||||
[PEP 503](https://peps.python.org/pep-0503/) compliant package sources. | ||||
{{% /note %}} | ||||
In addition to [PEP 503](https://peps.python.org/pep-0503/), Poetry can also han | ||||
dle simple API | ||||
repositories that implement [PEP 658](https://peps.python.org/pep-0658/) (*Intro | ||||
duced in 1.2.0*). | ||||
This is helpful in reducing dependency resolution time for packages from these s | ||||
ources as Poetry can | ||||
avoid having to download each candidate distribution, in order to determine asso | ||||
ciated metadata. | ||||
{{% note %}} | ||||
*Why does Poetry insist on downloading all candidate distributions for all platf | ||||
orms when metadata | ||||
is not available?* | ||||
The need for this stems from the fact that Poetry's lock file is platform-agnost | ||||
ic. This means, in | ||||
order to resolve dependencies for a project, Poetry needs metadata for all platf | ||||
orm specific | ||||
distributions. And when this metadata is not readily available, downloading the | ||||
distribution and | ||||
inspecting it locally is the only remaining option. | ||||
{{% /note %}} | ||||
#### Single Page Link Source | ||||
*Introduced in 1.2.0* | ||||
Some projects choose to release their binary distributions via a single page lin | ||||
k source that | ||||
partially follows the structure of a package page in [PEP 503](https://peps.pyth | ||||
on.org/pep-0503/). | ||||
These package sources maybe configured via the following command in your project | ||||
. | ||||
```bash | ```bash | |||
poetry config repositories.foo https://foo.bar/simple/ | poetry source add jax https://storage.googleapis.com/jax-releases/jax_releases.h tml | |||
``` | ``` | |||
This will set the url for repository `foo` to `https://foo.bar/simple/`. | {{% note %}} | |||
### Configuring credentials | All caveats regarding slower resolution times described for simple API repositor | |||
ies do apply here as | ||||
well. | ||||
{{% /note %}} | ||||
## Publishable Repositories | ||||
Poetry treats repositories to which you publish packages as user specific and no | ||||
t project specific | ||||
configuration unlike [package sources](#package-sources). Poetry, today, only su | ||||
pports the | ||||
[Legacy Upload API](https://warehouse.pypa.io/api-reference/legacy.html#upload-a | ||||
pi) when publishing | ||||
your project. | ||||
These are configured using the [`config`]({{< relref "cli#config" >}}) command, | ||||
under the | ||||
`repositories` key. | ||||
```bash | ||||
poetry config repositories.testpypi https://test.pypi.org/legacy/ | ||||
``` | ||||
{{% note %}} | ||||
[Legacy Upload API](https://warehouse.pypa.io/api-reference/legacy.html#upload-a | ||||
pi) URLs are | ||||
typically different to the same one provided by the repository for the simple AP | ||||
I. You'll note that | ||||
in the example of [Test PyPI](https://test.pypi.org/), both the host (`test.pypi | ||||
.org`) as | ||||
well as the path (`/legacy`) are different to it's simple API (`https://test.pyp | ||||
i.org/simple`). | ||||
{{% /note %}} | ||||
## Configuring Credentials | ||||
If you want to store your credentials for a specific repository, you can do so e asily: | If you want to store your credentials for a specific repository, you can do so e asily: | |||
```bash | ```bash | |||
poetry config http-basic.foo username password | poetry config http-basic.foo <username> <password> | |||
``` | ``` | |||
If you do not specify the password you will be prompted to write it. | If you do not specify the password you will be prompted to write it. | |||
{{% note %}} | {{% note %}} | |||
To publish to PyPI, you can set your credentials for the repository named `pypi` . | To publish to PyPI, you can set your credentials for the repository named `pypi` . | |||
Note that it is recommended to use [API tokens](https://pypi.org/help/#apitoken) | Note that it is recommended to use [API tokens](https://pypi.org/help/#apitoken) | |||
when uploading packages to PyPI. | when uploading packages to PyPI. | |||
Once you have created a new token, you can tell Poetry to use it: | Once you have created a new token, you can tell Poetry to use it: | |||
```bash | ```bash | |||
poetry config pypi-token.pypi my-token | poetry config pypi-token.pypi my-token | |||
``` | ``` | |||
If you still want to use you username and password, you can do so with the follo wing | If you still want to use your username and password, you can do so with the foll owing | |||
call to `config`. | call to `config`. | |||
```bash | ```bash | |||
poetry config http-basic.pypi username password | poetry config http-basic.pypi <username> <password> | |||
``` | ``` | |||
{{% /note %}} | {{% /note %}} | |||
You can also specify the username and password when using the `publish` command | You can also specify the username and password when using the `publish` command | |||
with the `--username` and `--password` options. | with the `--username` and `--password` options. | |||
If a system keyring is available and supported, the password is stored to and re trieved from the keyring. In the above example, the credential will be stored us ing the name `poetry-repository-pypi`. If access to keyring fails or is unsuppor ted, this will fall back to writing the password to the `auth.toml` file along w ith the username. | If a system keyring is available and supported, the password is stored to and re trieved from the keyring. In the above example, the credential will be stored us ing the name `poetry-repository-pypi`. If access to keyring fails or is unsuppor ted, this will fall back to writing the password to the `auth.toml` file along w ith the username. | |||
Keyring support is enabled using the [keyring library](https://pypi.org/project/ keyring/). For more information on supported backends refer to the [library docu mentation](https://keyring.readthedocs.io/en/latest/?badge=latest). | Keyring support is enabled using the [keyring library](https://pypi.org/project/ keyring/). For more information on supported backends refer to the [library docu mentation](https://keyring.readthedocs.io/en/latest/?badge=latest). | |||
{{% note %}} | ||||
Poetry will fallback to Pip style use of keyring so that backends like | ||||
Microsoft's [artifacts-keyring](https://pypi.org/project/artifacts-keyring/) get | ||||
a chance to retrieve | ||||
valid credentials. It will need to be properly installed into Poetry's virtualen | ||||
v, | ||||
preferably by installing a plugin. | ||||
If you are letting Poetry manage your virtual environments you will want a virtu | ||||
alenv | ||||
seeder installed in Poetry's virtualenv that installs the desired keyring backen | ||||
d | ||||
during `poetry install`. To again use Azure DevOps as an example: [azure-devops- | ||||
artifacts-helpers](https://pypi.org/project/azure-devops-artifacts-helpers/) | ||||
provides such a seeder. This would of course best achieved by installing a Poetr | ||||
y plugin | ||||
if it exists for you use case instead of doing it yourself. | ||||
{{% /note %}} | ||||
Alternatively, you can use environment variables to provide the credentials: | Alternatively, you can use environment variables to provide the credentials: | |||
```bash | ```bash | |||
export POETRY_PYPI_TOKEN_PYPI=my-token | export POETRY_PYPI_TOKEN_PYPI=my-token | |||
export POETRY_HTTP_BASIC_PYPI_USERNAME=username | export POETRY_HTTP_BASIC_PYPI_USERNAME=<username> | |||
export POETRY_HTTP_BASIC_PYPI_PASSWORD=password | export POETRY_HTTP_BASIC_PYPI_PASSWORD=<password> | |||
``` | ``` | |||
See [Using environment variables]({{< relref "configuration#using-environment-va riables" >}}) for more information | See [Using environment variables]({{< relref "configuration#using-environment-va riables" >}}) for more information | |||
on how to configure Poetry with environment variables. | on how to configure Poetry with environment variables. | |||
#### Custom certificate authority and mutual TLS authentication | If your password starts with a dash (e.g. randomly generated tokens in a CI envi | |||
ronment), it will be parsed as a | ||||
command line option instead of a password. | ||||
You can prevent this by adding double dashes to prevent any following argument f | ||||
rom being parsed as an option. | ||||
```bash | ||||
poetry config -- http-basic.pypi myUsername -myPasswordStartingWithDash | ||||
``` | ||||
## Certificates | ||||
### Custom certificate authority and mutual TLS authentication | ||||
Poetry supports repositories that are secured by a custom certificate authority as well as those that require | Poetry supports repositories that are secured by a custom certificate authority as well as those that require | |||
certificate-based client authentication. The following will configure the "foo" repository to validate the repository's | certificate-based client authentication. The following will configure the "foo" repository to validate the repository's | |||
certificate using a custom certificate authority and use a client certificate (n ote that these config variables do not | certificate using a custom certificate authority and use a client certificate (n ote that these config variables do not | |||
both need to be set): | both need to be set): | |||
```bash | ```bash | |||
poetry config certificates.foo.cert /path/to/ca.pem | poetry config certificates.foo.cert /path/to/ca.pem | |||
poetry config certificates.foo.client-cert /path/to/client.pem | poetry config certificates.foo.client-cert /path/to/client.pem | |||
``` | ``` | |||
### Install dependencies from a private repository | {{% note %}} | |||
The value of `certificates.<repository>.cert` can be set to `false` if certifica | ||||
Now that you can publish to your private repository, you need to be able to | te verification is | |||
install dependencies from it. | required to be skipped. This is useful for cases where a package source with sel | |||
f-signed certificates | ||||
For that, you have to edit your `pyproject.toml` file, like so | are used. | |||
```toml | ```bash | |||
[[tool.poetry.source]] | poetry config certificates.foo.cert false | |||
name = "foo" | ||||
url = "https://foo.bar/simple/" | ||||
``` | ``` | |||
From now on, Poetry will also look for packages in your private repository. | {{% warning %}} | |||
Disabling certificate verification is not recommended as it is does not conform | ||||
to security | ||||
best practices. | ||||
{{% /warning %}} | ||||
{{% /note %}} | ||||
{{% note %}} | ## Caches | |||
Any custom repository will have precedence over PyPI. | ||||
If you still want PyPI to be your primary source for your packages | Poetry employs multiple caches for package sources in order to improve user expe | |||
you can declare custom repositories as secondary. | rience and avoid duplicate network | |||
requests. | ||||
```toml | The first level cache is a [Cache-Control](https://developer.mozilla.org/en-US/d | |||
[[tool.poetry.source]] | ocs/Web/HTTP/Headers/Cache-Control) | |||
name = "foo" | header based cache for almost all HTTP requests. | |||
url = "https://foo.bar/simple/" | ||||
secondary = true | ||||
``` | ||||
{{% /note %}} | ||||
If your private repository requires HTTP Basic Auth be sure to add the username | Further, every HTTP backed package source caches metadata associated with a pack | |||
and | age once it is fetched or generated. | |||
password to your `http-basic` configuration using the example above (be sure to | Additionally, downloaded files (package distributions) are also cached. | |||
use the | ||||
same name that is in the `tool.poetry.source` section). If your repository requi | ||||
res either | ||||
a custom certificate authority or client certificates, similarly refer to the ex | ||||
ample above to configure the | ||||
`certificates` section. Poetry will use these values to authenticate to your pri | ||||
vate repository when downloading or | ||||
looking for packages. | ||||
### Disabling the PyPI repository | ## Debugging Issues | |||
If you encounter issues with package sources, one of the simplest steps you migh | ||||
t take to debug an issue is rerunning | ||||
your command with the `--no-cache` flag. | ||||
If you want your packages to be exclusively looked up from a private | ```bash | |||
repository, you can set it as the default one by using the `default` keyword | poetry --no-cache add pycowsay | |||
```toml | ||||
[[tool.poetry.source]] | ||||
name = "foo" | ||||
url = "https://foo.bar/simple/" | ||||
default = true | ||||
``` | ``` | |||
A default source will also be the fallback source if you add other sources. | If this solves your issue, you can consider clearing your cache using the [`cach | |||
e`]({{< relref "cli#cache-clear" >}}) | ||||
command. | ||||
Alternatively, you could also consider enabling very verbose logging `-vvv` alon | ||||
g with the `--no-cache` to see network | ||||
requests being made in the logs. | ||||
End of changes. 26 change blocks. | ||||
59 lines changed or deleted | 403 lines changed or added |