Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs #100

Merged
merged 4 commits into from
Sep 23, 2023
Merged

Docs #100

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# whl2conda changes

## [23.9.0] - 2023-9-23

* First official stable release

## [23.8.10] - 2023-9-16 (*prerelease*)

* Fix generation of entry points
Expand Down
13 changes: 9 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ help:
"build - build wheel and conda package in dist/\n" \
"check-upload - check uploadable wheels in dist/\n" \
"upload - upload the latest wheel in dist (requires pypi access)/\n" \
"update-stdrename - update stdrename.json file in source\n" \
"\n" \
"--- clean ---\n" \
"clean - remove generated files\n" \
Expand Down Expand Up @@ -124,14 +125,15 @@ open-coverage: htmlcov/index.html

MKDOCS_FILE := mkdocs.yml
CLI_SUBCMDS := $(shell $(CONDA_RUN) whl2conda --list-subcommands)
CLI_DOCS := doc/cli/whl2conda.md $(foreach subcmd,$(CLI_SUBCMDS),doc/cli/whl2conda-$(subcmd).md)
CLI_DIR := doc/reference/cli
CLI_DOCS := $(CLI_DIR)/whl2conda.md $(foreach subcmd,$(CLI_SUBCMDS),$(CLI_DIR)/whl2conda-$(subcmd).md)

# Build main cli man page
doc/cli/whl2conda.md: src/whl2conda/cli/main.py
$(CLI_DIR)/whl2conda.md: src/whl2conda/cli/main.py
$(CONDA_RUN) whl2conda --markdown-help > $@

# Build subcommand cli man page
doc/cli/whl2conda-%.md: src/whl2conda/cli/%.py
$(CLI_DIR)/whl2conda-%.md: src/whl2conda/cli/%.py
$(CONDA_RUN) whl2conda $* --markdown-help > $@

site/index.html: $(CLI_DOCS) $(MKDOCS_FILE) doc/*.md src/whl2conda/api/*.py
Expand Down Expand Up @@ -176,6 +178,9 @@ mike-serve: doc-serve-all
# Distribution targets
#

update-stdrename:
$(CONDA_RUN) whl2conda config --update-std-renames src/whl2conda/api/stdrename.json

# TODO - add targets from version
build-sdist:
$(CONDA_RUN) python -m build --sdist --no-isolation --outdir dist
Expand Down Expand Up @@ -213,7 +218,7 @@ clean-coverage:
$(RMDIR) htmlcov .coverage coverage.json coverage.xml

clean-doc:
$(RMDIR) site doc/whl2conda-cli.md
$(RMDIR) site $(CLI_DIR)/whl2conda*.md

doc-clean: clean-doc

Expand Down
File renamed without changes.
File renamed without changes.
4 changes: 4 additions & 0 deletions doc/about/nav.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
nav:
- Getting help: support.md
- Release notes: changelog.md
- License: license.md
File renamed without changes.
12 changes: 7 additions & 5 deletions doc/basic-usage.md → doc/guide/basic-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,13 @@ This is described in the section on [Dependency Modification](renaming.md)

## Downloading wheels

If you want to create a conda package for a pure pytyhon package on pypi that doesn't
If you want to create a conda package for a pure python package from [pypi] that doesn't
currently have one available on a public channel, you can download the wheel
using [pip download][pip-download]. You do not need to download depenences and do not want
an sdist, e.g.
using [pip download][pip-download]. You do not need to download dependencies and
want a wheel, not an sdist, so use:

```bash
$ pip download --only-binary:all: --no-deps some-package
$ pip download --only-binary:all: --no-deps <some-package>
```

Then you can convert the downloaded wheel using `whl2conda convert`.
Expand All @@ -48,7 +48,7 @@ $ whl2conda convert my-project/
```

When run this way, there must be a `pyproject.toml` or `setup.py` file
in the directory. If there isa `pyrpoject.toml` file, *whl2conda* will
in the specified directory. If there isa `pyproject.toml` file, *whl2conda* will
read any project-specific options from the `[tool.whl2conda]` section
(see [Configuring pyproject.toml](pyproject.md) for details) and will
then look for a wheel in the `dist/` subdirectory of the project.
Expand Down Expand Up @@ -84,3 +84,5 @@ debugging purposes.

[pip-download]: https://pip.pypa.io/en/stable/cli/pip_download/
[pip-wheel]: https://pip.pypa.io/en/stable/cli/pip_wheel/
[pypi]: https://pypi.org

18 changes: 8 additions & 10 deletions doc/limitations.md → doc/guide/limitations.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
*whl2conda* currently only supports conversion of generic pure python wheels
*whl2conda* supports the conversion of generic pure python wheels
into noarch python conda packages.

It has the following limitations, some of which will be addressed in future
releases.
It has the following limitations and known issues, some of which
will be addressed in future releases.

## Version specifiers are not translated

Expand All @@ -15,9 +15,8 @@ properly in conda. Specifically,
* the *compatible release* operator `~=` is not supported by conda.
To translate, use a double expression with `>=` and `*`, e.g.:
`~= 1.2.3` would become `>=1.2.3,1.2.*` in conda. This form is
also supported by pip, so this is a viable workaround for packages
you control.

also supported by pip, so switching to this format is a viable
workaround for packages under your control.

* the *arbitrary equality* clause `===` is not supported by conda.
I do not believe there is an equivalent to this in conda, but
Expand All @@ -35,7 +34,7 @@ back with compatible specifier syntax, e.g.:
whl2conda mywheel-1.2.3-py3-none-any.whl -D foo -A 'foo >=1.2.3,1.2.*'
```

This will be fixed in a future release
This will be fixed in a future release
(see [issue 84](https://github.com/zuzukin/whl2conda/issues/84)).

## Wheel data directories not supported
Expand All @@ -48,9 +47,8 @@ This will be addressed in a future release

## Cannot convert from sdist

Currently, only conversion from wheels is supported. Conversion from python sdist
distributions are not currently supported. This could possibly be supported in
the future (see [issue 78](https://github.com/zuzukin/whl2conda/issues/78)).
Conversion from python sdist distributions is not currently supported.
This could possibly be supported in the future (see [issue 78](https://github.com/zuzukin/whl2conda/issues/78)).

## Cannot convert from eggs

Expand Down
6 changes: 6 additions & 0 deletions doc/guide/nav.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
nav:
- Basic Usage: basic-usage.md
- Dependency Modification: renaming.md
- Configuring pyproject.toml: pyproject.md
- Installing and Testing: testing.md
- Limitations: limitations.md
17 changes: 11 additions & 6 deletions doc/pyproject.md → doc/guide/pyproject.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,18 @@ $ whl2conda config --generate-pyproject my-project-dir/
This is safe to use on an existing pyproject file and will not
overwrite any existing options.

In general, *whl2conda* command line options will override options
specified in the `pyproject.toml` file.

The available options are described below.

## pyproject.toml options

### conda-name

Overrides name of conda package generated by whl2conda.
If empty, default is taken from project.name.
If empty, default is taken from `project.name`
(or `tool.poetry.name` if build backend is [poetry][poetry-pyproject]).

This is overriden by the `--name` convert option.

Expand All @@ -43,11 +47,11 @@ Overridden by the `--out-dir` convert option.
The default output conda package format.
Valid values are:

* `"V1"` / `".tar.bz2"`: original .tar.bz2 package forma
* `"V2"` / `".conda"`: new V2 .conda package format',
* `"TREE"`: output package as directory tree (debug)',
* `"V1"` / `".tar.bz2"`: original .tar.bz2 package format
* `"V2"` / `".conda"`: new V2 .conda package format
* `"TREE"`: output package as directory tree (for debugging)

Overriddenn by the `--format` convert option.
Overridden by the `--format` convert option.

### dependency-rename

Expand All @@ -60,7 +64,7 @@ Each entry should be a two element list of the form:

Where `<pypi-name-pattern>` may either just be the pypi package name
or is a Python regular expression that will match the full name,
and where <conda-name> can contain subsitution expression,
and where <conda-name> can contain [subsitution expression](renaming.md#manual-rename),
For example, to rename all `acme-<foo>` packages to `acme.<foo>`:

```
Expand All @@ -71,3 +75,4 @@ For example, to rename all `acme-<foo>` packages to `acme.<foo>`:

An optional list of extra conda dependencies.

[poetry-pyproject]: https://python-poetry.org/docs/pyproject/
43 changes: 25 additions & 18 deletions doc/renaming.md → doc/guide/renaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@ using [conda]. For instance, you can install `pylint` using:
$ conda install -c conda-forge pylint
```

But pypi and conda are actually totally separate packaging systems and there
is no inherent requirement that package names match, and indeed there are some
exceptions, for example:
However, pip and conda use totally separate packaging systems and there
is no inherent requirement that package names match between the two, and
indeed there are quite a few exceptions, for example:

=== "pip"

Expand All @@ -42,10 +42,10 @@ these differences be handled by the tool.
## Standard renames

The *whl2conda* tool maintains a table of automatic renaming rules that
is taken from mappings collected aumatically from tools supporting the
is taken from mappings collected automatically from tools supporting the
public [conda-forge] repository. *whl2conda* includes a static copy of
this table collated when *whl2conda* was built, but also supports the
ability to maintain and update a locally cached copy dynmically. If
this table collated when the *whl2conda* package was built, but also
supports the ability to maintain and update a locally cached copy dynamically. If
there is package new to [conda-forge] that may have appeared since installing
*whl2conda*, you can update your local cache using:

Expand Down Expand Up @@ -109,7 +109,7 @@ drop any package that matches a pattern:
$ whl2conda -D 'acme-.*'
```

### Renaming dependencies
### <a name="manual-rename">Renaming dependencies</a>

To rename dependencies, use `-R` / `--dependency-rename` with two
arguments, the pypi name followed by the conda name.
Expand Down Expand Up @@ -152,17 +152,24 @@ If you are using a `pyproject.toml` file for your project, you can
instead specify how dependencies are modified in the tool options.
This is described in the [next section](pyproject.md).

## Hiding pip dependencies in dist-info directory for conda

We have sometimes seen problems in packages built using conda-build
in which the pip dependencies listed in the python packages
dist-info directory in the site-packages directory in the conda
environment where the conda package is installed can clash with
conda dependencies. To avoid this kind of problem, the `whl2conda`
by default will turn all regular dependencies in the dist-info
into extra dependencies using the name `original`. So if you
look at the `METADATA` file in the dist-info of the generated
conda package, you will see entries like:
## <a name="hide-pip">Hiding pip dependencies in dist-info directory for conda</a>

The standard method of building python conda package involves
running `pip install` or the equivalent, which results in a
the package's original pip dependencies and other metadata
being saved to the `METADATA` file in the package's `.dist-info`
directory in `site-packages/` for the environment in which it
is installed. However, in a conda environment, these dependencies
may seem to be incompatible with what is actually installed by conda.
This can sometimes cause serious problems if pip or other standard
python packaging tools attempt to check or update these dependencies.

Since the actual dependencies for packages installed by conda
are described by the conda package's metadata, and not the metadata
saved in the `.dist-info`, *whl2conda* by default will turn all
regular dependencies in the dist-info into extra dependencies using
the name `original`. So if you look at the `METADATA` file in the
dist-info of the generated conda package, you will see entries like:

```email
Requires-Dist: some-package >=1.2; extra == 'original'
Expand Down
File renamed without changes.
26 changes: 13 additions & 13 deletions doc/index.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
**Generate conda packages directly from pure python wheels**

*whl2conda* is a command line utility to build and test conda packages
generated directly from pure python wheels.
*whl2conda* is a command line utility for building and testing
conda packages generated directly from pure python wheels.

## Features

* **Performance**: because it does not need to create conda environments
for building, this is much faster than solutions involving *conda-build*.

* **Multiple package formats**: can generate both V1 ('.tar.bz2') and V2 ('.conda')
conda package formats. Can also generate a unpacked directory tree for debugging
or additional user customization.
in order to build, this is *much* faster than solutions involving *conda-build*.

* **Dependency renaming**: renames pypi package dependencies to their
corresponding conda name. Automatically renames packages from known
list collected from conda-forge and supports user-specified rename
patterns as well.

* **Project configuration**: project-specific options can be saved in
* **Multiple package formats**: can generate both V1 ('.tar.bz2') and V2 ('.conda')
conda package formats. Can also generate a unpacked directory tree for debugging
or additional user customization.

* **Project configuration**: *whl2conda* project-specific options can be read from
project's `pyproject.toml` file.

* **Install support**: supports installing conda package into a conda
* **Test install support**: supports installing conda package into a conda
environment for testing prior to deployment.

* **Hides pypi dependencies**: if the original pypi dependencies are included in
the python dist-info included in the conda package, this can result in
problems if pip or other python packaging tools are used in the conda environment.
To avoid this, *whl2conda* changes these dependencies to extras.
* **Hides pypi dependencies**: rewrites the original pip/pypi dependencies in the
installed dist-info to avoid [compatibility issues](guide/renaming.md#hide-pip).



6 changes: 6 additions & 0 deletions doc/nav.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
nav:
- User Guide: guide
- Installation: install.md
- Reference: reference
- About: about

File renamed without changes.
1 change: 1 addition & 0 deletions doc/reference/api/nav.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
order_by: title
File renamed without changes.
3 changes: 2 additions & 1 deletion doc/cli/.gitignore → doc/reference/cli/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
# This directory only contains generated files
*
*.md
!index.md
1 change: 1 addition & 0 deletions doc/reference/cli/nav.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
order_by: title
File renamed without changes.
4 changes: 4 additions & 0 deletions doc/reference/nav.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
nav:
- Command line: cli
- API (<i>not stable!</i>): api
- Reference Material: links.md
8 changes: 7 additions & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ dependencies:
- wheel >=0.41
# build
- build >=0.7.0
- pip >=23.2.1
- hatchling >=1.18,<2.0
- make >=4.3
# testing
Expand All @@ -27,5 +26,12 @@ dependencies:
- mkdocs-material >9.1
- mkdocstrings-python-xref >=1.5.2,<2.0
- linkchecker >=10.2.1
# for mkdocs-awesome-pages-plugin
- natsort >=8.4
- wcmatch >=8.5
- pip >=23.2.1
- pip:
- mkdocs-awesome-pages-plugin >=2.9.2



Loading
Loading