Floating cheeses are collections of carefully handpicked cheeses from the cheese shop and other sources.
Floating cheeses are collections of carefully handpicked cheeses from the cheese shop and other sources.
See also:
Analogies aside, the interplanetary wheels (IPWHL) are platform-unique, single-versioned Python binary distributions backed by IPFS. It aims to be a downstream wheel supplier in a similar fashion to GNU/Linux distributions, whilst take advantage of a content-addressing peer-to-peer network to provide a reproducible, easy-to-mirror source of packages.
On IPWHL, for each platform (architecture, operating system and Python implementation and version), there exists only one single binary distribution. The collection of these distribution packages are given as a single IPFS CID. An installer can use solely this content ID and packages names to reproduce the exactly same environment on every platform.
The official IPWHL repository will provide exclusively free software. However, deriving the repository should be trivial and is a supported use case.
The cheese shop is great, but choosing cheeses from it can often be confusing. Dependency resolution is expensive, and version requirements are not future-proof. In order to avoid breakage, people usually have to pin packages on the installer side, which is redundant and difficult to validate manually. Additionally, we believe it is not the packaging users' job to do this; they should be able to save their time doing what they do best: writing and using software.
Moreover, there are millions of ways for a piece of cheese to rot on the way home from the (almost) lawless cheese shop. Everyone can sell at the shop, and thus typosquatting is a common exploit. In addition, cheeses from the shop are not independently verifiable: the checksums are provided along with the files so the shop is the single point of failure for security attacks. There are ongoing efforts to integrate TUF into Python packaging toolchain, however it is unlikely that they can entirely mitigate this due to the centralized nature of the inherent architecture.
Centralization also makes it really difficult for mirrors to be useful for the users: the cheese shop is not aware if any of its mirrors, let alone redirecting to the closest one. Mirroring is hardly a collaborative effort, one either provide everything for an entire region, or give up. On the other hand, many organizations host their Python packages and their dependencies on dedicated machines running 24/7, but the resources are mostly gone to waste when unused by the companies themselves.
IPWHL makes use of IPFS and statically declared and carefully curated metadata to try to solve most the listed problems. In addition to providing only one wheel version at a time, source distributions are not supplied to avoiding executing untrusted code on the users' machine.
In order to add a wheel file to IPWHL, it must be statically declared and checked, then the declaration collection shall be published.
A IPWHL declaration is a flat TOML file consisting of the following fields:
source: Upstream URL to the wheelcontent-id: CIDv0
of the wheelrequires-python (optional): Specifier set representing
Python versions the wheel is compatible with, read from Requires-Python
in metadata 1.2extras: Array of optional features, read from Provides-Extra
in metadata 2.1dependencies: Array of requirements from Requires-Dist in metadata 1.2,
converted to PEP 508 formatThe file shall be placed in the following path:
$prefix/$project/${wheelname%.whl}.toml
where project is the (pre-normalized) project name, prefix is the smallest
8 bits in lowercase hexadecimal of the sum of characters in project
and wheelname is the file name of the wheel as specified
in PEP 427.
The following criteria are used to check the paths to declarations:
$prefix/$project as described above..toml and were it .whl
the hypothetical name must make up a valid wheel filename
according to PEP 427.The integrity of a declaration is verified by comparing the original wheel with the declared fields. Before performing this, one must ensure that the source URL has not been compromised. Due to the lack of warranty for the long-term availability of the source and to save resources, an implementation may only do this for recently modified declarations, e.g. ones added or updated by a patch.
Every wheels declared must have their dependencies, including extras, satisfied on every supported platform.
If all checks pass, a new repository release can be published from the declarations.
For a fresh build, every wheel declared is downloaded and placed according to PEP 503:
$project/$wheelname
The parent directory is then recursively added to IPFS. To leverage subsequent builds, IPFS mutable file system (MFS) can be utilized for caching: only new declarations and those not matching the content ID are downloaded and added to the previous repository version. This way, we need not rely on the upstream URLs' availability; however wheels from declarations having been updated or removed must be garbage collected.
Should MFS be used, the following files and directories must be removed to maintain the property of being platform-unique and single-versioned of IPWHL:
New releases must be tagged along with their respective CIDs and cryptographically signed on the data Git repository. Afterwards, the maintainers should announce it on the announce mailing list.
The mainline Git trees are hosted on SourceHut:
In ascending order of traffic:
Bugs and requests are tracked on SourceHut TODO, which also supports submitting via email to ~cnx/ipwhl@todo.sr.ht.
The maintainers of the floating cheeses can be found on Matrix at #ipwhl:matrix.org. However, there is no concept of an official chat room, so feel free to create ones on other platforms. We will consider bridging them to the Matrix room upon requests.
commit 53c1d332911a24592b0689da5d9bd3902c82e4df Author: Nguyễn Gia Phong <mcsinyx@disroot.org> Date: 2021-05-09T18:12:07+07:00 Detail declaration instruction