~cnx/ipwhl

Floating cheeses are collections of carefully handpicked cheeses from the cheese shop and other sources. Analogies aside, the interplanetary wheels (IPWHL) are platform-unique, single-versioned Python binary distributions backed by IPFS. It aims to be a downstream wheel supplier in a similar fashion to GNU/Linux distributions, whilst take advantage of a content-addressing peer-to-peer network to provide a reproducible, easy-to-mirror source of packages.

#What?

On IPWHL, for each platform (architecture, operating system and Python implementation and version), there exists only one single binary distribution. The collection of these distribution packages are given as a single IPFS CID. An installer can use solely this content ID and packages names to reproduce the exactly same environment on every platform.

The official IPWHL repository will provide exclusively free software. However, deriving the repository should be trivial and is a supported use case.

#Why?

The cheese shop is great, but choosing cheeses from it can often be confusing. Dependency resolution is expensive, and version requirements are not future-proof. In order to avoid breakage, people usually have to pin packages on the installer side, which is redundant and difficult to validate manually. Additionally, we believe it is not the packaging users' job to do this; they should be able to save their time doing what they do best: writing and using software.

Moreover, there are millions of ways for a piece of cheese to rot on the way home from the (almost) lawless cheese shop. Everyone can sell at the shop, and thus typosquatting is a common exploit. In addition, cheeses from the shop are not independently verifiable: the checksums are provided along with the files so the shop is the single point of failure for security attacks. There are ongoing efforts to integrate TUF into Python packaging toolchain, however it is unlikely that they can entirely mitigate this due to the centralized nature of the inherent architecture.

Centralization also makes it really difficult for mirrors to be useful for the users: the cheese shop is not aware if any of its mirrors, let alone redirecting to the closest one. Mirroring is hardly a collaborative effort, one either provide everything for an entire region, or give up. On the other hand, many organizations host their Python packages and their dependencies on dedicated machines running 24/7, but the resources are mostly gone to waste when unused by the companies themselves.

IPWHL makes use of IPFS and statically declared and carefully curated metadata to try to solve most the listed problems. In addition to providing only one wheel version at a time, source distributions are not supplied to avoiding executing untrusted code on the users' machine.

#How?

In order to add a wheel file to IPWHL, it must be statically declared and checked, then the declaration collection shall be published.

#Declare

A IPWHL declaration is a flat TOML file consisting of the following fields:

  • source: Upstream URL to the wheel
  • content-id: CIDv0 of the wheel
  • requires-python (optional): Specifier set representing Python versions the wheel is compatible with, read from Requires-Python in metadata 1.2
  • extras: Array of optional features, read from Provides-Extra in metadata 2.1
  • dependencies: Array of requirements from Requires-Dist in metadata 1.2, converted to PEP 508 format

The file shall be placed in the following path:

$prefix/$project/${wheelname%.whl}.toml

where project is the (pre-normalized) project name, prefix is the smallest 8 bits in hexadecimal of the sum of characters in project and wheelname is the file name of the wheel as specified in PEP 427.

#Check
#File Organization
#Declarations
#Dependencies
#Publish
#Build
#Garbage Collection
#Announcement

#Where?

About this wiki

commit b2301357846be3369c798e797a302e9e8fdce0f5
Author: Nguyễn Gia Phong <mcsinyx@disroot.org>
Date:   2021-04-12T11:12:38+07:00

Fix typo
Clone this wiki
https://git.sr.ht/~cnx/ipwhl-docs (read-only)
git@git.sr.ht:~cnx/ipwhl-docs (read/write)