sr.ht is a distributed system. Each service, such as git.sr.ht and builds.sr.ht, runs independently, with its own database and resources. They communicate with each other using mainly their respective APIs and webhooks, and are mostly tolerant to the temporary or permanent absence of their peers. The system is fairly complex, and is designed more for deployment at scale than for small-scale installations. However, you can choose a subset of these services to install as your needs demand, and it will work correctly.

Developers wishing to hack on or contribute to sr.ht, see hacking.md.

This page documents details common to the installation of all packages, but many services have specific needs. Specific details to each service are available on each service's installation page:

All sr.ht sysadmins are encouraged to subscribe to the sr.ht-admins mailing list, a low-volume list of sysadmin-oriented announcements regarding breaking changes, security vulnerabilities, and so on.

#Packages

The only supported installation is Alpine Linux hosts using the SourceHut Alpine Linux package repository. There are also community-maintained SourceHut distributions for Debian and Arch Linux, which are preferred before installing SourceHut from source.

SourceHut is still in alpha, and has no stable releases. If you are interested in packaging it for your distribution, please reach out to us for help in automating the maintenance of a third-party distribution which is automatically updated following our (frequent) upstream rolling releases. Please refrain from packaging SourceHut for your upstream distribution repositories until we have shipped stable versions of our software.

Packages also ship with the correct users and groups rigged up, and with whichever daemon configurations are particular to your distribution's init system (e.g. OpenRC scripts or systemd units).

#Prerequisites

Generally speaking, most services require the following services:

  • A PostgreSQL server: persistent storage
  • A Redis server: ephemeral storage, caching, work distribution
  • A mail server: incoming and outgoing mail
  • A cron daemon: running scheduled tasks

Additionally, many services are able to integrate with some optional tools:

#Installing SourceHut services

These instructions apply generally to all services. Consult service-specific documentation for amendments to these procedures.

#sr.ht core

core.sr.ht is a Python package which provides common functionality for all services. For users installing from packages, it is not necessary to concern yourself with this, as it is automatically pulled in as a dependency. For users building from source, you must build this package first. Be aware that the dependencies in setup.py are not generally kept up-to-date, consult the latest Alpine package for an up-to-date and comprehensive list.

#meta.sr.ht

meta.sr.ht is the only service which is required. It is responsible for storing your account details and profile, handling logins, storing SSH and PGP keys, keeping a secure audit log, and providing some management interfaces for administrators.

Consult the meta.sr.ht installation guide for details on the installation procedure.

#Service configuration

All services use a shared configuration file at /etc/sr.ht/config.ini. Each site provides an example configuration in config.example.ini in their respective source code repositories. It is the administrator's responsibility to consult these examples to produce a unified configuration file which is applicable to all of the services which they intend to operate.

Some configuration options are applicable to all services.

#Service keys

Encryption is used throughout SourceHut to encrypt and validate communications. There are three main keys which you need to generate: service keys, the network key, and the webhook key.

Service keys are used to encrypt session cookies, and if you configure load-balancing, it must be consistent between all nodes of that service. They can be generated with srht-keygen service.

The network key needs to be consistent throughout all services and nodes in your sr.ht installation. It is used to encrypt and sign internal communications between services. This key is generated with srht-keygen network.

The webhook key is also consistent throughout all services and nodes, but is an asymmetric key. It is used to sign webhook payloads to validate the authenticity of the request. You can generate this with srht-keygen webhook, store the private key in your config file and distribute to the public key to any parties interested in authenticating webhook payloads from your services. This is also used to validate webhooks used internally; it is not optional.

#Mail configuration

Outgoing emails from sr.ht are configured in the [mail] section. If you fill out the error-to address, the services will send exceptions to this address for diagnostics. You must also generate a PGP key (without a password) for sr.ht services to sign outgoing emails with (and optionally encrypt emails to each recipient).

#Database Configuration

Each service requires its own database, though they can co-exist on the same server. It is also recommended to give each service its own database login, with full access rights to that database so that it may manage its own schema migrations.

After you populate your config.ini's connection string, you may use the [module]-initdb script to populate the schema and stamp the latest revision for migrations, for example metasrht-initdb to set up meta.sr.ht's database.

#Schema Upgrades

We use alembic to manage schema migrations. We use custom scripts to automatically retrieve database credentials from your main sr.ht config file. If you have installed from distribution packages, set [service] migrate-on-upgrade=yes (where service is e.g. [meta.sr.ht]) to have migrations automatically performed during normal service upgrades.

Otherwise, you may use srht-migrate <service> upgrade head to run updates for core.sr.ht migrations, and <service>-migrate upgrade head to run service-specific upgrades. For example, to upgrade the database schema for git.sr.ht, run srht-migrate git.sr.ht upgrade head, then gitsrht-migrate upgrade head. Other alembic commands are available, use gitsrht-migrate --help for more information.

#Upgrade procedure
  1. Stop all services
  2. Run your distro's update commands (e.g. apk update && apk upgrade)
  3. Resume all services

We source your alembic config from your main sr.ht config.ini - no need to write an alembic.ini file. Run srht-migrate <service> stamp head && <service>-migrate stamp head once to tell alembic the schema is up-to-date (e.g. srht-migrate man.sr.ht stamp head && mansrht-migrate stamp head). Future upgrades will be managed automatically by the package, or if you're using the source code you can run srht-migrate <service> upgrade head && <service>-migrate upgrade head when you pull the latest version down.

#Start the daemons

Daemon configuration is provided in the distribution packages. Use the service manager appropriate to your distro to start the daemons, e.g. service git.sr.ht start or systemctl start git.sr.ht.

#Proxy configuration

The exact configuration you use will vary depending on your needs. Here is an example nginx configuration for meta.sr.ht:

server {
    listen 80;
    server_name meta.sr.ht;

    location / {
        return 302 https://$server_name$request_uri;
    }

    location ^~ /.well-known {
        root /var/www;
    }
}

server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name meta.sr.ht;
    client_max_body_size 100M;
    ssl_certificate /etc/ssl/uacme/meta.sr.ht/cert.pem;
    ssl_certificate_key /etc/ssl/uacme/private/meta.sr.ht/key.pem;

    location / {
        proxy_pass http://127.0.0.1:5002;
    }

    location /static {
        root /usr/lib/python3.6/site-packages/metasrht;
    }
}

Once the proxy is configured, you should be able to access your new service.

#OAuth configuration

For services other than meta.sr.ht, you have to create and configure an OAuth client before users can log into your service. To do this, visit your profile on your meta.sr.ht instance, select the OAuth tab and register a new client. Append the path /oauth/callback to the URL of your service instance and choose this value as the base redirect URI (for example https://git.sr.ht/oauth/callback). Update your service configuration file with the generated client ID and secret.

You then need to configure that client as "preauthorized", i.e. first-party. This skips the OAuth consent screen that third-party applications are subject to and enables some extra API features. Log into the meta.sr.ht database and run the following query (adjusting the client ID as appropriate):

update oauthclient set preauthorized = true where client_id = 'your client ID';

Now you should be able to log into your new service.

Alternatively, you can generally run most SourceHut services by pointing them at our meta.sr.ht instance and registering them as an OAuth client. For example, you could run a custom builds.sr.ht instance which logs in with hosted meta.sr.ht accounts by registering for an OAuth client there and skipping the preauthorized step. YMMV.

About this wiki

commit a587053d94365eca00c57a74185dd4b2fe0c99c1
Author: Jarkko Oranen <oranenj@iki.fi>
Date:   2020-10-21T20:26:23+03:00

OpenBSD 6.8 released, update compatibility docs accordingly
Clone this wiki
https://git.sr.ht/~sircmpwn/sr.ht-docs (read-only)
git@git.sr.ht:~sircmpwn/sr.ht-docs (read/write)