This subset of the manual documents our approach to the operations and maintenance of the hosted service, sr.ht. You may find this useful for running your own hosted sr.ht service, or to evaluate our practices & policies to consider if they meet your requirements for availability or robustness. You also might just find this stuff interesting, as SourceHut is one of the few largeish services which is not hosted in The Cloud™.

Additional resources:

• New sysadmin lecture
• Backups & redundancy
• Emergency planning
• High availability
• Monitoring & alarms
• Outage incident response
• Provisioning & allocation
• PostgreSQL robustness planning
• SourceHut scalability plans
• Security incident reports
• Outage incident reports
• TLS documentation

Next available port number: 5016/5116

#Publically available operational resources

We try to make as much of our operations available to the public as possible.

#Status page

status.sr.ht is hosted on third-party infrastructure and is used to communicate about upcoming planned outages, and to provide updates during incident resolution. Planned outages are also posted to sr.ht-announce in advance.

The status page is updated by a human being, who is probably busy fixing the problem.

#Monitoring & alarms

Our Prometheus instance at metrics.srht.network is available to the public for querying our monitoring systems and viewing the state of various alarms. Some alarms are also fed to the IRC channel and mailing list.

#Mailing list

The sr.ht-ops mailing list is used for automated reports from our services, including alarm notifications of "important" or "urgent" severity, and automated reports on operational status of backups and other systems.

#IRC channel

The #sr.ht.ops IRC channel on irc.libera.chat is used for triage and coordination during outages, and has a real-time feed of alarms raised by our monitoring system.