This is the manual for the tulip Lua web framework.

#What is Tulip?

Tulip is a web framework based on lua-http and PostgreSQL. While this is an accurate definition, it is incomplete as Tulip is a bit different than typical web frameworks - it is more a framework for building web systems, and doing so with a simple, minimal, yet thorough architecture and infrastructure.

The Small is Beautiful (The Developer's Edition) blog post provides some background and context about the vision and goals of this project.

It is a framework for web systems because it goes beyond building a Web server - it also enables:

• Message queues for asynchronous processing
• Cron-style scheduling of processing
• Publish-subscribe notifications support
• Transactional database migrations
• Complete account management based on best practices

A Tulip-based system requires an infrastructure of only three components:

• Lua
• Postgresql
• Linux / a POSIX operating system

Incidentally, those components are the inspiration for the name.

The architecture is explained in more details in its own section, but in short, it is based on a declarative-like configuration and an extendable packages system.

For example, this is a fully-functional web server that does pretty much what you'd expect:

local handler = require 'tulip.handler'
local App = require 'tulip.App'

local app = App{
  log = { level = 'debug' },
  server = { host = '127.0.0.1', port = 0 },
  middleware = {
    'log',
    handler.write{ body = 'Hello, Tulip!' },
  },
}
assert(app:run())

#Getting Started

Unless you want to completely configure the development environment yourself, it is highly recommended to use the tulip-cli tool to setup a standard Tulip environment for you. The command can be installed as usual using LuaRocks, and it is recommended to install it in your local tree. You can also install it in the system-wide tree if you prefer, but this will probably require root privilege (e.g. using sudo).

luarocks install --local tulip-cli

You should also make sure you have all the tools required to initialize a standard Tulip environment:

• Docker and docker-compose, for the postgresql database (https://docs.docker.com/get-docker/ and https://docs.docker.com/compose/install/)
• The mkcert command, to create localhost certificates for https (https://github.com/FiloSottile/mkcert#installation)
  • While not strictly required, this is recommended to minimize differences between your local development environment and the production environment, e.g. to set secure cookies and other https-related headers.
• The direnv command, to automatically configure environment variables when you enter your project's directory (https://direnv.net/docs/installation.html)
• It is also recommended to install the postgresql client package, to have access to the psql command

Also, before running the tulip-cli init command, you should make sure you have all the prerequisites required to install Tulip, otherwise it will fail to build some of its dependencies.

Then, assuming the binary directory of your LuaRocks tree is in your $PATH, you can initialize a tulip project by running:

tulip-cli init DIR

Where DIR is a directory path relative to the current working directory where the command will create and initialize the tulip environment. It must be either a non-existing or an empty directory.

What the command does is:

• Create a project-local luarocks configuration, so that modules are installed locally in the lua_modules directory
• Create the project-local directory for the postgresql database, as a docker image that includes the pg_cron extension
• Create the secrets for local development, such as the CSRF key, the account key and the root database user's password
• Create the localhost certificate for https support
• Create the .envrc direnv file with environment variables to connect to the database and to configure Lua search paths to use the local modules
• Install tulip

After running the command, you should manually:

• Approve the .envrc file by running direnv allow .
• Bring the database up by running docker-compose up

Once the postgresql database has completed its initial setup and the direnv file has been approved, you should be able to connect to the database with psql by running the command without any argument (the environment variables and the run/secrets/pgpass file take care of the connection options).

Note that if, for some reason, you want to remove your project's repository, you will have to use sudo to remove the db/postgres/data directory, as this is mounted as the data volume in the Docker image and its content is owned by the user in the Docker image.

#Installation

You must make sure you have the following prerequisites installed on your system prior to installing tulip:

• The Lua development files, required for all dependencies built from C (Fedora: lua-devel)
• The Postgresql development files, required for the luapgsql dependency (Fedora: libpq-devel)
• The Open SSL development files, required for the luaossl dependency (Fedora: openssl-devel)
• The argon2 development files, required for the argon2 dependency (Fedora: libargon2-devel)
• The zlib development files, required for the lua-zlib dependency (Fedora: zlib-devel)

The preferred method is to use the tulip-cli command documented in Getting Started, but you can also install tulip manually via LuaRocks:

luarocks install tulip

Not all dependencies currently support Lua 5.4 in their official LuaRocks name, so I adjusted them in my own mna/ namespace. Namely, those dependencies required some changes currently published under my username namespace:

• luapgsql: does not currently install with Lua 5.4 (request and PR to update was made).
• lua-cjson: fails with lua_objlen symbol error, as version 2.1.0.6 is the latest version published to LuaRocks (request to update was made); works with the HEAD of the repository (using a locally-modified rockspec file that removes the "tag" constraint).
• luaossl: requires special configuration on Fedora: luarocks install luaossl 'CFLAGS=-DHAVE_EVP_KDF_CTX=1 -fPIC'.

Everything should work fine when running luarocks install tulip (at least on Fedora systems), as it will pull those from my namespace.

#Architecture

At the core of tulip's architecture is the App's configuration, which is a simple Lua table. As such, your application can be as declarative (using the Lua table literal syntax) or as imperative (using Lua code to generate the table) as you want or require.

You configure an application by passing in the configuration to tulip.App, which is a function that creates an App instance, and then calling App:run on the instance. That's it.

local App = require 'tulip.App'

-- App is a function that takes a single table as argument (the configuration),
-- so it can take advantage of Lua's syntactic sugar for such functions and
-- drop the parentheses, making App creation look even more declarative:
local app = App{
  log = {level = 'i'},
  -- other configuration goes here...
}
assert(app:run())

The configuration is directly related to the pluggable, extendable packages support, allowing incredible modularity and flexibility of features. How it works is that each top-level key in the configuration table maps to a Lua module to "require", and then that module only has to implement a minimal interface to integrate nicely into Tulip.

Since the built-in packages are so frequently required, when a top-level key has no dots "." in it, a first try is done to require it as tulip.pkg.<name>, so this configuration is equivalent to the previous example's:

local app = App{
  ['tulip.pkg.log'] = {level = 'i'},
}

Third-party packages can just as easily be registered, and in fact it is recommended to implement all your system by means of registered packages, as it typically leads to better isolation, better defined contracts, and better tested code.

#The Packages Contract

At a minimum, a package (the returned value from the require of the corresponding Lua module) must implement a register function that receives the package-specific configuration and the App instance. This is the relevant part of the tulip.pkg.locals package:

local M = {}

-- The locals package adds a key-value dictionary on the app
-- under the 'locals' field. This can be used to set app-wide
-- information such as name, title, contact email address, etc.
function M.register(cfg, app)
  app.locals = cfg
end

return M

The register function of each referenced package is called during the tulip.App function call with the configuration table.

Additionally, the package can define an activate function. If it exists, it is called for each package during the App:run method call, before calling the application's main function, with the package-specific configuration and tha App instance. This call is used to make last-minute preparation, knowing that every package is now registered.

This is where, for example, the middleware names (strings) are resolved to their actual handler function, so that this configuration:

local app = App{
  log = {level = 'i'},
  middleware = {
    'log',
    function (req, res, nxt)
      -- handler implementation omitted...
    end,
  },
}

actually calls the middleware function registed by the log package where the 'log' string is found. This cannot be done in register, because there is no guarantee that the log package will be registered before the middleware package (registration - as well as activation - order is undefined and packages must not rely on it).

In addition to register and activate, the following package fields are relevant, although all are optional:

• requires: an array of strings that define the package's dependencies. The full name of the reuqired packages must be specified, i.e. tulip.pkg.database, not just database. This is verified before calling the package's register function, and the application's creation fails if a dependency is missing.
• replaces: a string identifying the full name of a package that the current package replaces - meaning that it is a drop-in replacement for that package. For example, if package 'X' declares itself a replacement for 'Y', then any package that depends on 'Y' will happily work with 'X'. Of course, the drop-in replacement must stay true to its promise.
• app: a table where each key-value pair is assigned to the app instance. This is done before calling any package's register function, and is typically used to create app methods like register_middleware so that other packages can use that method in their register function to indirectly interact with the package (i.e. middleware are not registered on the "middleware" package, but via the app's method, so there is no direct communication between packages - they are fully decoupled).

To summarize, the app and packages "startup" lifecycle goes like this:

1. Configure the package in the App's configuration
2. Call tulip.App's function with the application's configuration
   1. The package is required and recorded in the list of registered packages
   2. If replaces field is set, record its value in the list of registered packages
   3. If app field is set, assign each of its key-value pairs on the App instance
   4. If requires is set, validate that each dependency is met
   5. Call the register function with the package's configuration and the App instance
3. Call App:run
   1. If the package has an activate function, call it with the package's configuration and the App instance
   2. Call App:main, which must be set by one of the registered packages
   3. On return from main, call any registered finalizer with the app instance as argument

#Reference

#App and handler

• tulip.App
• tulip.handler

#Built-in Packages

• tulip.pkg.account
• tulip.pkg.cron
• tulip.pkg.csrf
• tulip.pkg.database
• tulip.pkg.flash
• tulip.pkg.gzip
• tulip.pkg.json
• tulip.pkg.locals
• tulip.pkg.log
• tulip.pkg.metrics
• tulip.pkg.middleware
• tulip.pkg.mqueue
• tulip.pkg.pubsub
• tulip.pkg.reqid
• tulip.pkg.routes
• tulip.pkg.sendgrid
• tulip.pkg.server
• tulip.pkg.template
• tulip.pkg.token
• tulip.pkg.urlenc
• tulip.pkg.validator
• tulip.pkg.wmiddleware
• tulip.pkg.worker
• tulip.pkg.wroutes

#Other Modules

• tulip.Semaphore
• tulip.crypto
• tulip.xerror
• tulip.xio
• tulip.xstring
• tulip.xtable