Functional Programming

Functional Programming

Functional Programming

Dec 1, 2014

Experimental package releases via Stackage Server

Experimental package releases via Stackage Server

Experimental package releases via Stackage Server

Right now, Hackage has no concept of a stable and an unstable

release of a package. As a result, authors are hesitant to release

code to Hackage unless it's already stable. But it's difficult to

get people to test new versions of packages if it's difficult to

install. Installing a single new package from Github may not be

difficult, but sometimes you want people to test out a new set of

versions for multiple packages, which can be tedious. This blog

post will demonstrate how you can use Stackage Server to make that

easy.


While the primary purpose of Stackage Server is to host the

official Stackage snapshots, it has been designed as completely

generic server for hosting any set of packages desired, including

custom packages not yet released to Hackage. All you need to do

is:


  1. Create an account on Stackage Server (by logging in with Google+ or Mozilla Persona)

  2. Create a tarball in the correct format (described below)

  3. Upload it from the snapshot upload page

Tarball format

You can download a sample bundle file by clicking on the "Bundle" link at the

top of any snapshot page. It might be useful to open one up as you looking

through the rest of this section.


You can view the tarball parsing code in the Stackage Server codebase

itself. The format is designed to be simple to replicate and

extensible for future functionality. (In fact, the slug file

feature I mention below was only recently added.)


The tarball must be tarred in a format that the tar package

can read, and then gzipped. Each file in the tarball is treated

indepedently. Directory structure inside the tarball is ignored.

Using tar cfz mybundle.tar.gz somedirectory is usually sufficient to meet these criterion.


Each file inside the tarball is treated separately. There are four kinds of files recognized:

  • desc gives the human-readable title and

    description for the snapshot. Put the title on the first line, and

    the description on the following lines. (Note that, currently, we

    only display the title on the site, though we may add the description to the display in the future.)

  • slug is a recommendation for the short name of the

    snapshot. For example, the most recent

    GHC 7.8 snapshot as I write this is

    https://www.stackage.org/snapshot/2014-11-26-ghc78-exc, which has a slug of 2014-11-26-ghc78-exc. Slugs must

    be globally unique, so if someone else has already taken that slug,

    Stackage Server will append a randomized token to the end.

  • hackage is a list of all the package/version combos

    to be included in this snapshot from Hackage. For example, you

    might have:


    foo-1.0.0
    bar-1.0.1

    You're free to have multiple versions per package.

  • Any file ending in .tar.gz will be treated as a custom sdist tarball, and will be made available for

    download from stackage.org. This is how you can provide custom

    versions of a package not released on Hackage. As an example of

    this, here's a

    snapshot with two unreleased packages in it (link removed).

Custom snapshot

Another use case is customizing an official Stackage snapshot.

For example, you may be using a certain snapshot, but want to get a

newer version of one of the packages from Hackage, or write a

custom patch for one of the package versions and use that. If so,

all you need to do is:


  1. Download the bundle file

  2. Tweak its contents

  3. Upload it

  4. Use the new URL

Replace or augment Hackage?

The instructions for using a Stackage snapshot mention replacing

the hackage.haskell.org remote-repo line in your cabal config file

with the stackage.org URL. This makes sense if you're providing a

snapshot that has all the packages from Hackage that you'll need.

However, if you're testing out a few new packages, it's simpler to

just provide those few extra packages, and add an extra

remote-repo line to your config file instead of replacing the

primary entry. Note that this trick can be used to augment a

Stackage snapshot in addition to adding extra packages to

Hackage.


Caveats

You should keep two things in mind when using Stackage Server in this manner:

  • Snapshots you create live forever. In cases of extreme issues

    (like accidentally uploading copyrighted data) we will of course

    assist in removing the snapshot. But generally speaking, a snapshot

    is forever, just like uploading a package to Hackage makes it

    available forever.

  • All snapshots are publicly listed, so you

    don't want to put any sensitive information in there. Of course,

    the Stackage Server codebase is open source, so you're free to run

    your own, private instance if you'd like. Alternatively, FP

    Complete provides private Stackage Server instances as a service,

    feel free to contact us for more

    information.

Other uses

Creating a generic tool like that has the advantage that it can

be (ab)used to purposes other than the original intent of the

author. In this case, I've described some intended alternate use

cases for this functionality. If people come up with other

unintended use cases, let me know!