Almost a full year ago, I made the first commit into the Stackage repository, and the announcement blog post describes the project’s motivations. Since then, as

part of our work at FP Complete, there’s been a lot of activity on

this project. The community contributions have been amazing as

well, with a huge number of package authors signing up to

contribute and maintain their packages on Stackage. I want to recap

where the project is right now, in terms of the package set, its

own codebase, and how FP Complete is working with Stackage.

The package set

To date, users have contributed roughly 200 separate packages to

the Stackage package collection. When you include all of the deep

dependencies of these packages, we are currently building about 480

total packages in a full Stackage build, or roughly 9% of all

Hackage packages.

This is great news for anyone using Stackage directly to build a

package database, as they gain direct access to a huge number of

the most popular Haskell packages, without worries of

compatibility. (We’ll touch on that a bit more in the FP Complete

section.) But the work we’re doing with Stackage has probably

already directly benefitted all Haskell users, even those who have

never touched Stackage.

As part of regular Stackage maintenance, I maintain a total of

three separate daily Stackage builds, one using the most recent

Haskell Platform, one using the previous release, and one using

vanilla GHC 7.4 (no Haskell Platform constraints). Every day, these

jobs run, try to compile all of these packages, and run the full

set of test suites. The process has found a large number of simple

compilation issues, overly restrictive cabal version bounds, and a

number of actual bugs. In my opinion, Stackage is living up to its

goal of helping maintainers raise the quality of their code.

At this point, I’ve certainly lost track of how many patches

either myself or one of my coworkers at FP Complete have sent based

on Stackage build results. In the cases where we haven’t been able

to put a patch together, maintainers have been incredibly

responsive, and I’d really like to thank everyone in the community

who has participated.

So on this note, there’s only one thing I’d like to ask from

people: get involved! If you’ve written some code that you’re

actively maintaining, get it in Stackage. You’ll be widening the

potential audience of users for your code by getting your package

into FP Haskell Center, and you’ll get some helpful feedback from

the automated builds so that users can more reliably build your

code.

The codebase

The Stackage codebase itself has become pretty stable. The

initial goal was simply to try building a bunch of packages at the

same time. At this point, it handles a lot more than that:

• Performs an audit of uploaders to Hackage, to ensure no one

  accidentally (or purposefully) hijacks another user’s package. I’ve

  been running these audits approximately twice a week for the past

  few months, and as usual have gotten great responses from the

  community. (Once Hackage 2 goes live, this process will be

  obsoleted, but for now it’s providing a valuable service.)

• The ability to add local patches to packages from Hackage. This

  has been very useful in quickly modifying cabal version

  constraints.

• Ability to snapshot builds, which will capture all packages and

  their versions to be built. When combined with the previous point,

  this allows us to create stable package sets which can include bug

  fixes.

• Fully sandboxed builds, optional Haskell Platform constraints,

  and helpful output to determine the cause of conflicts in version

  numbers.

• And for my own sanity: the ability to take advantage of

  cabal-install’s parallel build option to build all 480 packages in

  a (semi-)reasonable amount of time. It usually takes about 1.5

  hours to do a full build on my desktop.

But we’re not finished yet! Security is a big concern for both

FP Complete and Galois, and we’ve been talking about ways to

leverage Stackage to improve the security situation of Hackage.

Some concrete steps we’re hoping to take in the near future

are:

• Package signing, to ensure authorship.

• Facilitate code reviews.

• Set up a web of trust for code.

Since this blog post is about the state of Stackage, and not its

future, I don’t want to belabor these points too much. But expect

to hear more about them in the future. In the meanwhile, if you’re

interested in taking part in this process, please be in touch with

me, or even better, write to the Stackage mailing list.

FP Complete

FP Complete is the company behind Stackage. So what are we

getting out of this? Firstly, everything I’ve listed above. FP

Complete is obviously very invested in the Haskell community, and

benefits greatly from all of the wonderful work being done by that

community. So anything we can do to help facilitate that continued

progress is a boon to our progress as well. Getting a higher

quality set of packages on Hackage and lowering the barrier to

entry by reducing installation problems is something we’re very

interested in.

More directly, many people know that we use Stackage to build

the set of packages we provide on FP Haskell Center itself. When

you write a School of Haskell tutorial, you have access to all 480

packages in our package database. When you start a new project in

FP Haskell Center, you can immediately import any module in that

package database, without concerns of diamond dependencies.

By running Stackage as such an open project, and having our

system built on top of Stackage, we’ve created a natural way for

users to get additional libraries onto our system. Instead of

having a centralized body controlling which packages can make it

onto our servers, we can open things up to a much more user-driven

approach. This takes us out of the role of gate-keepers (or, said

another way, bottlenecks) and instead lets us simply be

enablers.

The Hackage uploader vetting has been invaluable as well. We

take security very seriously, as do our users, and we need some way

to ensure that when you start using package X, it hasn’t been

replaced by a nefarious user. Of course, package signing will give

us even more guarantees about this.

FP Complete is promoting two sets of packages currently:

bleeding edge (called unstable), and the stable 13.09 package set.

This latter set is based on a snapshot taken just before our first

release at the beginning of this month. Using Stackage’s

snapshotting and patching abilities, we are able to provide users

with stable APIs from vetted packages, while fixing any bugs or

security holes found in those packages on behalf of our users. And

for those of you looking to be more experimental, our unstable

environment can be updated much more regularly.

Stackage is fulfilling the role I’d hoped it would, both for the

community in general, and at FP Complete. We’re going to be

broadening its intended feature set over the next few months to

include more security features, and hope to provide yet another

benefit to the Haskell community. This is an exciting project to be

a part of, and I hope even more of you will take part in it.