FP Complete now does blockchain audit services. Why have we

chosen to work in this field, and what are we aiming to

accomplish?

Our corporate mission is to drive the successful adoption of better IT engineering tools and practices.

Experience shows us again and again: quality and productivity are driven more by these substantive improvements than

by simply deciding to try harder. Any engineer, and any team, can

be more successful using the right tools and best practices. This

was true when I built and ran Microsoft’s Productivity Tools Team

(for Windows and Office engineering), and when I was in charge of

Visual C++ and parts of Visual Studio. And it remains true today as

we see with powerful tools like Stack for Haskell, or Kubernetes,

or a wide range of corporate projects FP Complete has worked

on.

Blockchain Now Needs Stronger Engineering

Good engineering involves a lot of pieces beyond just having a

strong algorithm paper: coding standards, continuous integration,

automated test coverage, documentation management, reproducible

cloud deployment, dependency tracking, and more. The stronger the

engineering infrastructure, the more likely you can expect a

reliable and secure result that works as intended under a wide

range of conditions—in other words, quality.

The blockchain field, including cryptocurrency, is of course

fairly new. And these technologies are of course very sensitive to

quality. Unfortunately as we have all seen, they don’t all live up

to their promises. Engineering teams, perhaps feeling the pressure

to get to market quickly, sometimes overlook valuable opportunities

to improve quality. To be blunt, a lot of blockchain implementation

work needs improvement.

We believe over the next few years the bar for engineering

excellence is going way up. People are staking their money, their

privacy, their businesses on the correct operation of these

systems. So we’ve been asking directly: what can be done to

increase the quality of engineering in the whole blockchain

industry?

More even than other open technologies, blockchain relies upon

community trust. We need to give blockchain groups a way to

earn that trust by actually doing proper work, with

an independent inspection that it’s being done right. A

cryptocurrency cannot be a hack job—and if it is done right, users want to know. Thus the audit, an inspection

to verify that the project lives up to good engineering standards.

By making these standards clear, we give teams something specific

to shoot for and give credit to those who've got it right. For

users and investors, knowledge is power.

Months ago Cardano announced their decision to appoint us as the auditors of their

cryptocurrency engineering. This cryptocurrency has a market

capitalization over US$ 4 Billion, and they want users to know that

the system can be trusted. We’ve already provided them with interim

results which are being published, and the work is ongoing.

At the same time, we’re working on several other non-published

cryptocurrency projects, and in talks with more. So we decided it

was time to formalize the audit program and announce it

publicly.

Levels of Auditing

We hope to encourage a great many blockchain and cryptocurrency

projects to seek an outside engineering audit, whether from FP

Complete or another qualified firm. We look forward to the day when

users expect to see an audit on any sensitive

cryptocurrency or blockchain work. And that means we need to

provide people with a path to get started.

Therefore we’ve chosen to offer several audit plans, using

different amounts of labor (and thus, costs) to achieve different

amounts of scrutiny and certification. For ease of understanding by

general audiences we are calling these Bronze, Silver and Gold; and

we will use “stars” to further summarize how well the project is

doing. We will be publishing the criteria for each level; obviously

the more auditing work is done, the more parts of the engineering

can be checked and potentially certified. What's crucial right now

is to get every project on the path to verifiable quality.

Auditing is not the same as a 100% inspection. Given that all

blockchain projects are moving targets, our goal is to achieve a

reasonable level of scrutiny with sampling, and report

accurately on whether each audited project appears to be living up

to a reasonable standard of engineering practices. As part of any

public certification we will report on the nature of what we’ve

inspected, what standards it met, and exceptions we’ve found.

At a basic level of scrutiny, we will focus on the tools,

development processes, and quality control processes in use: are

good engineering systems used, in line with best practices for

predictable results? At a higher level of scrutiny we will delve

much deeper into a larger percentage of the source code, tests, and

so on, greatly increasing the density of checks that can be done.

Are the software and the distributed system being built in a way

that is most likely to operate as specified? Or is the team

operating on just caffeine and hope?

Clearly, signing up for an audit is no guarantee of a passing

grade: a project may fail an audit and earn no certification at

all. In such cases, our intention is to provide the team with as

much constructive feedback as possible on how they can improve. We

hope in such cases the chance to work up to a certification will

serve as a “carrot,” an incentive to implement improvements that

would lead to a passing grade or better.

As you probably know, FP Complete offers extensive services in

FinTech software engineering, cloud engineering, and DevOps. To

avoid any conflict of interest, of course we will not issue an

audit grade for a project where we ran the engineering. In any such

case we will bring in an outside firm to compare the engineering

work with the published criteria and determine the grade.

Raising the Standards

Right now we see a wide range of engineering quality levels on

blockchain projects. Frankly, I don’t expect to see many Gold or

even Silver certifications in the short term. However, we hope to

see some. Moreover, as industry standards rise (as they must), we

expect to add further criteria, increasing the bar for each level

of certification. Even a Bronze certification in 2020 may involve

far more requirements than one in 2019 or 2018. This will be

spelled out in the published criteria for each level at any given

time.

FP Complete does not have the capacity to audit the over 1600

cryptocurrencies already in existence, plus all of the other

blockchain projects and wallets. We certainly hope to make a dent,

but realistically other companies will need to enter this space as

well. We will welcome them to use criteria modeled on our own, or

to create their own lists of what constitutes proper engineering.

What’s important is that they not lower the bar, but raise the bar,

for quality in this industry. The blockchain engineering audit

field needs to grow rapidly for the public good, and we will

promote its growth in a constructive and timely manner.

Note that a technology audit will never be the same thing as a

financial audit. Technical excellence doesn’t mean that a

particular cryptocurrency is a good investment, or that a

particular blockchain is suited for some particular use. But it

should mean that the implementation team is following best

practices to bring their implementation in line with what’s been

described and specified.

We hope the day will come when consumers of any blockchain will

ask: where’s the audit? It’s long been expected in the stock

market, and crypto users deserve no less. Home and business users

alike deserve to know if they can trust the technology on which

they are staking so much. By demanding evidence of excellence, we

give providers the backing they need to invest more in quality,

safety, and security.

For further reading

How to select the right level of QA for your project

Best practices: multifaceted testing

Best practices: DevOps priorities for FinTech

DevOps to prepare for a blockchain world (video presentation)

From my colleague Steve Bogdan: getting past IT operations into DevOps

From my colleague Niklas Hambüchen: the Haskell language and cryptocurrencies

Details on our blockchain auditing services