Candid Compliance

icpp-pro decodes every message of the Candid Interface Specification (spec 0.1.8, dfinity/candid@572c80a), within its limits: all 472 asserts of the official suite pass, from icpp-pro 7.1.0rc1. Full compliance, recursive types included in encoding, comes in 7.1.0rc2.

Every message your canister receives is decoded by one decoder, which follows the spec: the typed API (CandidType* with ic_api.from_wire) and the dynamic CandidValue both use it. So what is tested here holds for your own methods.

The tests

The compliance canister runs the official test suite of the specification, and the Candid tests of the Rust and Motoko CDKs, ported. Every assert passes, natively and deployed to a local network, in CI on every push. There is no skip list.

Source Asserts
Official suite: prim.test.did 168
Official suite: construct.test.did 169
Official suite: reference.test.did 50
Official suite: subtypes.test.did 58
Official suite: spacebomb.test.did 17
Official suite: overshoot.test.did 10
Official suite, dfinity/candid@572c80a 472
Rust CDK, rust/candid and rust/candid_parser tests, same commit 508
Motoko, test/run-deser and test/run-drun/idl-*.mo, at 01aee06 181
icpp-candid's own encode vectors 14
Total 1175

An assert is one of:

  • A message that must decode at the given types, or must be rejected.
  • Two values that must decode to the same value, or to different ones. A side is a message or a textual Candid value.
  • A value that the typed API must encode to exactly the reference's types (from didc), and that must decode back to the same value: 109 of the asserts.

Motoko's test/idl/*.did tests check the parsing of .did files, which icpp-candid does not do, and are not ported.

Decoding follows the spec

From 7.1.0rc1, the decoder coerces each value to the type you expect, as the spec defines it:

  • Subtyping. A nat decodes as an int; a value of type T decodes as an opt T; anything decodes as reserved; a service decodes as a principal. A record may have more fields on the wire than you expect, and a variant may have fewer cases.
  • Opt. A value that does not coerce to the type in an opt decodes as null, as do null and reserved on the wire.
  • Arguments. Arguments on the wire that you do not expect are read and skipped. An argument you expect that is not on the wire must be an opt, null or reserved, and is null.
  • References. A func or service reference decodes when its type is a subtype of the one you expect, also for recursive types.

The decoder rejects, with a trap and a clear message, what earlier versions accepted:

  • Bytes after the last argument.
  • A bool other than 0 or 1.
  • Text that is not valid UTF-8, also in a value that is skipped.
  • Record or variant field ids that are not sorted, are repeated, or are larger than 32 bits.
  • A principal longer than 29 bytes.
  • Argument types that are not valid, also when you expect no arguments.

Encoding

You encode with the typed API, for every type that does not refer to itself. A recursive type, such as type List = opt record { head : int; tail : List }, can be decoded into a CandidValue, but cannot be encoded yet: that comes in 7.1.0rc2.

Limits

The spec leaves limits to the implementation. A message over a limit traps.

Limit Value
Nesting depth of values, also of skipped ones 128
Values that take no bytes on the wire, per message 1,000,000
Entries in the type table 10,000

A value that takes no bytes on the wire is a null, a reserved, or a record whose fields are all such values. A few bytes can declare a great many of them, so they are counted, as the spacebomb tests require.

The decoder holds such values once: a vec null holds one element and its length, and the records of a nested record of such fields are held once per type. A message of a few bytes does not make the decoder hold much more memory than its size.

Two things to know when you decode untrusted messages:

  • The typed API gives every element of a vec to your code. A vec null of a million elements, decoded as a vec opt nat into a std::vector, makes a million std::nullopt elements in your vector, as it does in Rust's Vec<Option<T>>.
  • CandidValue::to_text() writes a nat or int in decimal, which takes time quadratic in its length. Decoding a large number is linear; printing one from an untrusted message is not.

Hardening

  • The decoder reads the message through two functions only, both bounds-checked. make check-decoder-reads, part of the static checks in CI, fails when any other code reads it.
  • The decoder is fuzzed with libFuzzer, under AddressSanitizer and UndefinedBehaviorSanitizer. The seeds are every message of the compliance tests. Each input is also checked for the memory it makes the decoder hold, against the size of the message.
  • CI fuzzes on every push: every seed and every saved input, then 60 seconds per fuzz target. Before each release, each target is fuzzed for an hour, and the result is in the release notes.