This is a guide for enclone developers, both inside and outside 10x Genomics.

There is no reason to read this, unless you want to experimentally modify enclone, or are curious. We distribute pre-built binaries for enclone, and that's all you need if you just want to run the existing code.

The enclone components

enclone has the following software and data components:

• A GitHub repo https://github.com/10XGenomics/enclone that is maintained by 10x Genomics. Contributions are welcome. The software is licensed freely, however only for use with 10x Genomics data. Please read the license.
• A web site https://10xgenomics.github.io/enclone that is built automatically by code in the above repo.
• A separate GitHub repo https://10xgenomics.github.io/enclone-data that contains some datasets. If you build enclone, this is automatically downloaded as part of the build process. If you install enclone, this is downloaded into a separate location (~/enclone), unless you choose the small option, in which case you get only a subset of it. The reason for having a separate repo for data is to keep the enclone repo itself small.
• A larger collection of datasets that are downloaded if you install with the large option; the colossus option results in downloading of additional GEX data.

Software architecture

enclone is written in Rust.

We used Rust because the language makes it extremely easy to use Rust code written by other people, because Rust has its own build system, and because it is fast and safe.

Prerequisites

enclone can be built and run on an x86-64 Linux server or a Mac. It can also be built under Windows, but we do not provide instructions for this.

The following software are needed to build and test enclone:

• Rust. Detailed instructions on how to install Rust can be found here. You can confirm that you have successfully installed the Rust compiler by running rustc --version.
• cargo-license. Probably this is needed only for internal use. You can install it using cargo install cargo-license, assuming that you have already installed Rust.
• cargo-sweep. Install via cargo install cargo-sweep.
• standard utilities, including git, curl and wget
• samtools.

Building enclone

1. Follow the installation instructions on the main page to install enclone, using the large option. Doing this downloads data, which you will want for experimentation and testing. If you are developing the code, then the downloaded enclone binary is superfluous, but you do need to follow the download process to get test data, unless you are at 10x Genomics. In that case the data will be accessible to you in a shared location.
2. If you are using a Mac, you may need to install the xcode tools. The standard way to do this is to type xcode-select --install, however this can be very slow. An alternative method is to get the package from the Apple developer site at https://developer.apple.com/download/all/?q=xcode. Downloading and installing from there still takes some time, but at a minimum, it is more transparent in what it's doing.
3. There are a couple of minor prerequisites. If you're using homebrew, you can install these as follows:
   brew install graphviz
brew install samtools
4. Go to a directory where you want to put the enclone code, and clone the enclone repository using

   git clone git@github.com:10XGenomics/enclone.git

   If you have a very slow internet connection, you might wish to download only the current version (and not the history) using

   git clone --depth=1 git@github.com:10XGenomics/enclone.git

5. Type cd enclone and then ./build. Depending on your computer, this will take roughly 5-10 minutes. Subsequent invocations (after code changes) will be much faster. You can also run cargo b, which is faster, but omits some steps. Please let us know if you encounter difficulties with compilation.
6. Add the full path of enclone/target/debug to your PATH, and make sure this comes before ~/bin, which was where the precompiled version of enclone was downloaded by the install command. You may need to close and reopen your terminal window to get the change to PATH to take effect.
7. Now when you type enclone, you'll get the version you just compiled.

Testing enclone

enclone has an extensive test suite, with three components as described here:

To varying degrees, the tests can either be run from the command line or through a CI which is triggered by pull requests. The CI runs via GitHub Actions and Jenkins. Because it is so slow we tend not to use it. The GitHub Actions part of the CI test is given by .github/workflows/test.yaml. We strongly encourage that when tests are added to the CI, they are also added to test 1, as otherwise we force people to use the CI to test code changes.

Notes on test 1.

• An individual test can be run e.g. by cargo t test_enclone_d.
• The last part just checks that the total wallclock for the entire test is within bounds. This is obviously dependent on the server that is used and system load. It is currently calibrated for use on one large Linux server at 10x.
• The test will fail sporadically because it tests for broken links on the site, and some of those links break transiently, sometimes for about a day. To run the tests without testing links, one can use ./test linkless.
• There are a few subtests that only run under Linux.
• See the following files:
  • enclone_core/src/testlist.rs
  • enclone_main/tests/enclone_test*.rs
  • test
  • .cargo/config.

Notes on test 2. See the file enclone/enclone.test.

Notes on test 3.

• This takes over the Mac while running. It uses regression test images that are somewhat specific to a particular desktop, so the test may only work verbatim for one person. However other people can run the test with their own personal regression test images.
• There is an annoying little bit. When run at 10x Genomics, the code connects to a Linux server and uses the code there as part of the test process. To get versions fully synced, you need to touch enclone/enclone_core/defs.rs on the Linux server and then cargo b. Otherwise you’ll get an error message.

There are miscellaneous gnarly issues that we do not have automated testing for:

• The code for NOPAGER accidentally broke at one point and we don't want that to recur. Some test cases that could be verified:
  • enclone BCR=...
  • enclone BCR=... NOPAGER
  • enclone help all
  • enclone help all NOPAGER
  • enclone help faq
  • enclone help faq NOPAGER.
• In OS X Catalina, in full screen mode, at one point enclone appeared to have no output, because the output was going to the alternate screen.
• We allow paths that start with ~ or ~user_name, but do not have automated testing for this.
• We had buggy behavior that when enclone was called from a terminal window, and it failed, it returned zero exit status, even though when called e.g. from a script, it would return a nonzero value.

What could go wrong?

If the code crashes, you should get a clean traceback. If this happens on the distributed version of enclone or the master version, that's a bug that should be reported.

If you run out of disk space while building, you may get weird error messages from the linker, in which case you should fix your disk space problem. You might also need to cargo clean.

Somewhat rarely, you'll get weird error messages from the linker, with no apparent reason. If this happens, cargo clean should fix the problem.

Editing the enclone web site

This is done by editing the files in the subdirectory pages, and never by editing .html files directly. All the .html files are auto-generated.

There is a script ./local_view that copies the site to your public_html directory so that it can be previewed.

Releasing enclone

enclone binaries for OSX and Linux can be released by “pushing a button”. This only works at 10x Genomics.

First, if the release warrants announcement, edit pages/history.html.src to note the relevant changes. Running build will then update the actual file that goes on the site.

Run the enclone test suite.

From master on enclone, from a Linux server, type

start_release

Cross building for Windows on a Mac

In principle the following should work:

rustup target add x86_64-pc-windows-gnu
cargo b --target=x86_64-pc-windows-gnu