History

This page provides a selective history of what was changed in enclone and when.

We show changes that affect users like new features, changes to results, and the like. This log starts with with initial public availability. The complete history may be seen by cloning the enclone repo and typing git log.

Breaking changes are shown in red.

Please be aware that our workflow when we make changes is to automatically update the GitHub site, including all the website pages and this page too. This happens in advance of actually making a release (which might follow in a couple days). This means that the website may describe features that are not yet available in a release (although they will be in the source code that's available). We apologize for this asynchrony! Note however that the command-line help that comes with your copy of enclone will always match its behavior.

5/9/22: Add ability to create enclone_visual sessions from the command line. Please see enclone_visual.

5/8/22: enclone now runs under Windows, with some limitations, see here.

3/31/22:

• Add lvars jun_mat and jun_sub.
• Add option NOGRAY to turn off gray color in per-cell clonotype lines.

3/26/22: Add lvar jun_ins, please see enclone help lvars.

3/15/22: Add DATASET filter, please see enclone help filter.

3/14/22:

• Add PNO_HEADER option to suppress CSV header line in parseable output, please see enclone help parseable.
• Add grouping options GROUP_NAIVE and GROUP_NO_NAIVE. Please see enclone help display.
• Add lvar hcomp, please see enclone help lvars.

3/11/22:

• A major change to the clonotype joining algorithm, which joins based on sharing in the heavy chain junction region, and which substantially increases clonotyping sensitivity. Please see enclone help how. This change also makes enclone about 30% slower.
• Delete the following genes from the human VDJ reference: IGHV4-30-2, IGKV1D-33, IGKV1D-37, IGKV1D-39, IGKV2D-28. These are identical to other genes in the reference, except that the reference provides a longer 5'-UTR in one case. We observe that clonotypes having one of these genes often have heterogeneous assignments between the gene and its duplicate, and that's bad, as it implies that assignment of genes to clonotypes in these cases is effectively random.
• Add lvars allele and allele_d. Please see enclone help lvars.

3/4/22: Add ability to select functional groups by donor, see enclone help display.

3/3/22:

• Import a large set of data from iReceptor.
• Add grouping option v_heavy_refname.
• Add PREPOST=x to append /x to PRE entries.
• Allow multiple META arguments and allow each META to be a comma-separated list of filenames.

3/2/22:

• Add grouping options cdr3_heavy_len and cdr3_light_len.
• Clonotypes within groups are now reverse sorted by cell count.
• Add GROUP_CDR3 to only show groups containing the given CDR3 amino acid sequence.

• Add ability to plot by a categorical variable. Please see enclone plots.

2/28/22: Add an argument BC_JOINT=filename, which functions like BC, but uses only a single file. Please see enclone help input.

2/25/22:

• For the medium and large enclone installation options, some published customer datasets are now included, from LIBRA-seq and for Kawasaki disease, Wang et al. 2021 Nature Communications, "Single-cell RNA sequencing of peripheral blood mononuclear cells from acute Kawasaki disease patients".
• Change the default for JOIN_CDR3_IDENT from 80 to 85.

2/18/22: Introducing enclone_visual! This is alpha software for a graphic user interface (GUI) for enclone, that is implemented initially for Macs. Please see enclone_visual.

2/16/22: Add new grouping options, please see enclone help display.

2/14/22:

• Two more changes to the clonotyping join algorithm, see here.
• Add SPLIT_PLOT_BY_DATASET, see enclone plots.

2/7/22: Remove some genes from the reference that were identical except for extension on the 3' end.

2/4/22:

• Improve the correctness of V gene assignment. The changes comprise several technical changes to the alignment code and the addition of one gene IGHV3-9 to the VDJ reference. About 2% of cells are affected.
• Add lvars vname_orig and dref_max.

1/26/22: Add ability to define variables as expressions involving other variables, using an argument VAR_DEF=.... Please see variables.

1/13/22: Make three small changes to BCR clonotyping:

• Change the default value for MAX_SCORE to 100000.
• Fix bug that resulted in the constraint cd ≤ 15 still being imposed.
• Allow joins with high score to pass so long as they have at least 15 shared mutations. The parameter 15 is accessible as AUTO_SHARE.

1/5/22:

• Several changes that improve clonotyping sensitivity and specificity. Details are here and also reflected in enclone help how.
• Add an option JOIN_BASIC=... to use a much simpler clonotyping join criterion. See enclone help how.

12/27/21:

• Fix bug in computation of the cvars vj_seq_nl and vj_aa_nl. These were not correctly handling the case where there is an indel in the leader sequence.
• Fix bug in computation of cvars fwr3_dna and fwr3_aa. These were not correctly handling the case where there was a deletion in the sequence before the CDR3 start.

12/24/21: Add filter option NMAX to allow barcodes having more than four contigs. Also, this is now turned on by NALL and NALL_CELL. This does not affect default behavior.

12/13/21: This version includes three changes that substantially improve clonotyping specificity. See below and enclone help how.

• Change clonotyping algorithm by rejecting joins where the number of differences in CDR3 is at least CDR3_MULT times the number of non-shared differences outside CDR3. The default value of CDR3_MULT is 5. This algorithmic change may be overridden by using a large value instead, e.g. by CDR3_MULT=100.
• Change clonotyping algorithm by computing the number N in a different and simpler fashion, as 80^cd. The old behavior may be obtained by adding the argument OLD_MULT.
• Change clonotyping algorithm by preventing merger in cases where light chain constant regions are different. The old behavior maybe be obtained by adding the argument OLD_LIGHT.
• Add lvar nbc which is the numerically encoded barcode. See enclone help lvars.

11/9/21:

• Add option NOSPACES to not show spaces between features specified by the AMINO option. See enclone help amino.
• Add option COLOR=codon-diffs, that causes certain "unchanged" amino acids to be grayed out. This can make it much easer to see changes. See enclone help color.

10/26/21:

• Add NSEG and NSEGN, to allow exclusion of clonotypes having particular VDJC genes.
• Add MIN_ORIGINS.
• Add option MIN_DONORS. For example MIN_DONORS=2 only shows clonotypes containing cells from at least two donors. If you use MIN_DONORS=2 (or more), MIX_DONORS will be automatically turned on, to allow generation of clonotypes having cells from multiple donors.

10/18/21: Allow coloring of honeycomb plots by dataset. Please see enclone plots.

10/12/21:

• Remove clonotypes that are not placed in a group. This fixes a bug.
• Add symmetric grouping statistics to the summary.
• Fix a bug that caused feature counts to be incorrectly reported if the value for a particular cell for a particular feature exceeded 65535. This only happened if the NH5 option was used.

10/4/21: Add argument MIN_GROUP_DONORS=... to set a lower bound on the number of donors contributing to a group, for it to be printed. Please see enclone help display.

9/21/21:

• Simplify the install script and expand the installation debugging page. This may resolve some problems people have had and also provides help in the event that installation fails.
• Add new lead variable nchains_present. Please see enclone help lvars.

9/20/21: Add a new default filter called signature filtering that under certain rare circumstances will decompose complex clonotypes that are incorrectly glued together. This filter can dramatically affect certain datasets, but has almost no effect on typical data (less than one per million clonotypes tested). Please see enclone default filters.

9/17/21:

• Add KEEP_CLONO_IF_CELL_MIN. See enclone help filter.
• Tweak handling of NOPRINT PARSEABLE=stdouth to make columns line up.

9/15/21: Add control over PNG resolution for cell coloring by variable.

9/14/21: Fix some issues with cell coloring, affecting cell coloring by variable and SPLIT_PLOT_BY_ORIGIN.

9/11/21: Add the ability to color cells in a honeycomb plot using the values of a variable. Please see enclone plots.

8/27/21: Add new argument DVARS=... that can display dataset-level variables, for now very limited, as described at see enclone help display. In addition, values for GVARS now appear inside the summary.

8/24/21:

• SPLIT_PLOT_BY_ORIGIN now allows creation of side-by-side honeycomb plots, representing e.g. different cell types or preparations. See enclone plots.
• enclone installation now includes a test for whether the correct enclone executable is accessible from the command line, and prints debugging information if not.

8/17/21: KEEP_CELL_IF now allows feature variables to appear in constraints, see enclone help special.

7/22/21:

• Add option for xy plots to make them symmetric, which is appropriate e.g. for plotting data from replicates. See enclone plots.
• Add option INFO_RESOLVE that modifies INFO to handle cases where identical immune receptor sequences have been provided. See enclone help input.

7/14/21: Add new arguments HONEY_OUT and HONEY_IN to allow preservation of cell positions between two honeycomb plots of the same data. Please see enclone plots.

6/30/21: Add cvar cigar that is the CIGAR string of the V..J contig sequence versus the universal reference. See enclone help cvars.

6/24/21: Now enclone UPDATE updates to the latest version of enclone.

6/17/21: Add SIM_MAT_PLOT, to display the pairwise cosine similarity between a list of variables, see enclone plots.

6/4/21: Tweak the order of default filters so as to increase the likelihood that clonotypes that are held together tenuously (or not at all) will be pulled apart. This has a tiny effect on our datasets (changing about 1 in 10^5 clonotypes), but could affect some user datasets disproportionally.

6/3/21: Fix bug in the computation of datasets_cell.

6/2/21:

• Allow PCHAINS=max. See enclone help parseable.
• Add MAX_HEAVIES=1 to ignore any cell having more than one IGH or TRB chain.
• Fix bug that caused VDJ gene name and id fields to be displayed in parseable output even when a chain was absent for a given exact subclonotype.

5/6/21: Clonotype grouping has been completely reworked. Please see enclone help display. The pre-existing options are still present but now undocumented, and their behavior might have changed slightly. We have not optimized the new options for run time, so let us know if there's a problem and we'll see what we can do.

5/4/21: Several changes that involve D gene assignment, and which allow for the VDDJ case:

• Add cvars d1_name, d2_name, d1_score, d2_score, d_delta or equivalently d_Δ, see enclone help cvars and D genes and junction regions.
• Add options ALIGN<n> and JALIGN<n> to exhibit visual alignments of V..J sequences or just the junction regions to the concatenated VDJ reference, see enclone help display and D genes and junction regions. There are also ALIGN_2ND<n> and JALIGN_2ND<n> to use instead the second best D gene alignments.
• Add global variables d_inconsistent_% and d_inconsistent_n that provide statistics on overall consistency of D gene assignment, see enclone help display and D genes and junction regions.
• Add filters D_INCONSISTENT to only show clonotypes having an inconsistent D gene assignment, D_NONE to show clonotypes have a null D gene assignment, and D_SECOND to show only VDDJ clonotypes.
• Add grouping options GROUP_VJ_REFNAME_HEAVY and GROUP_VDJ_REFNAME_HEAVY, see enclone help display.

4/8/21:

• Fix bug that caused CONP to produce the wrong answer in some cases.
• Fix bug that caused FWR4 values to be wrong in some cases.

4/5/21: Add page enclone default filters that describes the order of the filters that enclone uses, and provides technical details about some of the filters, which was previously on the heuristics page.

4/2/21:

• Almost any parseable variable can now be used in a linear condition. Of course it doesn't makes sense to use such a variable unless its values are numeric.
• A number of other restrictions on variables were removed, including by promotion of parseable variables to lvars or cvars.
• Clicking on the enclone banner at the top of any page on the site now takes you back to the help section on the main page.
• There is a new page that inventories all the enclone variables.
• We turned on overflow checking in the enclone executables. It is possible that you will now see error messages (tracebacks) where you did not previously see them. Please report these events!

3/23/21:

• Add new option top plot two variables, see bottom of enclone plots.
• Add SOURCE=filename to import some arguments from another file.
• Add option SCAN_EXACT to scan exact subclonotypes rather than clonotypes. See enclone help filter.

3/17/21:

• Gene scanning now has an option SCAN_EXACT to scan using exact subclonotypes rather than clonotypes. See enclone help filter.
• Add SOURCE=filename to read extra arguments from the named file, see enclone help command.

3/16/21:

• enclone can now take values for fields associated to particular immune receptor sequences via INFO=path.csv; see enclone help input.
• We now allow TREE=v1,...vn where the vi are parseable variables, resulting in labeling by these variables. See here. For backward compatibility, const is translated to const1. The new display is slightly different than the old display, showing e.g. const1=IGHG1 rather than IGHG1.
• There is now a detailed developers guide.

3/4/21:

• Add a plotting option QUAD_HIVE that forces honeycomb plots into the first quadrant. See honeycomb plots.
• Tweak honeycomb plotting to cause smaller clonotypes to be generally "farther out" than smaller clonotypes. See honeycomb plots.
• Add new options CONX and CONP that add amino acid consensus rows to clonotype tables. Please see enclone help display, which refers to enclone help cvars.
• For developers, add an option PROFILE.

2/14/21: A VDJ reference file can now be tested for defects using a command of the form enclone REF=<fasta file name>.

2/13/21:

• We redesigned the algorithm that determines the columns in clonotype tables, see notes on heuristics. This addresses some very rare cases where chains were incorrectly placed in the same column. This also results in better disintegration of "merged" clonotypes in some cases. There is an additional change that came about. In rare cases, a cell will have two contigs with identical CDR3 sequences. This could be the "other" allele or more likely an artifact. Previously one of these contigs was suppressed in the clonotype table. Now they both appear.
• For weak chain filtering, we now allow deletion of chains having up to 20 cells, rather than 5. See notes on heuristics. This fixes some pathological examples involving clonotypes having many chains.
• Added option FOLD_HEADERS to reduce horizontal space usage. See also the discussion of line wrapping in frequently asked questions.

2/6/21: Fix the computations of median of a vector. For a vector x of length n, these were computed as x[n/2], which is incorrect in the case where x has an even number of elements: it should be the mean of the innermost two elements. However, in cases where the vector elements are integers, we now round this mean up to the nearest integer, yielding a "rounded median", as the advantage of presenting the exact median is outweighed by the loss of readability resulting from adding .5 or .0 to the display.

2/4/21: Add option SUPPRESS_ISOTYPE_LEGEND.

2/3/21: Add chain variable cdr3_aa_conx that is the consensus of the CDR3 amino acid sequences across the clonotype, showing X for each variant residue, and cdr3_aa_conp, which instead of X shows an amino acid property symbol. See enclone help cvars. We thank Albert Vilella for suggesting these features.

1/19/21: Add lead variables count_cdr1_<regex> and similar, see enclone help lvars.

1/14/21: Add chain variables corresponding to extended or trimmed versions of CDR sequences, and North versions. Please see enclone help cvars.

1/13/21:

• Move onesie merger to a later stage in the process, after certain "junk" deletion steps, to increase the likelihood of merges happening. This significantly increases the number of merges, so that after this change, many more clonotypes contain onesies.
• The BC argument now may be either a CSV or TSV file.
• Automatically find multi/vdj_b and multi/vdj_t subdirectories of outs.
• Allow BCR_GEX=p as a shorthand for BCR=p and GEX=p, which simplifies syntax when a cellranger multi run is used as input; ditto for TCR_GEX.

1/9/21: Following a suggestion of Ganesh Phad, single-cell clonotypes in honeycomb plots are now localized by color.

1/7/21:

• Greatly speed up honeycomb plots for large numbers of clonotypes.
• Change the installation command to use bash rather than sh to avoid failures resulting from sh being instantiated as a different shell.

1/4/21:

• Fix the output of the parseable variable cdr3_start<i> when there was an insertion earlier in the transcript.
• Add an option PLOT_BY_ISOTYPE_COLOR to allow color specification for the PLOT_BY_ISOTYPE argument.

12/14/20: Add option METAX that allows meta information to be provided on the command line, rather than through a separate file.

12/9/20:

• Delete the _ag antigen feature label (which was documented at enclone help lvars) because this was in fact never supported by cellranger.
• Change the algorithm for deciding to use a donor reference allele, so that it checks all donor reference alleles for all V genes having the same name as the one originally assigned to a contig. In some cases this changes results. Note that this change comes after Cell Ranger 5.0 and with high probability will be integrated into the next major Cell Ranger release.
• Change the default value of MAX_DIFFS from 50 to 55. Change MAX_DEGRADATION from 3 to 2. Change the default value for MAX_SCORE from 1,000,000 to 500,000. Change the default value for MAX_CDR3_DIFFS from 10 to 15. Ditto comments about Cell Ranger.
• Allow up to 5 (instead of 4) CDR3 amino acid differences when joining chains.
• Find input data correctly if the cellranger multi pipeline is used.
• Add a new filter that removes some doublets, and which can be turned off by supplying the argument NDOUBLET (see enclone help special and notes on heuristics).

12/5/20:

• The argument F has been renamed to KEEP_CLONO_IF_CELL_MEAN, although for backward compatibility, F still works. We have added the similar KEEP_CLONO_IF_CELL_MAX. See enclone help filter for both.
• Rename FCELL to KEEP_CELL_IF, but retain FCELL for backward compatibility.

12/2/20: CDR3 sequences are now recomputed by enclone, rather than taken from the value in all_contig_annotations.json. This will result in changed values at a very low frequency, and only if the CDR3 calculation algorithm was changed between the time of the relevant cellranger release and the time of the enclone release.

11/24/20:

• Allow arbitrarily long CDR3 sequences. This change is not part of Cell Ranger 5.0, but may be part of a subsequent release. In advance of that, in order to exploit this feature, you will need to add the option REPROD to the enclone command line, to cause productive pairs to be recomputed. Allowing very long CDR3 sequences likely introduces some artifacts. For example, in human BCR data, we observe CDR3 sequences longer than 40 amino acids at a rate of about 1/10,000 cells, and these are typically part of a third chain for a given cell, suggesting that such chains are likely nonproductive. However it may be that long CDR3 sequences occur as part of productive chains in other species.
• Add a chain variable cdr2_aa_ref the gives the value of the CDR2 amino acids for the universal reference sequence, and similarly for other cases, see enclone help cvars.

11/20/20:

• Generalize the FCELL argument to allow arbitrary expressions. Please see enclone help special. (This is a breaking change.)

11/19/20:

• Add lead variable count_ that counts the number of occurrences of a given pattern in the V..J sequences for all chains.

11/4/20:

• Add SUBSET_JSON option, which allows construction of a small json file associated to specified clonotypes, which can be sent to the enclone developers.
• The NOPRINT option no longer applies to parseable output generated using POUT=stdout or POUT=stdouth. This change facilitates piping of parseable output to other programs.

10/20/20: If a variable is of the form abbr:name, and POUT is set to a file name, then the field label that is shown is now abbr, rather than abbr:name. This is really a bug fix.

10/9/20: Add parseable variables for the start and frame of the D region.

9/17/20: Clarify documentation for the case where GEX=(antibody data) and fix bug in handling of n_gex for that.

9/16/20:

• Test for defective reference sequences and exit if they're found (allowing for override). There is also a new argument BUILT_IN to force recomputation using the built-in reference (for human or mouse).
• We turned off part of the graph filtering (which can be turned off completely with the argument NGRAPH_FILTER). We did this because in some cases this code was deleting significant numbers of real cells.
• Tweak feature computations. The page this page has been updated accordingly. The results for tests of species described on that page did not change, but there were changes to other species, including a few changes for dog and horse.

9/4/20: Add the capability of seeing which default filters would have been applied, if the filters are turned off. See the lead variable "filter" at enclone help lvars.

9/2/20:

• Add options DIFF_STYLE=C1 and DIFF_STYLE=C2 that allow alternate labeling for the diff row in a clonotype table, which by default shows x's and dots.
• Fix bugs in the diff row printing: sometimes it was not shown, and it was not correctly handled when COLOR=property was specified.

9/1/20:

• Add parseable field vj_seq_nl, the DNA sequence of V..J, but starting after the leader.
• Add parseable field vj_aa_nl, the amino acid sequence of V..J, but starting after the leader.
• Add chain variable aa% = amino acid percent identity with donor reference, outside junction region.
• Add chain variable dna% = nucleotide percent identity with donor reference, outside junction region.
• Add summary of barcode fate if SUMMARY option is used.

8/31/20: add FWR4 support

8/28/20: add new filters CONST_IGH and CONST_IGKL that allow filtering of cells by isotype. Please see enclone help special.

8/27/20:

1. Fix bugs in reporting of insertions in the notes field.
2. Now CDR1, CDR2, FWR1, FWR2 and FWR3 can be can be computed and displayed. Please see enclone help amino and enclone help cvars for how to specify these, and this page for how we calculate these features.

8/24/20:

1. Add lead variable dref_aa, the number of amino acid differences with the donor reference.
2. Add estimated doublet rate to summary stats.
3. We are deprecating the lead variables sec and mem that measure UMIs supporting secreted versus membrane-bound antibody transcripts. Because the exon boundaries that determine this are far from the 5' end, there is rarely enough signal to be informative. We removed the documentation.

8/20/20:

1. Allow "Levenshtein distance patterns" for CDR3 filter specification. For more information, type or click here: enclone help filter.
2. Change what the weak onesies filter does: unless NWEAK_ONESIES is used, now, rather than delete certain dubious onesie clonotypes, such clonotypes are disintegrated into single cell clonotypes.
3. Change what the improper filter does: unless NIMPROPER is used, now, clonotypes having more than one chain but all of the same type are deleted.
4. Add a new filter that prevents merges of onesie clonotypes into other clonotypes; this filter may be turned off with a new argument NMERGE_ONESIES. This behavior is mostly (but not completely) a splitting out of previous functionality so as to simplify and clarify.

8/17/20: deprecate KEEP_IMPROPER; the new name is NIMPROPER.

8/10/20: tweak the definition of the weak onesies filter so that it does not delete single-cell clonotypes. Superceded by subsequent change on 8/18/20.

8/7/20: add the ability to analyze alternate splicing using GEX data to characterize UMIs as secreted or membrane, and display this information using lvars "sec" and "mem". See enclone help lvars for limitations. This is highly experimental.

8/5/20:

1. A reference file is now required as part of the Cell Ranger outs directory, if Cell Ranger version 4.0 or greater was used.
2. Deprecate MAX_SCORE and replace by MAX_LOG_SCORE, the base 10 logarithm of it.
3. Add cvar cdr3_len
4. Add and document knobs that allow nearly all clonotype join filtering to be turned off. Please see "enclone help how" and the end of "enclone help faq".

6/24/20:

1. Add support for iNKT and MAIT cells.
2. Make enclone faster. This is most noticeable in cases where many GEX datasets are provided as input.

6/19/20:

1. We now use a data hierarchy of donor (top), origin, dataset (bottom), where an origin is a set of 1 or more datasets from the same source (tube of cells, tissue, timepoint, etc.). (This breaks previous invocations of META.)
2. Improve the alternate (faster) internal storage structure for the GEX matrix created using the option NH5 as described using the command enclone help input. This will speed things up, particularly for the case where several datasets are combined. If you have already used the NH5 option for a given dataset, then the next time you run enclone on it, the file will be automatically rewritten. This would also apply to some datasets obtained as part of the large download.

6/17/20:

1. Now TREE=const can be used to show a tree with heavy chain constant region names attached to the leaves.
2. SEG and SEGN are now cumulative, so that multiple instances may be used to progressively filter.
3. The clonotype joining heuristic parameters MAX_SCORE and MAX_CDR3_DIFFS are now accessible.

6/10/20:

1. Add a complex of features for generating phylogenetic trees from clonotypes, see here.
2. New "single button" installation procedure.
3. Change the default value for PRE to ~/enclone/datasets,~/enclone/datasets2.
4. Add argument NALL to turn off all filters.
5. Add new lead variables nd<k> that display the number of cells in the top datasets for a given clonotype.
6. Add new lead variable dref that shows the distance of V..J from the reference outside the region of recombination.
7. Add argument COMPLETE to remove exact subclonotypes that do not have all chains.
8. Test for consistency between VDJ and GEX barcodes, and exit if this is not the case.
9. Add option COLOR=property to color amino acids by their properties.
10. Add option FCELL to allow filtering by cells.

5/29/20: Add new "UMI ratio" filter that further reduces noise in certain cases. This can be turned off using the argument NUMI_RATIO.

5/22/20: Add major new "UMI" filter that greatly reduces noise in certain cases. This can be turned off using the argument NUMI.

5/12/20: Add PLOT_BY_ISOTYPE to generate honeycomb plots colored by isotype.

5/1/20: Change the definition of the fields "edit" and "comp" to be based on alignment from the beginning of the CDR3 up to the end of the J, rather than stopping at the end of the CDR3. The intention is to capture the full region of recombination, which may not have been done before.

4/30/20: First release.