meaning of maxdist and maxdiff

[combioconference]

Hi dear author,
we want to use and cite your tool in our paper, but we are not so sure about the meaning of these two parameters: maxdist and maxdiff. For example: what is "cluster variant" ? What is haplotype difference?
Is there a paper link or preprint that we can cite?
Thank you!

[DonFreed]

There is currently no paper or preprint that you can use to cite the tool as it is still under active development.

The tool is designed to compare variants within the same genomic regions between two VCFs. In the first part of this comparison, the tool needs to decide the size of the genomic region and which variants from each VCF fall within this region. Currently the tool will consider variants that are within some distance along the reference genome to be part of the same region (controlled by the --maxdist argument). Within this region, all of the variants in the calls VCF are compared to all of the variants in this region in the truth VCF, and the TP/FP/FN assignment is applied for the entire region.

The tool then performs a comprehensive assembly of all variants in the truth and calls VCFs (and nearby reference bases) to generate all possible haplotypes from the two VCFs. The assembled haplotypes are then compared between the truth and sample VCFs using either the Levenshtein ratio (default) or a comparison of the event size (with --metric Length).

The tool allows some flexibility in the variant representation, so the haplotypes do not need to exactly match between the calls/truth VCFs. The --maxdiff metric controls the strictness of this matching. If both haplotypes match with (1 - Levenshtein_ratio) < --maxdiff, then the tool considers the truth and calls VCFs to fully match across this region (an MM match). If either haplotype matches with a ratio > --maxdiff, then the region is considered a combined FP/FN site (with either an MX or XX match).

Hope this helps clarify these arguments!

[combioconference]

Thank you so much! But I'm not sure about the procedure of "assembly of all variants in the truth and calls VCFs (and nearby reference bases) ", could you give a little more detail about that, please?

When you compare two "haplotypes", what are you comparing? SNPs or SVs?

Thanks!

[DonFreed]

Happy to provide more details.

By "haplotypes", I mean just a string of ACGT bases representing the DNA sequence in the sample at a particular locus, incorporating SNVs, indels, and SVs present in the sample.

Say we have a VCF with the following variants:

chr1 100 A C ... GT 0/1
chr1 105 G T ... GT 0/1

And the reference sequence from chr1:95-110 is:

AGAGGAGTTTGTAGTA
|    |    |
95   110  105

The sample is diploid at this locus, and we need two haplotypes to fully represent these variants. One potential haplotype representation for the sample is:

AGAGGCGTTTGTAGTA (100 A->C)
AGAGGAGTTTTTAGTA (105 G->T)

The VCF does not contain phase information, so it is also possible the variants occur on the same haplotype:

AGAGGCGTTTTTAGTA (100 A->C and 105 G->T)
AGAGGAGTTTGTAGTA (ref)

When we compare the haplotypes, we just compare the base sequences.

By "a comprehensive assembly of all variants in the truth and calls VCFs", I mean that we take all variants within a region and then generate all possible haplotype combinations for the VCF over the given region, similar to the simple example above. The above example has only 2 possible haplotype representations. In actual samples the number of possible haplotypes can be much higher at some loci.

[archmageirvine]

Can you explain how the comparison works when the calls, say, do not contain nucleotide resolved events. Is that like using --maxdiff 1 ?

[ljohansson]

I am also trying to get my head around the MX and XX calls and them being counted as combined FP/FN, meaning they will affect both recall and precision and therefore have a double impact on the F1 score. Would it make sense to distinguish MX from XM and have the first counted as a FP and the second as a FN result for the statistics?

The most difficult part to understand is the effect of --maxdiff? This is the difference from the combined variants in a region compared to the reference sequence if I am correct.

• With the default threshold of --maxdiff = 0.2 this means that an MM will change to an MX or XX when the Levenshtein ratio is < 0.8. Can I read that as less than '80%' similarity between the sequence in the region as represented by the vcf compared to the reference?.
• If so, what does this mean? Is it that the region is very complex?
• And could the present of and MX or XX region mean that in that region there is a complex variant?
• What if the truth vcf and the vcf to check have quite similar variants, but one has for instance a Levenshtein score of 0.79 and the other of 0.81, will it become an MX, even when the results are almost the same?

[DonFreed]

Would it make sense to distinguish MX from XM and have the first counted as a FP and the second as a FN result for the statistics?

It might make sense to do this in some cases. For example, if the base VCF contains a 200bp deletion and a 300bp insertion while the comp VCF only contains the 300bp insertion, we might mark the site as an FN instead of an MX.

Can I read that as less than '80%' similarity between the sequence in the region as represented by the vcf compared to the reference? If so, what does this mean? Is it that the region is very complex?

hap-eval will assemble haplotype strings from the variants in the VCF. The strings contain the variants from the VCF, any intervening reference sequence (if the region contains multiple variants that are separated by some distance), and 100bp of padding on each side of the variant. If the haplotype strings from the comp VCF have a Levenshtein ratio >80% compared to the haplotype strings from the base VCF, the site will be marked as MM, otherwise the site is marked as MX or XX.

And could the present of and MX or XX region mean that in that region there is a complex variant?

No, a MX or XX call does not necessarily imply a complex variant. An MX call can occur when the base VCF has a homozygous 300bp deletion, and the comp VCF has a heterozygous 300bp deletion, for example.

What if the truth vcf and the vcf to check have quite similar variants, but one has for instance a Levenshtein score of 0.79 and the other of 0.81, will it become an MX, even when the results are almost the same?

hap-eval is very generous with the Levenshtein ratio calculation given that 200bp of padding and any intervening reference sequence is added to both the truth/base and comp haplotypes. With a Levenshtein score of 0.79, the variants would not be similar.

[DonFreed]

I have updated the README to include more information on the --maxdist and --maxdiff arguments. I hope that these new sections are helpful in clarifying some questions.

[DonFreed]

Can you explain how the comparison works when the calls, say, do not contain nucleotide resolved events. Is that like using --maxdiff 1 ?

hap-eval requires nucleotide-resolved events in order to construct haplotypes from the variant alleles. With <DEL> and <DUP> symbolic alt alleles, hap-eval will infer the nucleotide sequence from the allele information. Other symbolic alt alleles or breakend events (BND) are not supported.

[ljohansson]

Thank you. The updated README makes it much more clear. I tend to struggle to interpret the 'MX' scores. I know they represent the two alleles, but sometimes I think they refer to the baseline and comparison sequences. So my thought that XM would represent a FP result is false, since it would just mention it is the 'second' haplotype with a too high maxdiff.

The second mistake in my thought process is the maxdiff calculation itself. I thought there were two: 1. base vcf to reference and 2. comparison vcf to reference. However, now I understand the reference is only used to create the base and comp sequences and than for each of the haplotypes a single Levenshtein ratio is calculated that directly compares the base sequence with the comparison sequence.

I have a small secondary question: my comparison vcf includes a few inversions, however my baseline vcf only includes deletions and insertions (and SNVs and Indels), how would this affext the Levenshtein ratio?

For instance, in the HG002 sample I found 5 such variants. I extracted for each region the largest variant (based on SVLEN) in the baseline vcf and the largest variant in the comparison vcf. I know this is not perfect, but I thought it would give an indication on what is the largest contributor to the sequence. Especially the third one is interesting. According to UCSC blat the insertion of the middle sequence baseline vcf maps perfectly to the minus strand of the region of the inversion of the comparisonvcf. Yet, this region is labeled as FN. Can you explain the logic behind this?

I have looked into the separate vcfs and realized that more variants were listed. I have pasted them below. Perhaps this affects the comparison?

Label Region Fields     BaselineVariant ComparisonVariant
FN chr3:82152697-82154955 2 0 2259 (0,-1382) (0,0) 1.0  chr3    82152798        .       TTCCCTCTTTTTCTATTGATTGGAATAGTTTCAGAAGGAATGGTACCAGTTCCTCCATGTACCTCTGGTAGAATTCGGCTGTGAATCCATCTGGTCCTGGACTCTTTTTGGTTGGTAAACTATTGATTATTGCCACAATTTCAGAGCCTGTTATTGGTCTATTCAGGGATTCAACTTCTTCCTGGTTTAGTCTTGAGAGAGTGTATGTGTCGAGGAATGTATCCATTTCTTCTAGATTTTCTAGTTTATTTGCGTAGAGGTGTTTGTAGTATTCTCTGATGGTAGTTTGTATTTCTGTGGGATCGGTGGTGATATCCCCTTTATCATTTTTTATTGTGTCTATTTGATTCTTCTCTCTCTTTTTCTTTATTAGTCTTGCTAGCGGTCTATCAATTTTGTTGATCCTTTCAAAAAACCAGCTCCTGGATTCATTGATTTTTTGAAGGGTTTTTTGTGTCTCTATTTCCTTCAGTTCTGCTCTGATTTTAGTTATTTCTTGCCTTCTGCTAGCTTTTGAATGTGTTTGCTCTTGCTTTTCTAGTTCTTTTAATTGTGATGTTAGGGTGTCAATTTTGGATCTTTCCTGCTTTCTCTTGTAGGCATTTAGTGCTATAAATTTCCCTCTACACACTGCTTTGAATGCGTCCCAGAGATTCTGGTATGTGGTGTCTTTGTTCTCGTTGGTTTCAAAGAACATCTTTATTTCTGCCTTCATTTCGTTATGTACCCAGTAGTCATTCAGGAGCAGGTTGTTCAGTTTCCATGTAGTTGAGCAGCTTTGAGTGAGATTCTTACTCCTGAGTTCTAGTTTGATTGCACTGTGGTCTGAGAGATAGTTTGTTATAATTTCTGTTCTTTTACATTTGCTGAGGAGAGCTTTACTTCCAACTATGTGGTCAATTTTGGAATAGGTGTGGTGTGGTGCTGAAAAAAATGTATATTCTGTTGATTTGGGGTGGAGAGTTCTGTAGATGTCTATTAGGTCTGCTTGGTGCAGAGCTGAGTTCAATTCCTGGGTATCCTTGTTGACTTTCTGTCTCGTTGATCTGTCTAATGTTGACAGTGGGGTGTTAAAGTCTCCCATTATTAATGTGTGGGAGTCTAAGTCTCTTTGTAGGTCACTGAGGACTTGCTTTATGAATCTGGGTGCTCCTGTATTGGGTGCATAAATATTTAGGATAGTTAGCTCCTCTTGTTGAATTGATCCCTTTACCATTATGTAATGGCCTTCTTTGTCTCTTTTGATCTTTGTTGGTTTACATAGTTCACACAAAGGCTCTTTAAAGCCTTAGTAAATGCACATGTCGCTGCCACTAACTAGATTACCTGGAGAGGAAACTGAAAGTAGAGAGAGATAGAGGTTTAAAGTCTGTTTTATCAGAGACTAGGATTGCAACCCCTGCCTTTTTTTGTTTTCCATTGGCATAAAGAAATGCTCATTTTCTGTTGCTTGTACTGTTGTTGAGTCCTAGTTCTTTCCTGACTTATTTCCATAAGAACCTCGTGCTTAAGAATCCTGTATATTTCATAGTCAAGGAGGATAGAGAACCATAAAGCAGGTTTAGATTTTTAATCTATGACTTTTTGCCTTTAACAAGAGCAATGCTCAGTCTTGCAAGGAAGATAGACATTACATTATTGAAATGACTTTTAAGGCAGAAGAGCAGCCGAATTCTACATATTTTATTTTTTGGCTTTTTAAATGAAAACTGTCTTTAGCTTTGTATTTTTAAGAGATTCAAATTCAATACTTTAAGAAATTCATGAATAAAGCATGTTTCAACCTTAAATTACATATAAGCATGATTTATAATTTATTGCCCATAAAACTTTTATCAAAGAAACAAGAAGAGTGGTATGTATGCAAATACAGGGATGCAAAAATAAACATATTTCTATAAGCTGTGCACCAGAAAAGAAAGGGCATTTGATTTATATTGGATTGTAGCCACATTTAAAAGAGAGGTACAAGGAGGAGCTGGTACCATTCCTTCTGAAACTGAAGAACAGTATAGATTTTGCTTTCTGAGTGTACTCTTTCAAGCAGACACGATTCCC  T       30      .       SVTYPE=DEL;SVLEN=2057;LCR=0.960004;REMAP=partial        GT:AD   0|1:1,1 chr3    82152798        cuteSV.INV.350  T       <INV>   39.7    PASS      PRECISE;SVTYPE=INV;SVLEN=1668;END=82154466;RE=17;STRAND=++;AF=0.3542    GT:DR:DV:PL:GQ  0/1:31:17:40,0,173:39
FN chr6:12428009-12433073 24 0 5065 (5,565) (0,0) 1.0   chr6    12432447        .       C       CATTCCCAGCACTGTGCTCCTCTGAGTAAGTCTGGTGTACAATGTGGAGCATGTGAAGCCTGATTGTTCTAGGCACAAATTATAACTGCCTCCTCTTCAGTGGGAGAATTTAGAGTGGGCTTACAAGAAAAATAGGATCCAGCAAGAGTAAGTCTCTCTATGAATAATAGATAAAAGATTTTTTCAATAATCCCCAGAAACTGCCTGCTCTTTCCTCATCTGGCAACTTTGATTTGGGTTTATATGGATTTTATTAGTTTAAAATGTTATGCATACATAGAGCCCTAACTTGCCTTGTTATTTACTTCAAGTGCAGACCACATAGGTTTTGATGTATTATAAGTAATCCTATTATAATGTTTAGGTGGCCTGAGCTAACTTGAGCTTTCTGTCCAAACACATTACAGAAACAGCAATGCTTCCTTCCTCAGCAAATGCTTAAATGCTGATAGTGATGCCCGGGGCCAGACTCTTTGCCTAGCCCTTTGGGGAATAAAAAATCTATTCTTGACATGCCTTGGAAATTATTCTGTAACAAGATTCAATAGTTATTTCAAAAAGCCCAGAGCCGAGTGAAGGAATTTTGATACACAGACTAAGGAACTTGAGCAAAAGGATCCAACCGCTGCATCTTAGAGCTTAATACTTAGCCTGATTCAGTACTAGATCCCTGGTACTGATGCCCTGACTTTACAATGGCCTCCCACACTGATCATTAAGTTGATGACTTAGCTCTTGACCCACTAGTTGGGGATGTGATAAAACTACACCGGTGGGTCTATAACTCCATAAAAATGGTGATCATAAAAACAGAACTACCATTCAACTCAGCAATCCCATTACTGAATATATACCCAGAGGAATATAAGTCATTCTACCATAAAGACATATGCACATAAATGTTCATTGCAGCACTATTCACTATAGCAAAGACATGAAATCAACCTAAATGCCCATCAATGACAGATTGGATAAAGAAAATGCGGTACATATACACCATAGAATACTATGCAGCCATAAAAAAGAATGAGATTATGTCTTTTGTGGGAACGTGGATGGAGCTAGAGGCTATTATCATCAGCAAACTAATGCAGGAACAGAAAAACAAATGTCAAATGTTCTCACTTATAAATGGGAGCTAACTGGTAAGAATTTATGAACACAAGGAAACAAAAGATACTAAGGTCTACTTGAGGGTGGTGCTTCTTCTTTTGTAATGTTT      30      .       SVTYPE=INS;SVLEN=1221;LCR=0.978807;REMAP=partial        GT:AD   0|1:1,1 chr6    12431799        cuteSV.INV.411  A       <INV>   72.8      PASS    PRECISE;SVTYPE=INV;SVLEN=649;END=12432448;RE=21;STRAND=--;AF=0.42       GT:DR:DV:PL:GQ  0/1:29:21:73,0,149:72
FN chr8:6280018-6297962 85 0 17945 (1893,1875) (0,0) 1.0        chr8    6297862 .       G       GTGTATATGTATGCATACACACATCTCCTATCAGTTCTATCTCTTTGGAGAACCCTAACTAAAACGTGACTTGAAAAACACAATTTAGCAAAAATGATGCAAGAAGAAATAGAAAAGTTGAGGGCTATTATTAATATATGCACTTAAAAAATTTAATTCATAATTTACAATCTTTTCCAGAAGCAAATTGCAGGCCCAGATGATTTCACCATTATATTCTTCCTACTATTTAAGGAAGAAATAACATCAATCTTACATGAAATCTTACAGAACAATTTTAAAAAGGAAAAAGTTGAAATGCCAACTCTTTTGATAACAGTGACATAGCCTGGGCTGGGCGTGGTGGCTCATGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGTGGATCACGAGGTCAAAAGATTGACACTACCCTGGCTAACATGGTGAAACCCCGTCTCTACTAAAAATACGAAAAATTAGCCGGGCGTGGTGGCTGGCGCCTGTAGTCCCAGCCACTTGGGAGGCTGAGGCAGGAGAAAGGCGTGAACCCAGGAGGCGGAGCTTGCAGTGAGCCGAGATCGCACCACTGCACTCCAGCCTGGGTGACAGAGTGAGACTCCATCAAAAAAAAAAAAAAAAAAAAAAAACCACAAGAAACATAACCTGGATACTAAAACCTAATAAGGACATTGCAATAAAAAGAAATATTACAGGTCAGTAATTCTCATGAACACAGGTGCAAAACAATACATAGAATAATAGTGAATCAAACCTAGTAATATGTAATAATACATAACCATGAAGTTGGTTTTATTCCAGGAATGCAAGATTGAGTTAAGATTAGAAAATTGATCAATGATATTATTCACATTAACAGAAGAAAAGAGAAAAAGAAAATAATCATCAACACAAATGTGCTGCAGGCAGTGGTGGTCGTGACACCCATTCTACAGCTAAAAAATTAGTGTGGGCCAGACGCGGTGGCTCACGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGTGGATCATGATGTCAGGAGTTCAAGACCAGCCTGGCCAAGATGGTGAAACCCCGTCTCTACTAAAAATCCAAAAATAATTACCTGCGCATGGTGACAGGTGCCTGTAATCCCAGGTACTAGCTGAGAGGCAGAGAATTGCTTGAACTCGGGAGATGGAGGTTGCAGTGAGCTGAGATCACACCACTGCACTCCAGCCTGGGCAACAGAGCGAGACTACGTCTCAAAAAACAAAACAAAACAAAACAAAACAAAAAAGTATAAAAAACAACAACAACAACAAAACTAATGTGAAAATCACTGTCCTATGAAGTTATTAATAGCTTTGAATGACCTCATTATACTTCTTCCAACTGGCAGTTTACTTCCATTGTTAATGCTCATCTCTGTTTGATTACAGTACAACTCTGCTCACCTTTCTACCACCAGCATTCAATTGCTTTTTAAAGAATTGTAAGGAATAGAGCACGAGATCTGGAGTCTGAAGATTTAGAAAAAGTCACACAACTCTCTAAAATTGTTTCCTCATCAATGAAATAGAGGTACCAACACCTGCTTTGTTACATTGCAGGCAATGAATTGGAAAATGCTCAACTAATTATGAAGCACCTACAAATACTGTGATTGTTTTACCCTCCCAAAACTTAAATGGAGCAATTAACGGTGAAAGATATTGGGTTTCATTCAGGATACAACCCTCTTAATAAAAATACGTTTATCTATTTTACATATGGAAATTAAAAGTGCTTACCTTGCTAGATGGAAAATTGAATGAATACAAATGATCGGAATGATAACAGCTAAGTGAATGCTTTAGTTGAAAAATGATACCCCTAAGTCATCAAAGATGTCAAATCATCTCCCTCTCCAAAAAGAGTCACTCTCCACCCCACCCCC       30      .       SVTYPE=INS;SVLEN=1906;LCR=0.97496;REMAP=interspersed    GT:AD   0|1:0,1 chr8      6297863 cuteSV.INV.464  G       <INV>   162.2   PASS    PRECISE;SVTYPE=INV;SVLEN=1907;END=6299770;RE=19;STRAND=++;AF=0.9048     GT:DR:DV:PL:GQ  1/1:2:19:162,35,0:34
FN chr20:10806164-10807916 4 0 1753 (611,0) (0,0) 1.0   chr20   10807816        .       T       TCTGTCGCCCAGGCTGGAGTGCAGAGGCGCGATCTCGGCTCACTGCAAGCTCCGCCTCCCGGGTTCACACCATTCTCCTGCCTCAGCCTCCCGAGTAGCTGGGACTACAGGTGCCCACCACCATGCCCAGCTAATTTTTTGTAATTTTAGTAGAGACGGGGTTTCACCGTGTTAGCCAGGATGGTCTCGGCCTCCTGACCTCGTGATCCGCCAGCCTCAGCCTCCCAAAGTGCTGGGATTACAGACGTGAGCCACTGCGCCCGGCCCAACATGGCTTTTTTCTTCTTTCTACCACTCTTTCCCTGTCTCTGTCTTTCATTTAAACCTCACCAACAGAGAGAATACCATTGAGTCAGGAGGTGACCACCCATTAGGGACCCTCCACTGGTTAGAGTGCTCATTTTAGGCTTCCTCAGAGCTAGCTCAGCCCATGGATGGTTGCATTTGATTCAGGATCCCTGGAACCATCTGCAAAGGCAAGGCCAATAGGGTCATATGACATTAAACACAGCTATGTTGCTCAGTAAGGGACTGTCCTTGGTAAGCTCCACCTGCCCCTTCAGATCCATTCTCCACTCTTCTCTGCCCTGCTATGTACCCTGAAGGCTGAGCTCTATAA   30      .       SVTYPE=INS;SVLEN=618;LCR=0.991243;REMAP=interspersed    GT:AD   1|0:1,1 chr20   10807797        cuteSV.INV.312    G       <INV>   99      PASS    PRECISE;SVTYPE=INV;SVLEN=639;END=10808436;RE=20;STRAND=--;AF=0.5556     GT:DR:DV:PL:GQ  0/1:16:20:99,0,61:60
FN chrX:6217321-6219108 14 0 1788 (1680) (0,0) 1.0      chrX    6219008 .       A       ATCTTCATAAAATATTTTCCTCTGAGACAATGCAATCTATGCCCCATCTTTAAGCAATCCTAGCTCACCAGTATGAGTAATGTTGTCTATTTTTAAAGTTATCTCATTATTCTAAAAGACTTTAAATTGTTGAAAAATAAATTGTGTGAGGGGTGGTAGAGTTTGAAACAATTATCTGTGATGTTACCAGATATTTTAGACTAAAATATATTAGAATCCAAGGTATTGTTCATGCCTTAAAAATGCTGAAATATCTGACTGTTGCTTATTAATTTTAAAAAGAATATAGGAAATAGCCATTAATTAATGAGGCTGTTTCCACTACCACATAAAAAAAAAAAAAGCTCACAGGTGCCTGTATGTTTTTGTCGAATCAAAGTAATCTGCTTTATGTATGCATTTATTCATAGAATTTACTAAAATCAAAATCAAGGTTTTATAAATATAGGGTTTGACAAAGTTTTAAAATATAACCAGCTATACAAATATGGCATGTGGGAAATTCTATTAAATTGTCATGAACATGCTTCTTTGTCATTCCAGGAGTCTTTCTTTTCATTACTCTTTCCTATTTGATCTGTTATTCTATAGAATTATCTTCATTTTCTCTTTAATACTTTAAGGATCCCTGAGACCTTGTCACTCATCCAAATAGAATAAAGGAATGAGGGAAGAAAGAAGGAAGGGAAGAAGGAAGGAAGAAAGGACAGAAGGAAGAAGGAAGGGATGGAGGAACGAAGGGAGGAAGGAAGGGAAGGAAAGAAGGAAGGAAGGAAGGAAGAAAGAAAGAAAAGAGAAAAGGAAGGAAGAAAGGAGGCAGGGAGGGAGGGAGAAGGATGATAGATCGAAGGGAAGGAGAGAAGGAGGGAAGGAGGGAAAGAAGAAAGGAAGGAAAGAAAGAAGAGAAAAAGGTAGGAAGGAGAGAGGGAGGGAATGAAGGAAAGAAGGAAGGAAAGAAAGAAGAGAAAAAGAAGGAAGGAAGGAGGGAAGGAAGGAAGGAAAGAAGAAAGGAAGGGAAGAGGGAAGGAAGCTGTGGTTTCTGTGCACCATTAACTAACAATAGCTCCTGTGAACCAGCCTGGAAGTTCATTCACCACATACAATATAGTTTCTTTTGGAAAATGCATGTCAAACTATATATATGGTTTGTGTGTGTGTATATATATATATATACACACACACACATACATATATATACGTATATATGTATATACATATAAAATTTGCATGTACTTTTCATACAAAAGAACAAGAACTATATATATACAATATATATACATTGGAATATATTTATTATAGATGTATATGGATTTTACTATATATTATTTCATGTGTAGGTAGAATTCTAAAGAACCTGTGCACAGTGGTGAGTGAACAACCTAAATGAGAAACCCTCTAGCATTCATTCCCTTTTGCTCCTGTCGTCACCTGGAATTGTCAATGCTTTTCAGTAGGTGCAATGTGAATGAGTAGCGCACATTAAATTTGAATTCAAGGAAGTCTGCAGGAAGAGAGGGACCCAAGAACATAAAGAGACTGTAATGAGGTTCTCAGCTTTTGTGACTTCAAACTTCTCCATATTTATTAAAAGTGAATGCAGAGTTTCTGAAGCTGGGCTGATACAATGCTCTACTGACTTTGCCACAAGGCTTCAATGGGAACAGTCTCTCAGCGAAGCCTTTG     30      .       TRF;SVTYPE=INS;SVLEN=1682;LCR=0.953016;REMAP=partial    GT:AD   1:0,1   chrX    6219010 cuteSV.INV.531    T       <INV>   219.5   PASS    PRECISE;SVTYPE=INV;SVLEN=1341;END=6220351;RE=23;STRAND=--;AF=1  GT:DR:DV:PL:GQ  1/1:0:23:219,59,0:58

Note (for the first two the baseline vcf actually had two variants next to eachother:

chr3    82152797        .       A       AGACACACGGAGAAAACTCCAATGTGAAGACAGAAGATTGGACTGATGTATTAAAAGCTAAGAAACACCTATGGCTAGCAGAAGACAGGGAAGTGGCATGGAATGGTTCTTTTCCCAGCACCTTCAGTGGGAGCATAGTCCTGACACACCTTGACCTTGGAAGTCTGGCCAGTAGAACTGCAAGACCACACATTTATTTTGTTTTAAGTCACCCTGTTTGTGGAACTTTGTTAAGGCAGCCCTAAGAAACTCTTTAATAATACTGTTTCTCATTACTTCTAAAATATATCTTGTACTCTTTAAGATAGATCTCAGGCCTTTCAGGATCTTGGCCCCGCACACCTTTTCAGCTCCTGCATTACTAACGTGTTCCTTTACACCCTGTCATGCAACAATAGAAGACACCTTTATTCCTCAAAATATCCTGTTTTCTCTCACCTATAGCCTTCTCCATTCTCCTCCCTGCAAAGTTAGCATTATCCTTTTCCCTGACACACCGCCATCCCAAATCTTCAACCGATGACTCCTAACCATCTTTTAGACCTGAACGTCAATATTACTTCATCTTTCAGAAGGACTTTCTTGTTTTTCTAGTCTCGCATGGGACCCTGCTCTTCTGCCTTAAAAGTCATTTCAATAATGTAATGTCTATCTTCCTTGCAAGACTGAGCATTTC  30      .       SVTYPE=INS;SVLEN=675;LCR=0.980461;REMAP=partial GT:AD   0|1:1,1
chr3    82152798        .       TTCCCTCTTTTTCTATTGATTGGAATAGTTTCAGAAGGAATGGTACCAGTTCCTCCATGTACCTCTGGTAGAATTCGGCTGTGAATCCATCTGGTCCTGGACTCTTTTTGGTTGGTAAACTATTGATTATTGCCACAATTTCAGAGCCTGTTATTGGTCTATTCAGGGATTCAACTTCTTCCTGGTTTAGTCTTGAGAGAGTGTATGTGTCGAGGAATGTATCCATTTCTTCTAGATTTTCTAGTTTATTTGCGTAGAGGTGTTTGTAGTATTCTCTGATGGTAGTTTGTATTTCTGTGGGATCGGTGGTGATATCCCCTTTATCATTTTTTATTGTGTCTATTTGATTCTTCTCTCTCTTTTTCTTTATTAGTCTTGCTAGCGGTCTATCAATTTTGTTGATCCTTTCAAAAAACCAGCTCCTGGATTCATTGATTTTTTGAAGGGTTTTTTGTGTCTCTATTTCCTTCAGTTCTGCTCTGATTTTAGTTATTTCTTGCCTTCTGCTAGCTTTTGAATGTGTTTGCTCTTGCTTTTCTAGTTCTTTTAATTGTGATGTTAGGGTGTCAATTTTGGATCTTTCCTGCTTTCTCTTGTAGGCATTTAGTGCTATAAATTTCCCTCTACACACTGCTTTGAATGCGTCCCAGAGATTCTGGTATGTGGTGTCTTTGTTCTCGTTGGTTTCAAAGAACATCTTTATTTCTGCCTTCATTTCGTTATGTACCCAGTAGTCATTCAGGAGCAGGTTGTTCAGTTTCCATGTAGTTGAGCAGCTTTGAGTGAGATTCTTACTCCTGAGTTCTAGTTTGATTGCACTGTGGTCTGAGAGATAGTTTGTTATAATTTCTGTTCTTTTACATTTGCTGAGGAGAGCTTTACTTCCAACTATGTGGTCAATTTTGGAATAGGTGTGGTGTGGTGCTGAAAAAAATGTATATTCTGTTGATTTGGGGTGGAGAGTTCTGTAGATGTCTATTAGGTCTGCTTGGTGCAGAGCTGAGTTCAATTCCTGGGTATCCTTGTTGACTTTCTGTCTCGTTGATCTGTCTAATGTTGACAGTGGGGTGTTAAAGTCTCCCATTATTAATGTGTGGGAGTCTAAGTCTCTTTGTAGGTCACTGAGGACTTGCTTTATGAATCTGGGTGCTCCTGTATTGGGTGCATAAATATTTAGGATAGTTAGCTCCTCTTGTTGAATTGATCCCTTTACCATTATGTAATGGCCTTCTTTGTCTCTTTTGATCTTTGTTGGTTTACATAGTTCACACAAAGGCTCTTTAAAGCCTTAGTAAATGCACATGTCGCTGCCACTAACTAGATTACCTGGAGAGGAAACTGAAAGTAGAGAGAGATAGAGGTTTAAAGTCTGTTTTATCAGAGACTAGGATTGCAACCCCTGCCTTTTTTTGTTTTCCATTGGCATAAAGAAATGCTCATTTTCTGTTGCTTGTACTGTTGTTGAGTCCTAGTTCTTTCCTGACTTATTTCCATAAGAACCTCGTGCTTAAGAATCCTGTATATTTCATAGTCAAGGAGGATAGAGAACCATAAAGCAGGTTTAGATTTTTAATCTATGACTTTTTGCCTTTAACAAGAGCAATGCTCAGTCTTGCAAGGAAGATAGACATTACATTATTGAAATGACTTTTAAGGCAGAAGAGCAGCCGAATTCTACATATTTTATTTTTTGGCTTTTTAAATGAAAACTGTCTTTAGCTTTGTATTTTTAAGAGATTCAAATTCAATACTTTAAGAAATTCATGAATAAAGCATGTTTCAACCTTAAATTACATATAAGCATGATTTATAATTTATTGCCCATAAAACTTTTATCAAAGAAACAAGAAGAGTGGTATGTATGCAAATACAGGGATGCAAAAATAAACATATTTCTATAAGCTGTGCACCAGAAAAGAAAGGGCATTTGATTTATATTGGATTGTAGCCACATTTAAAAGAGAGGTACAAGGAGGAGCTGGTACCATTCCTTCTGAAACTGAAGAACAGTATAGATTTTGCTTTCTGAGTGTACTCTTTCAAGCAGACACGATTCCC        T30       .       SVTYPE=DEL;SVLEN=2057;LCR=0.960004;REMAP=partial        GT:AD   0|1:1,1

chr6    12431797        .       CAAAGAGTCTGGCCCCGGGCATCACTATCAGCATTTAAGCATTTGCTGAGGAAGGAAGCATTGCTGTTTCTGTAATGTGTTTGGACAGAAAGCTCAAGTTAGCTCAGGCCACCTAAACATTATAATAGGATTACTTATAATACATCAAAACCTATGTGGTCTGCACTTGAAGTAAATAACAAGGCAAGTTAGGGCTCTATGTATGCATAACATTTTAAACTAATAAAATCCATATAAACCCAAATCAAAGTTGCCAGATGAGGAAAGAGCAGGCAGTTTCTGGGGATTATTGAAAAAATCTTTTATCTATTATTCATAGAGAGACTTACTCTTGCTGGATCCTATTTTTCTTGTAAGCCCACTCTAAATTCTCCCACTGAAGAGGAGGCAGTTATAATTTGTGCCTAGAACAATCAGGCTTCACATGCTCCACATTGTACACCAGACTTACTCAGAGGAGCACAGTGCTGGGAATGGCTTGGGTGGTGAACTTGCAATATGGCTTCAAGAGGCCCAGCAGTCCCATCCAGAGTTGCATGACACTTCTTATACAGAAGAGCAGAGATGCCCAAGAGTTCATGTCATTCACTTAACAAGTCAGCATGGTAAAGGTTCTTTTTTTCCACAGCACATTAAATTAAAAACATAAA    C       30      .       SVTYPE=DEL;SVLEN=649;LCR=0.983825;REMAP=partial GT:AD   0|1:1,1
chr6    12432447        .       C       CATTCCCAGCACTGTGCTCCTCTGAGTAAGTCTGGTGTACAATGTGGAGCATGTGAAGCCTGATTGTTCTAGGCACAAATTATAACTGCCTCCTCTTCAGTGGGAGAATTTAGAGTGGGCTTACAAGAAAAATAGGATCCAGCAAGAGTAAGTCTCTCTATGAATAATAGATAAAAGATTTTTTCAATAATCCCCAGAAACTGCCTGCTCTTTCCTCATCTGGCAACTTTGATTTGGGTTTATATGGATTTTATTAGTTTAAAATGTTATGCATACATAGAGCCCTAACTTGCCTTGTTATTTACTTCAAGTGCAGACCACATAGGTTTTGATGTATTATAAGTAATCCTATTATAATGTTTAGGTGGCCTGAGCTAACTTGAGCTTTCTGTCCAAACACATTACAGAAACAGCAATGCTTCCTTCCTCAGCAAATGCTTAAATGCTGATAGTGATGCCCGGGGCCAGACTCTTTGCCTAGCCCTTTGGGGAATAAAAAATCTATTCTTGACATGCCTTGGAAATTATTCTGTAACAAGATTCAATAGTTATTTCAAAAAGCCCAGAGCCGAGTGAAGGAATTTTGATACACAGACTAAGGAACTTGAGCAAAAGGATCCAACCGCTGCATCTTAGAGCTTAATACTTAGCCTGATTCAGTACTAGATCCCTGGTACTGATGCCCTGACTTTACAATGGCCTCCCACACTGATCATTAAGTTGATGACTTAGCTCTTGACCCACTAGTTGGGGATGTGATAAAACTACACCGGTGGGTCTATAACTCCATAAAAATGGTGATCATAAAAACAGAACTACCATTCAACTCAGCAATCCCATTACTGAATATATACCCAGAGGAATATAAGTCATTCTACCATAAAGACATATGCACATAAATGTTCATTGCAGCACTATTCACTATAGCAAAGACATGAAATCAACCTAAATGCCCATCAATGACAGATTGGATAAAGAAAATGCGGTACATATACACCATAGAATACTATGCAGCCATAAAAAAGAATGAGATTATGTCTTTTGTGGGAACGTGGATGGAGCTAGAGGCTATTATCATCAGCAAACTAATGCAGGAACAGAAAAACAAATGTCAAATGTTCTCACTTATAAATGGGAGCTAACTGGTAAGAATTTATGAACACAAGGAAACAAAAGATACTAAGGTCTACTTGAGGGTGGTGCTTCTTCTTTTGTAATGTTT      30      .       SVTYPE=INS;SVLEN=1221;LCR=0.978807;REMAP=partial        GT:AD   0|1:1,1

The third one in the baseline had almost the same variant on both alleles:

chr8    6297862 .       G       GTGTATATGTATGCATACACACATCTCCTATCAGTTCTATCTCTTTGGAGAACCCTAACTAAAACGTGACTTGAAAAACACAATTTAGCAAAAATGATGCAAGAAGAAATAGAAAAGTTGAGGGCTATTATTAATATATGCACTTAAAAAATTTAATTCATAATTTACAATCTTTTCCAGAAGCAAATTGCAGGCCCAGATGATTTCACCATTATATTCTTCCTACTATTTAAGGAAGAAATAACATCAATCTTACATGAAATCTTACAGAACAATTTTAAAAAGGAAAAAGTTGAAATGCCAACTCTTTTGATAACAGTGACATAGCCTGGGCTGGGCGTGGTGGCTCATGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGTGGATCACGAGGTCAAAAGATTGACACTACCCTGGCTAACATGGTGAAACCCCGTCTCTACTAAAAATACGAAAAATTAGCCGGGCGTGGTGGCTGGCGCCTGTAGTCCCAGCCACTTGGGAGGCTGAGGCAGGAGAAAGGCGTGAACCCAGGAGGCGGAGCTTGCAGTGAGCCGAGATCGCACCACTGCACTCCAGCCTGGGTGACAGAGTGAGACTCCATCAAAAAAAAAAAAAAAAAAAAAAAACCACAAGAAACATAACCTGGATACTAAAACCTAATAAGGACATTGCAATAAAAAGAAATATTACAGGTCAGTAATTCTCATGAACACAGGTGCAAAACAATACATAGAATAATAGTGAATCAAACCTAGTAATATGTAATAATACATAACCATGAAGTTGGTTTTATTCCAGGAATGCAAGATTGAGTTAAGATTAGAAAATTGATCAATGATATTATTCACATTAACAGAAGAAAAGAGAAAAAGAAAATAATCATCAACACAAATGTGCTGCAGGCAGTGGTGGTCGTGACACCCATTCTACAGCTAAAAAATTAGTGTGGGCCAGACGCGGTGGCTCACGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGTGGATCATGATGTCAGGAGTTCAAGACCAGCCTGGCCAAGATGGTGAAACCCCGTCTCTACTAAAAATCCAAAAATAATTACCTGCGCATGGTGACAGGTGCCTGTAATCCCAGGTACTAGCTGAGAGGCAGAGAATTGCTTGAACTCGGGAGATGGAGGTTGCAGTGAGCTGAGATCACACCACTGCACTCCAGCCTGGGCAACAGAGCGAGACTACGTCTCAAAAAACAAAACAAAACAAAACAAAACAAAAAAGTATAAAAAACAACAACAACAACAAAACTAATGTGAAAATCACTGTCCTATGAAGTTATTAATAGCTTTGAATGACCTCATTATACTTCTTCCAACTGGCAGTTTACTTCCATTGTTAATGCTCATCTCTGTTTGATTACAGTACAACTCTGCTCACCTTTCTACCACCAGCATTCAATTGCTTTTTAAAGAATTGTAAGGAATAGAGCACGAGATCTGGAGTCTGAAGATTTAGAAAAAGTCACACAACTCTCTAAAATTGTTTCCTCATCAATGAAATAGAGGTACCAACACCTGCTTTGTTACATTGCAGGCAATGAATTGGAAAATGCTCAACTAATTATGAAGCACCTACAAATACTGTGATTGTTTTACCCTCCCAAAACTTAAATGGAGCAATTAACGGTGAAAGATATTGGGTTTCATTCAGGATACAACCCTCTTAATAAAAATACGTTTATCTATTTTACATATGGAAATTAAAAGTGCTTACCTTGCTAGATGGAAAATTGAATGAATACAAATGATCGGAATGATAACAGCTAAGTGAATGCTTTAGTTGAAAAATGATACCCCTAAGTCATCAAAGATGTCAAATCATCTCCCTCTCCAAAAAGAGTCACTCTCCACCCCACCCCC       30      .       SVTYPE=INS;SVLEN=1906;LCR=0.97496;REMAP=interspersed    GT:AD   0|1:0,1
chr8    6297862 .       G       GTGTATATGTATGCATACACACATCTCCTATCAGTTCTATCTCTTTGGAGAACCCTAACTAAAACGTGACTTGAAAAACACAATTTAGCAAAAATGATGCAAGAAGAAATAGAAAAGTTGAGGGCTATTATTAATATATGCACTTAAAAAATTTAATTCATAATTTACAATCTTTTCCAGAAGCAAATTGCAGGCCCAGATGATTTCACCATTATATTCTTCCTACTATTTAAGGAAGAAATAACATCAATCTTACATGAAATCTTACAGAACAATTTTAAAAAGGAAAAAGTTGAAATGCCAACTCTTTTGATAACAGTGACATAGCCTGGGCTGGGCGTGGTGGCTCATGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGTGGATCACGAGGTCAAAAGATTGACACTACCCTGGCTAACATGGTGAAACCCCGTCTCTACTACAAATACGAAAAATTAGCCGGGCGTGGTGGCTGGCGCCTGTAGTCCCAGCCACTTGGGAGGCTGAGGCAGGAGAAAGGCGTGAACCCAGGAGGCGGAGCTTGCAGTGAGCCGAGATCGCACCACTGCACTCCAGCCTGGGTGACAGAGTGAGACTCCATCAAAAAAAAAAAAAAAAAAAACCACAAGAAACATAACCTGGATACTAAAACCTAATAAGGACATTGCAATAAAAAGAAATATTACAGGTCAGTAATTCTCATGAACACAGGTGCAAAACAATACATAGAATAATAGTGAATCAAACCTAGTAATATGTAATAATACATAACCATGAAGTTGGTTTTATTCCAGGAATGCAAGATTGAGTTAAGATTAGAAAATTGATCAATGATATTATTCACATTAACAGAAGAAAAGAGAAAAAGAAAATAATCATCAACACAAATGTGCTGCAGGCAGTGGTGGTCGTGACACCCATTCTACAGCTAAAAAATTAGTGTGGGCCAGACGCGGTGGCTCACGCCTGTAATCCCAGCACTTTGGGAGGCCGAGGCGGGTGGATCATGATGTCAGGAGTTCAAGACCAGCCTGGCCAAGATGGTGAAACCCCGTCTCTACTAAAAATCCAAAAATAATTACCTGCGCATGGTGACAGGTGCCTGTAATCCCAGGTACTAGCTGAGAGGCAGAGAATTGCTTGAACTCGGGAGATGGAGGTTGCAGTGAGCTGAGATCACACCACTGCACTCCAGCCTGGGCAACAGAGCGAGACTACGTCTCAAAAAACAAAACAAAACAAAACAAAACAAAAAAGTATAAAAAACAACAACAACAACAAAACTAATGTGAAAATCACTGTCCTATGAAGTTATTAATAGCTTTGAATGACCTCATTATACTTCTTCCAACTGGCAGTTTACTTCCATTGTTAATGCTCATCTCTGTTTGATTACAGTACAACTCTGCTCACCTTTCTACCACCAGCATTCAATTGCTTTTTAAAGAATTGTAAGGAATAGAGCACGAGATCTGGAGTCTGAAGATTTAGAAAAAGTCACACAACTCTCTAAAATTGTTTCCTCATCAATGAAATAGAGGTACCAACACCTGCTTTGTTACACTGCAGGCAATGAATTGGAAAATGCTCAACTAATTATGAAGCACCTACAAATACTGTTATTGTTTTACCCTCCCAAAACTTAAATGGAGCAATTAACGGTGAAAGATATTGGGTTTCATTCAGGATACAACCCTCTTAATAAAAATACGTTTATCTATTTTACATATGGAAATTAAAAGTGCTTACCTTGCTAGATGGAAAATTGAATGAATACAAATGATCGGAATGATAACAGCTAAGTGAATGCTTTAGTTGAAAAATGATACCCCTAAGTCATCAAAGATGTCAAATCATCTCCCTCTCCAAAAAGAGTCACTCTCCACCCCACCCCC   30      .       SVTYPE=INS;SVLEN=1902;LCR=0.975669;REMAP=interspersed   GT:AD   1|0:0,1

And the comparison vcf had three homozygous variants (which is a bit strange):

chr8    6297863 cuteSV.INV.464  G       <INV>   162.2   PASS    PRECISE;SVTYPE=INV;SVLEN=1907;END=6299770;RE=19;STRAND=++;AF=0.9048     GT:DR:DV:PL:GQ  1/1:2:19:162,35,0:34
chr8    6297864 cuteSV.INV.465  G       <INV>   86      PASS    PRECISE;SVTYPE=INV;SVLEN=1906;END=6299770;RE=11;STRAND=++;AF=0.8462     GT:DR:DV:PL:GQ  1/1:2:11:86,14,0:14
chr8    6297864 cuteSV.INV.466  G       <INV>   267.2   PASS    PRECISE;SVTYPE=INV;SVLEN=1906;END=6299770;RE=30;STRAND=--;AF=0.9375     GT:DR:DV:PL:GQ  1/1:2:30:267,63,0:62

The fourth case had two heterozygous variants in the comparisonvcf:

chr20   10807797        cuteSV.INV.312  G       <INV>   99      PASS    PRECISE;SVTYPE=INV;SVLEN=639;END=10808436;RE=20;STRAND=--;AF=0.5556     GT:DR:DV:PL:GQ  0/1:16:20:99,0,61:60
chr20   10807817        cuteSV.INV.313  T       <INV>   92      PASS    PRECISE;SVTYPE=INV;SVLEN=614;END=10808431;RE=19;STRAND=++;AF=0.5429     GT:DR:DV:PL:GQ  0/1:16:19:92,0,63:63

The fifth case also has tho homozygous variants in the comparisonvcf

chrX    6219009 cuteSV.INV.530  A       <INV>   229     PASS    PRECISE;SVTYPE=INV;SVLEN=1336;END=6220345;RE=24;STRAND=++;AF=1  GT:DR:DV:PL:GQ  1/1:0:24:229,61,0:61
chrX    6219010 cuteSV.INV.531  T       <INV>   219.5   PASS    PRECISE;SVTYPE=INV;SVLEN=1341;END=6220351;RE=23;STRAND=--;AF=1  GT:DR:DV:PL:GQ  1/1:0:23:219,59,0:58

[DonFreed]

I have a small secondary question: my comparison vcf includes a few inversions, however my baseline vcf only includes deletions and insertions (and SNVs and Indels), how would this affext the Levenshtein ratio?

hap-eval currently does not handle inversions; variants with a <INV> alt allele are completely skipped. This is why these regions are labeled as FNs.

[DonFreed]

I created a new issue for adding inversion support, #16.