BIO ML

Carbon-3B: Measuring 6-mer Token Phase Sensitivity

Ilho Ahn

May 23, 2026

A test of how much 6-mer token phase changes Carbon-3B scores relative to score IQR across 500 BRCA2 MAVE SNVs, and how the corresponding FNS pipeline differs.

Figure 1. Normalized phase ranges across scoring conditions for 500 SNVs sampled from the BRCA2 MAVE. Each range is divided by its own scorer’s pooled score IQR; bar heights therefore compare relative sensitivity on different denominators. The dashed line marks the prespecified 0.10 reference threshold. All three token conditions exceeded it; both FNS conditions were lower than their corresponding token conditions but remained above the threshold. Open full-size figure.

Ben Allal et al. present Carbon’s non-overlapping 6-mer tokenization as an efficiency trade-off: encoding six nucleotides per token expands the nucleotide context covered by a fixed token budget, and the authors report that this scheme worked better than BPE for DNA. FNS is presented as the bridge from this coarse representation to single-nucleotide supervision and scoring [1] [2].

Because Carbon also reports training-free VEP (variant effect prediction) results on BRCA2, phase stability is a practical complementary question: does the same SNV receive a stable reference-minus-alternative score across the six possible token offsets? This note tests that scoring-protocol question in 500 BRCA2 MAVE SNVs and compares the corresponding token and FNS pipelines; it does not re-evaluate overall Carbon VEP performance.

6-mer phase is the offset occupied by the variant base within a 6 bp token. Here, the same SNV is scored at all six phases.

MAVE stands for multiplexed assay of variant effect. The BRCA2 resource from Huang et al. measures the functional effects of many variants in parallel.

FNS stands for Factorised Nucleotide Supervision, Carbon’s mechanism for connecting coarse 6-mer modeling to position-wise nucleotide supervision and scoring.

Primary resources

Carbon-3B HuggingFaceBio/Carbon-3B Carbon evaluation README huggingface/carbon evaluation BRCA2 MAVE Huang et al. Nature 2025

Summary

• The evaluation covers six token offsets for 500 BRCA2 MAVE SNVs: 85 LOF (loss-of-function) and 415 FUNC (functional)/INT (intermediate). Phase range is normalized by scorer-specific score IQR, with 0.10 as the prespecified reference threshold.
• Median normalized ranges were 0.409 for Full-window, 0.349 for Target-token, and 0.466 for Downstream-only; 93.6–100% of variants met or exceeded 0.10.
• Target-token is the most direct measure of sensitivity in scoring the variant-containing token. Full-window and Downstream-only also include global segmentation and edge/context changes; the shifted-reference control prevents a purely local attribution.
• FNS normalized ranges were lower at 0.353 for Full-sequence and 0.331 for Local-target. Median paired differences were 0.0544 and 0.0198, but both FNS conditions remained above 0.10. Scorer code and IQR denominators differ, so this is a pipeline comparison.
• Six-phase mean-score AUROC was 0.913 for both full-sequence conditions after rounding. This confirms retained label-direction signal in the subset; phase-specific AUROC and ranking changes are not reported here.

Evaluation setup

The data are an SNV subset reconstructed from the Huang et al. BRCA2 MAVE resource against the hg19 chr13 reference [4] [5] [6]. Of 6,836 source variants that passed the reference-allele match and SNV filters, 500 were sampled with stratification and seed=20260523. Strata covered label, functional-score quantile, genomic-position decile, reference/alternative base, and genomic position modulo 6. The 100-variant protocol pilot is included in this subset.

The six-phase protocol shifts the genomic window around the same SNV so that the variant occupies each possible offset within a 6-mer. Each phase contains a matched reference/alternative sequence pair. Moving the window also changes edge bases and the segmentation of the surrounding sequence; this is not an intervention on the variant-containing token alone.

The primary window is 8,190 bp. Its length is divisible by six, so it creates no tail padding in the Carbon tokenizer. An 8,192 bp window matching the Carbon BRCA2 evaluation was checked separately [3] [7] and was not mixed into the primary phase-score result.

Item	Fixed value
Model	HuggingFaceBio/Carbon-3B token fe755cb5; FNS revision bf6f6bec
Primary data	500-SNV subset of the BRCA2 MAVE LOF 85, FUNC/INT 415; seed 20260523
Primary window	8,190 bp 6-mer clean window; no partial-token tail
Primary score	Reference score minus alternative score
Reference threshold	Median phase range / score IQR ≥ 0.10 10% of score IQR as a scale-normalized effect-size threshold

Table 1. Evaluation scope for testing scoring-protocol phase sensitivity in the 500-SNV BRCA2 MAVE subset.

Within Table 1’s scope, each variant score is the reference aggregate minus the alternative aggregate. A positive value favors the reference sequence under that scorer. The token conditions differ in which log-probabilities enter the aggregate:

• Full-window averages log-probability scores over all valid DNA tokens in the 8,190 bp window.
• Target-token uses only the single 6-mer token containing the variant.
• Downstream-only uses only tokens after the variant, measuring how the changed variant context affects subsequent token predictions in the autoregressive model.

FNS uses nucleotide-resolution scores: Full-sequence aggregates across the sequence, while Local-target restricts the score to the variant position. These are also on different raw scales from token scores. Raw ranges are therefore not compared across scoring methods. For each variant, the numerator of the normalized phase range is the difference between the maximum and minimum scores across six phases. The denominator is one IQR calculated over all score_ref_minus_alt values for the same scorer across 500 variants × 6 phases. Reported values are the medians of the 500 variant-level normalized ranges. Each scorer thus has one fixed denominator for all variants, but the denominator changes between scorers. A lower normalized range can reflect changes in either the phase range or the scorer’s overall spread; it is not an absolute reduction on a shared raw scale.

The 0.10 reference threshold was prespecified during experiment design as 10% of score IQR. It is a descriptive effect-size reference, not a statistical significance threshold or a validated cutoff for biological harm.

Results

The primary result is phase sensitivity in token scores.

Token phase sensitivity

Across the 500-SNV BRCA2 subset, normalized token-score phase ranges were 0.409 for Full-window, 0.349 for Target-token, and 0.466 for Downstream-only. These values were 3.5–4.7 times the prespecified 0.10 reference threshold, and 93.6–100% of variants met or exceeded it in each condition. Within this subset, phase sensitivity was distributed across variants rather than confined to a few outliers.

Target-token provides the most direct measure of sensitivity in scoring the variant-containing token; scorer-specific normalization is required for comparisons with the other token conditions.

Token score condition	Median phase range	P90 phase range	Median range / score IQR	Variants at or above 0.10
Full-window	0.00634	0.01801	0.409	99.6%
Target-token	1.87500	4.12500	0.349	93.6%
Downstream-only	0.01206	0.03633	0.466	100.0%

Table 2. Phase sensitivity by token-score condition. Because raw phase ranges use different score scales, the primary comparison is the median variant-level phase range divided by the scorer-specific overall score IQR.

Figure 2 expands the Table 2 distribution by variant to test whether the observation is driven by outliers. It shows the magnitude of per-variant phase sensitivity, not a direction associated with any phase. Each y-value is the range across a variant’s six phase scores divided by the Full-window score IQR. The median was 0.41, the p90 was 1.16, and 498 of 500 variants met or exceeded the 0.10 threshold.

Figure 2. Each point represents one variant. The y-axis shows its six-phase score range divided by the Full-window score IQR on a log scale, and the x-axis ranks variants from the smallest to largest range. The dashed line marks 0.10; 498/500 variants were at or above it.

Padding and shifted-reference control

An 8,190 bp sequence is divisible by six and creates no partial-token tail. The 8,192 bp window matching the Carbon BRCA2 evaluation creates a length-2 tail. Primary phase scores therefore use the 8,190 bp clean-window condition; the 8,192 bp condition is kept as a separate compatibility check.

Check	Condition	Observation
Clean window	8,190 bp	0/3,000 partial-token tails
Carbon BRCA2 compatibility window	8,192 bp	3,000/3,000 length-2 tails

Table 3. Padding checks. The 8,190 bp window is the primary condition without a partial-token tail; 8,192 bp is used only for compatibility with the Carbon BRCA2 evaluation.

Even after separating padding conditions, the six shifted windows change both edge bases and global 6-mer segmentation. The Full-window range therefore cannot be attributed to a purely local token-boundary effect.

For the shifted-reference control, reference windows without the alternative allele were moved through the same six-shift protocol. The median diagnostic ratio of this control range to the reference-minus-alternative Full-window phase range was 4.162, and the control range was at least as large as the Full-window phase range for 93.6% of variants. Because numerator and denominator measure different score quantities, this ratio is not an estimate of edge contribution.

Figure 3. Distribution of shifted-reference control ranges. The x-axis is the six-shift range in reference full-window likelihood and the y-axis is variant count. The control range was at least as large as the reference-minus-alternative Full-window phase range for 93.6% of variants; the median diagnostic ratio between the two ranges was 4.162.

Target-token and Full-window should therefore be interpreted separately. Target-token indicates sensitivity of the token-containing-variant score to the six-shift protocol. Full-window is a protocol-level score that does not separate local tokenization, global segmentation, and window edge/context changes.

Paired differences in the FNS pipeline

Within these bounds, FNS base-level scoring was compared with the corresponding token conditions. The normalized range was 0.353 for FNS Full-sequence versus 0.409 for token Full-window, and 0.331 for FNS Local-target versus 0.349 for token Target-token. Both FNS values exceeded the 0.10 threshold.

Figure 4 places both pairs in dimensionless range/IQR units, using a separate IQR for each scorer.

Figure 4. Median normalized phase ranges for FNS and token scores, each divided by its own scorer-specific IQR. FNS was lower in both pairs, but both FNS values remained above 0.10. Variant-level paired differences and bootstrap confidence intervals are reported in Table 4.

Comparison	Token range / IQR	FNS range / IQR	Token - FNS	95% CI	Variants with lower FNS
Full-sequence FNS vs token	0.409	0.353	0.0544	[0.0362, 0.0709]	64%
Local-target FNS vs token	0.349	0.331	0.0198	[0.0118, 0.0294]	58%

Table 4. Paired normalized-range differences in the FNS pipeline. Token - FNS is the median per-variant paired difference, not the difference between displayed medians. The final column reports the proportion of variants with lower FNS values. Post-hoc CIs use 10,000 variant bootstrap resamples with fixed scorer-specific IQR denominators (seed=20260523).

The token and FNS revisions use the same safetensor weight blob, but their scorer configuration and modeling/tokenizer code differ. Table 4 therefore reports an observed pipeline-level difference in normalized range, not the causal effect of the FNS objective alone.

Label-direction signal

The results above establish sensitivity to the six-shift scoring protocol. As a separate sanity check, LOF was treated as the positive class and AUROC was used to test whether Carbon scores aligned with the BRCA2 MAVE label direction.

Scores were first averaged across six phases for each variant. On the 500-SNV subset, this mean-score AUROC was 0.913 for both Full-window token and Full-sequence FNS after rounding to three decimals. Both exceeded the p95 of scorer-specific null distributions created by 500 label permutations, indicating that label-direction signal remained in the phase-sensitive scores.

Score	Observed AUROC	Shuffled null p95	Interpretation
Full-window token Six-phase mean score	0.913	0.553	Signal separating LOF from FUNC/INT remained.
Full-sequence FNS Six-phase mean score	0.913	0.559	AUROC was nearly identical to Full-window token.
Local-target FNS Six-phase mean score	0.760	0.559	This local score has a different role from the full-sequence conditions.

Table 5. Observed AUROC was calculated after averaging the six phase scores for each variant. Shuffled null p95 is the 95th percentile of 500 label-permuted AUROCs generated with seed 20260523.

Interpretation and conclusion

Across the 500-SNV BRCA2 subset, six-shift sensitivity in Carbon-3B token scores was present for most variants rather than a few outliers. Target-token is the most direct measure here of sensitivity in scoring the token containing the variant. Full-window and Downstream-only additionally include changes to global segmentation and window edge/context.

Under this protocol, the nucleotide-resolution FNS pipeline showed lower normalized ranges—from 0.409 to 0.353 for the full-sequence pair and from 0.349 to 0.331 for the local-target pair—but did not eliminate the observed six-shift sensitivity; both FNS conditions remained above 0.10. Mean-score AUROC showed that label-direction signal remained, but it does not alter the phase-sensitivity finding or establish overall Carbon VEP performance.

For Carbon SNV scoring under similar conditions, reporting a six-phase summary and range is therefore more appropriate than reporting a single reference-phase score. The mean used here is one possible summary, not the result of an aggregation-method comparison. Storing only one value makes allele-associated score differences difficult to separate from variation introduced by window and tokenization choices.

Three reporting rules follow for this setting:

1. Report the six-phase range or an instability flag alongside the reference-minus-alternative score.
2. Where FNS is available, compare token and FNS scores with paired normalized metrics and report the result as an observed pipeline-level difference.
3. Keep the 8,190 bp clean-window score separate from the 8,192 bp compatibility check with padding. If 8,192 bp is used, report the padding-tail check and its scores separately.

Limitations

• Results are limited to 500 SNVs from the BRCA2 MAVE and an 8,190 bp window. The 100-variant protocol pilot is included in the subset, although normalized ranges for all three token scorers also exceeded 0.10 in the remaining 400 variants.
• Bootstrap confidence intervals were not calculated for the primary phase effects. The values 0.409, 0.349, and 0.466 are observations above the prespecified effect-size threshold in this subset; generalization to other genes or window conditions requires further evaluation.
• The shifted-reference control changes both edge context and global 6-mer segmentation phase. It does not separate a pure edge effect, a local token-boundary effect, or their causal contributions.
• Table 5 AUROCs exceeded the shuffled-label null, but no simple sequence/position baseline, reverse-complement evaluation, or external VEP benchmark was included. These results do not establish overall Carbon VEP performance or improved biological performance from FNS. Phase-specific AUROC, rank correlations across phases, and candidate-selection changes are not reported here, so score sensitivity cannot be translated into those performance effects.
• The paired FNS confidence intervals are from a post-hoc audit rather than the prespecified primary analysis. Differences in scorer code and normalization scale between token and FNS revisions prevent isolation of the FNS objective’s effect.

Appendix: Reproduction conditions

Computation	Role	Items	Runtime	Peak CUDA memory
Token scoring	Token-based variant scores	12,000 scoring items 3,000 window checks; 0 errors	1,192.93 s	14.40 GB
FNS scoring	FNS-based scores	6,000 scoring items 6,000 paired-comparison items; 0 errors	1,149.77 s	8.33 GB
Auxiliary checks	Padding checks	6,000 padding-check items	13.21 s	N/A

Appendix Table 1. Reproduction conditions by computation. Model revisions, input hashes, sampling seed, and score artifacts are preserved in the separate experiment record.

References

1. Ben Allal, L., Li, Q., Fiusco, M. et al. Carbon: Decoding the Language of Life. bioRxiv, 2026.05.22.727119, 2026. DOI: 10.64898/2026.05.22.727119
2. Hugging Face Biology Research. Carbon-3B model card and pinned scorer revisions. Hugging Face, 2026. Model card; token revision fe755cb5; FNS revision bf6f6bec. Accessed 2026-07-30.
3. Hugging Face Biology Research. Carbon evaluation README. GitHub, 2026. pinned evaluation README. Accessed 2026-05-23.
4. Huang, H., Hu, C., Na, J. et al. Functional evaluation and clinical classification of BRCA2 variants. Nature 638, 528-537, 2025. DOI: 10.1038/s41586-024-08388-8
5. Huang et al. Functional evaluation and clinical classification of BRCA2 variants, Supplementary Table 3. Nature/Springer, 2025. Supplementary Table S3 XLSX
6. UCSC Genome Browser. hg19/GRCh37 chr13 chromosome FASTA, goldenPath. chr13.fa.gz. Accessed 2026-05-23.
7. Hugging Face Biology Research. Carbon BRCA2 data preparation script. pinned prep_brca2.py. Source content frozen with the experiment inputs; accessed 2026-05-23.

Appendix Table 1 summarizes the computations. The public source links identify the model and input resources; the summary tables do not provide the complete per-variant score artifacts needed to independently recompute the reported statistics.

Citation

Text citation:

Ilho Ahn, "Carbon-3B: Measuring 6-mer Token Phase Sensitivity", Mini Research, May 23, 2026.

BibTeX:

@misc{ahn2026carbon_6mer_phase_sensitivity,
  author = {Ahn, Ilho},
  title = {Carbon-3B: Measuring {6-mer} Token Phase Sensitivity},
  year = {2026},
  month = {May},
  howpublished = {Mini Research},
  url = {https://muted-color.github.io/research/2026/05/23/carbon-6mer-phase-sensitivity/}
}