torchcell.data.ExperimentReferenceIndex

class ExperimentReferenceIndex(*, reference: ExperimentReference | FitnessExperimentReference | GeneInteractionExperimentReference | GeneEssentialityExperimentReference | SyntheticLethalityExperimentReference | SyntheticRescueExperimentReference | CalMorphExperimentReference | MicroarrayExpressionExperimentReference | MicroarrayExpressionScoreExperimentReference | MicroarrayExpressionRatioExperimentReference | InductionTimecourseExpressionExperimentReference | RNASeqExpressionExperimentReference | PseudobulkExpressionExperimentReference | VisualScoreExperimentReference | MetaboliteExperimentReference | ProteinAbundanceExperimentReference | ProteinFoldChangeExperimentReference | EnvironmentResponseExperimentReference | StrainEnvironmentResponseExperimentReference | SegregantGrowthExperimentReference | SegregantPerturbationExperimentReference | ProductTiterExperimentReference | ProteinTurnoverExperimentReference | ProteinSynthesisRateExperimentReference | FluxExperimentReference | PromoterActivityExperimentReference | PooledSortSeqExpressionExperimentReference | BiosensorFluorescenceExperimentReference | GradedGeneEssentialityExperimentReference | PhosphositeIntensityExperimentReference | MorphologyExperimentReference | MrnaNumberFractionExperimentReference | FitnessContrastExperimentReference | SegregantExpressionExperimentReference | LifeCycleExperimentReference | PseudobulkUmiCountExperimentReference | GenomeArchitectureExperimentReference, member_indices: list[int])

Bases: ModelStrict

A reference paired with the sorted record indices of the experiments using it.

SPARSE by construction (WS15 streaming redesign): every experiment belongs to exactly ONE reference, so member_indices across a dataset’s references sum to N – NOT the former dense index: list[bool] mask, which stored a length-N boolean per reference and was therefore O(references x N). That blowup (~60 GB at Hoepfner’s ~30M records x ~1.8k references) made genome-scale datasets infeasible to build or store; the sparse form is O(N) total for any dataset size, small or large, with ONE representation and ONE (always-streaming) build path.

mask(n) reconstructs the dense length-n boolean mask ON DEMAND for consumers that want it (small datasets / vectorized code). Genome-scale consumers iterate member_indices directly and never materialize the dense mask.

classmethod from_stored(item: dict[str, Any]) → ExperimentReferenceIndex

Build from a stored JSON entry, accepting BOTH the new and legacy formats.

New entries carry member_indices; legacy entries (written before the WS15 sparse redesign) carry a dense index: list[bool] mask. Converting on read keeps every already-built dataset’s experiment_reference_index.json loadable without a forced rebuild; the next rebuild writes the new sparse format.

mask(n: int) → list[bool]

Reconstruct the dense length-n boolean membership mask (opt-in, small sets).

combine(other: ExperimentReferenceIndex) → ExperimentReferenceIndex

Return a new index UNION-ing two member-index sets that share a reference.

References match by reference_key, the same key ReferenceIndex refuses duplicates on.

model_config: ClassVar[ConfigDict] = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].