torchcell.data.ExperimentDataset
- class ExperimentDataset(root: str, io_workers: int = 0, transform: Callable[[...], Any] | None = None, pre_transform: Callable[[...], Any] | None = None, skip_process_file_exist: bool = False)[source]
-
Abstract PyG dataset storing experiment items in an LMDB store.
- visibility: ClassVar[Visibility] = 'public'
Whether this dataset may be served publicly or published in a release. A ClassVar, not a field: it is a property of the LOADER, decided once in code, and the build and release gates read it off the class without instantiating it.
- organism: ClassVar[Organism]
each loader declares its own, the build’s organism switch reads it off the class (
dataset_organism), and a loader that forgot it is refused by name instead of inheriting an organism it may not have.- Type:
The host organism every record describes. Annotated here WITHOUT a value
- has_gene_perturbations: ClassVar[bool] = True
Whether the records carry gene-keyed perturbations. True for every screen of edited strains, where an empty gene set after a build is a broken build and the
gene_setsetter refuses it. A dataset whose every record is the unedited host in a varied environment (the in-house Bioscreen runs,Genotype(perturbations=[])on a typed background) declares False here, in code, and its gene set is legitimately empty: the edits that define the strain live on the reference’s background.
- abstract property experiment_class: type[Experiment]
Return the Experiment subclass produced by this dataset.
- abstract property reference_class: type[ExperimentReference]
Return the ExperimentReference subclass produced by this dataset.
- abstract property raw_file_names: list[str]
Return the raw file names expected under the raw directory.
- property processed_file_names: str | list[str]
Return the processed artifact name (the LMDB directory).
- abstractmethod process() None[source]
Build the processed LMDB store from raw data (implemented by subclasses).
- abstractmethod download() None[source]
Download raw data into the raw directory (implemented by subclasses).
- abstractmethod preprocess_raw(df: DataFrame, preprocess: dict[str, Any] | None = None) DataFrame[source]
Clean/normalize the raw DataFrame (implemented by subclasses).
- abstractmethod create_experiment() None[source]
Construct experiment records from preprocessed data (subclasses).
- get(idx: int | list[int] | ndarray[Any, Any]) Any[source]
Return one item, or a list of items for list/array/boolean indices.
- get_single_item(idx: int) Any[source]
Return the deserialized item at
idx, or None if absent.Resolves any interned
$refpointers back to the full sub-objects, so the returned three-key dict is identical to the pre-interning format (a no-op for legacy inline stores).
- static extract_systematic_gene_names(genotype: dict[str, Any]) list[str][source]
Return the systematic gene names of all perturbations in a genotype.
A SITE-keyed called variant (
bacterial_site_variant, issue #731) is left out: its identifier is the derived<replicon>:<position>, and the whole point of that leaf is that no locus of the assembly holds the call, so putting it in the gene set would create a gene node for a place that is not a gene. The call’s flanking loci, where the release names them, ARE loci and are collected, so an intergenic variant still reaches the genes it sits between.
- compute_gene_set_sequential() GeneSet[source]
Compute the gene set by scanning every LMDB entry in order.
The genotype is resolved through the sibling
internedenv, so a store that interns a genotype shared by many records (a natural isolate’s perturbation set, constant across its environments) reads the same as an inline one.
- compute_gene_set_parallel(batch_size: int = 10000, io_workers: int = 1) GeneSet[source]
Compute the gene set using a multiprocessing batch loader.
- property experiment_reference_index: list[ExperimentReferenceIndex] | None
Load or compute (and cache) the experiment reference index.