torchcell.data

The data layer between the knowledge graph and model training. ExperimentDataset is the abstract LMDB-backed base for experiment datasets, Neo4jCellDataset queries Neo4j, caches the raw records in LMDB and turns each experiment into a perturbed cell graph, and the graph processors (for example SubgraphRepresentation and LazySubgraphRepresentation) define how a genotype perturbation is applied to that graph. Deduplicators and aggregators merge repeated measurements of the same genotype, and the reference-index helpers group experiments by their shared reference state.

Classes

ExperimentReferenceIndex	A reference paired with the sorted record indices of the experiments using it.
ReferenceIndex	A collection of reference-indices partitioning a dataset's experiments.
Deduplicator	Base class that deduplicates experiment records into an output LMDB store.
MeanExperimentDeduplicator	Deduplicate experiments by averaging duplicates of the same genotype.
Aggregator	Abstract base that groups experiment pairs into an aggregated LMDB store.
DeletionKeyedGenotypeAggregator	Aggregator keyed on the DELETION gene set only; other axes are background.
GenotypeAggregator	Aggregator keyed by a hash of the experiment's full perturbation set.
ExperimentDataset	Abstract PyG dataset storing experiment items in an LMDB store.
Neo4jCellDataset	Dataset for loading cell data from Neo4j with file locking for DDP support.
RawSha256MismatchError	A file's bytes do not match the sha256 pin its loader records.
ManifestPinMismatchError	A raw-mirror manifest records a sha256 other than its loader's module pin.
SubgraphRepresentation	Processor that removes perturbed genes and returns the induced subgraph.
LazySubgraphRepresentation	Graph processor that returns full graph with masks instead of filtered subgraphs.
IncidenceSubgraphRepresentation	Graph processor using precomputed node-to-edge incidence structures.
Perturbation	Store only perturbation-specific information without duplicating the graph.
DCellGraphProcessor	Graph processor for DCell model that applies perturbations directly to the gene ontology graph.
Unperturbed	Preserve the original graph, storing perturbation and phenotype data alongside.

Functions

compute_sha256_hash	Compute the sha256 hash of a string.
compute_experiment_reference_index_sequential	Build the reference index in ONE streaming grouping pass (memory-bounded).
compute_experiment_reference_index_parallel	Build the reference index (streaming member-index grouping).
post_process	Decorate process to rebuild the gene set/reference index and validate.
check_manifest_pin	Raise ManifestPinMismatchError unless the manifest's recorded is pin.
file_sha256	Hex sha256 of a file's bytes (symlinks are followed), read in chunks.
verify_sha256	Raise RawSha256MismatchError unless path hashes to expected.
verify_raw_files	Verify every pinned file in raw_dir ({file name: sha256}).
copy_verified	Copy src to dest after verifying src against expected.
write_verified	Write data to dest only if it hashes to expected.
link_verified	Symlink dest to src after verifying src; an existing link is kept.