torchcell.knowledge_graphs

Builders for the BioCypher knowledge graph and the manifest that governs changes to the served graph. dataset_adapter_map pairs each dataset class with its adapter, and the create_* modules run a BioCypher build over a configured set of datasets. kg_manifest records, per served dataset, the schema fingerprints and adapter code the graph was built under, and decides whether a new dataset can be admitted by incremental import or requires a full rebuild; incremental_import turns one BioCypher output directory into a Neo4j incremental import; releases names and lists served releases. The package resolves its two listed submodules lazily, because each imports every dataset loader and adapter.

`build_time_projection`

Project the tcdb knowledge-graph BUILD (CSV-generation) time for any subset config.

Classes 

`build_time_projection.AdapterTiming`	One adapter's measured CSV-generation timing from a calibration build.
`build_time_projection.CalibrationTimings`	Parsed calibration file (per-adapter timings + the measured total).
`build_time_projection.SubsetConfig`	A KG-build subset config, mirroring the hydra `kg_*.yaml` `subset` block.
`build_time_projection.DatasetSize`	Full (uncapped) record count for one dataset, with its provenance source.
`build_time_projection.AdapterRate`	Per-adapter calibrated generation rate (seconds per record processed).
`build_time_projection.AdapterContribution`	One adapter's projected contribution to a build under some subset config.
`build_time_projection.BuildTimeProjection`	Projected build (CSV-generation) time for a subset config, with a breakdown.

Functions 

`build_time_projection.load_timings`	Load and validate a calibration-timings JSON file (e.g. job 966).
`build_time_projection.dataset_sizes`	Return the committed full record counts as typed `DatasetSize` rows.
`build_time_projection.calibrate`	Fit a per-record rate for each adapter from the calibration build.
`build_time_projection.project_build_time`	Project total build time for `subset_config` from calibrated `rates`.
`build_time_projection.gather_dataset_full_records`	Regenerate full record counts by reading each dataset's LMDB `stat` entries.

`create_kg`

Build a S. cerevisiae BioCypher knowledge graph from configured datasets.

Functions 

create_kg.get_num_workers

Get the number of CPUs allocated by SLURM.

`create_scerevisiae_kg`

Build the S. cerevisiae BioCypher knowledge graph from torchcell datasets.

Functions 

create_scerevisiae_kg.get_num_workers

Get the number of CPUs allocated by SLURM.

`create_scerevisiae_kg_small`

Build a small S. cerevisiae BioCypher knowledge graph from Costanzo/Kuzmin data.

Functions 

create_scerevisiae_kg_small.get_num_workers

Get the number of CPUs allocated by SLURM.

`gene_interactions_scerevisae_kg`

Build the S. cerevisiae gene-interaction BioCypher knowledge graph.

Functions 

gene_interactions_scerevisae_kg.get_num_workers

Get the number of CPUs allocated by SLURM.

`head_ontology`

The sha256-pinned local mirror of BioCypher’s head ontology (Biolink 3.2.1).

Classes 

`head_ontology.HeadOntologyError`	The configured head ontology is remote, missing, or not the pinned bytes.
`head_ontology.HeadOntologyMirror`	Provenance of one mirrored head-ontology file, written beside it as JSON.

Functions 

`head_ontology.record_path`	The provenance JSON that sits beside an ontology file.
`head_ontology.load_mirror_record`	Read the provenance record beside `ontology_path`; a missing record raises.
`head_ontology.configured_head_ontology`	The `biocypher.head_ontology` mapping of a BioCypher config YAML.
`head_ontology.verify_head_ontology`	Check the config's head ontology is a local file with its pinned sha256.
`head_ontology.build_mirror_record`	Record a freshly fetched Biolink file; `check` re-runs the retriever once.

`incremental_import`

Turn one BioCypher output directory into a Neo4j incremental import.

Classes 

`incremental_import.CsvGroup`	One BioCypher label's files: a header plus its `-partNNN.csv` data files.
`incremental_import.ReferenceAnalysis`	What this increment's relationships point at, split by where the nodes live.
`incremental_import.IncrementalImportPlan`	Everything written into a BioCypher output directory to make it incremental.
`incremental_import.ExistingEdgeFilter`	Outcome of dropping relationship rows that the served store already holds.

Functions 

`incremental_import.discover_csv_groups`	Group a BioCypher output directory's files by label, sorted by label.
`incremental_import.incremental_node_header`	Rewrite a BioCypher node header for incremental import.
`incremental_import.write_incremental_headers`	Write `<Label>-header.incremental.csv` beside every node header; return them.
`incremental_import.constraints_cypher`	Cypher creating THE uniqueness constraint keying the global id space.
`incremental_import.analyze_references`	Count nodes/edges and find endpoints that lie outside this increment.
`incremental_import.incremental_import_call`	The `neo4j-admin database import incremental` command for this increment.
`incremental_import.prepare_incremental_import`	Write headers, constraints, the call script and the analysis into `out_dir`.
`incremental_import.query_existing_edges`	Which `(start, end)` pairs already carry a relationship of this group's type.
`incremental_import.filter_existing_edges`	Rewrite every edge part file without the rows `lookup` reports as existing.

`kg_manifest`

The served knowledge graph’s build manifest, and the admission check for adding a dataset to it without rebuilding the others.

Classes 

`kg_manifest.GraphSchemaEntry`	One BioCypher class: what the graph stores for it.
`kg_manifest.SupersetLineage`	The entry a served dataset grew from when it was re-admitted as a superset.
`kg_manifest.KgDatasetEntry`	A dataset as it exists in the served store.
`kg_manifest.KgEvent`	One change to the served store.
`kg_manifest.KgBuildManifest`	What the served knowledge graph was built from, dataset by dataset.
`kg_manifest.ServedDrift`	A served dataset whose schema closure no longer matches the store.
`kg_manifest.AdapterDrift`	Adapter code changes that could move ids or properties of SERVED nodes.
`kg_manifest.SupersetCheck`	Proof that a served dataset's records are all still produced by its dev LMDB.
`kg_manifest.AdmissionReport`	Verdict on adding one dataset to the served store incrementally.
`kg_manifest.BatchAdmissionReport`	Verdict on adding SEVERAL datasets to the served store in ONE increment.

Functions 

`kg_manifest.graph_schema_from_yaml`	Parse `torchcell_schema_config.yaml` into per-class entries.
`kg_manifest.cell_adapter_surface`	`(method fingerprints, conf-name -> method)` of `CellAdapter` from its source.
`kg_manifest.adapter_file_relpaths`	Every per-dataset adapter module and conf yaml in the working tree.
`kg_manifest.dataset_adapter_files`	Repo-relative adapter module + the conf yaml its adapter CLASS binds.
`kg_manifest.dataset_conf_methods`	Conf method names (node + edge) the dataset's adapter enables.
`kg_manifest.loader_relpath`	Repo-relative path of the module that defines `dataset_class`.
`kg_manifest.surface_at_ref`
`kg_manifest.surface_in_worktree`
`kg_manifest.value_surface_from_sources`	`relpath -> sha256` of shared value-file CONTENT.
`kg_manifest.value_surface_in_worktree`	The value surface of the working tree (files that exist).
`kg_manifest.value_surface_at_ref`	The value surface as it was at `ref` (files that existed at that commit).
`kg_manifest.value_surface_drift`	`(changed, added)` between a recorded value surface and the current one.
`kg_manifest.closure_at_ref`	Symbol -> fingerprint for a loader's closure, both taken at `ref`.
`kg_manifest.closure_in_worktree`	Symbol -> fingerprint for a loader's closure, from the working tree.
`kg_manifest.package_version_from_source`	The `__version__` string assigned in `torchcell/__version__.py`'s text.
`kg_manifest.package_version_at_ref`	`torchcell.__version__` as the file read at `ref`.
`kg_manifest.package_tag_at_ref`	The tag pointing at `ref` (`git tag --points-at`), or None when untagged.
`kg_manifest.checkout_package_version`	`(torchcell.__version__, tag)` of the checkout at `repo_root`.
`kg_manifest.load_manifest`	Read a manifest JSON file.
`kg_manifest.save_manifest`	Write the manifest, stamping `updated_at`.
`kg_manifest.bootstrap_manifest`	Reconstruct the manifest of a store built by a FULL build at `commit`.
`kg_manifest.experiment_node_id`	The content-addressed id `CellAdapter._experiment_node` gives an experiment.
`kg_manifest.dev_experiment_ids`	Experiment node ids the dev-tree LMDB of `dataset_class` produces, in order.
`kg_manifest.superset_check`	Compare the ids a store holds for a dataset with the ids its dev LMDB produces.
`kg_manifest.live_experiment_ids`	Experiment node ids under one Dataset node of a running store.
`kg_manifest.adapter_drift_against`	Adapter changes since the build that touch served datasets; plus new methods.
`kg_manifest.check_admission`	Decide whether `dataset_class_name` can be added to the store incrementally.
`kg_manifest.batch_report_from_members`	Aggregate per-dataset admission reports into one batch verdict.
`kg_manifest.check_batch_admission`	Decide whether every named dataset can be added in ONE incremental import.
`kg_manifest.record_admission`	Append an admitted dataset to the manifest (call after the import succeeded).
`kg_manifest.record_batch_admission`	Record every member of an admitted batch under ONE event.
`kg_manifest.live_dataset_counts`	`Dataset.id -> number of ExperimentMemberOf edges` from a running store.
`kg_manifest.live_neo4j_version`
`kg_manifest.format_report`	One dataset's verdict, with the evidence each blocker is decided from.
`kg_manifest.format_batch_report`	The batch verdict, every member's report, and the two cross-member views.
`kg_manifest.load_report`	Read an admission report; a BATCH report is the one carrying `members`.
`kg_manifest.split_dataset_args`	Dataset names from repeated `--dataset` values, each optionally a comma list.
`kg_manifest.parse_n_experiments`	`--n-experiments` values as `dataset -> count`.

`release_snapshot`

Committed snapshots of served knowledge-graph releases.

Classes 

`release_snapshot.SnapshotDataset`	One served dataset as the release snapshot records it.
`release_snapshot.SnapshotEvent`	One manifest event, reduced to what the compatibility page needs.
`release_snapshot.KgReleaseSnapshot`	The committed description of one served release.

Functions 

`release_snapshot.composite_sha256`	sha256 over the sorted `content_sha256` values, newline-joined, trailing newline.
`release_snapshot.snapshot_from_manifest`	The snapshot of a stamped manifest.
`release_snapshot.bootstrap_package_version`	Fill the package version of a snapshot whose manifest predates the spine.
`release_snapshot.pair_package_tag`	The snapshot paired with package `tag`, cut after the build.
`release_snapshot.snapshot_paths`	`(<release>.json, <release>.closures.json)` under `database/releases/`.
`release_snapshot.write_snapshot`	Write the snapshot and its closures companion; returns the two paths.
`release_snapshot.load_snapshot`	Read one snapshot file.
`release_snapshot.load_snapshots`	Every committed snapshot, oldest build first (ties broken by release id).
`release_snapshot.load_closures`	The per-dataset closure fingerprints committed beside `release`'s snapshot.

`releases`

Knowledge-graph releases: what version a served store is, which datasets it holds, and whether a dataset’s bytes changed between two versions.

Classes 

`releases.ReleaseDataset`	One dataset as one release serves it.
`releases.KgRelease`	The self-description a served store carries as its `KgRelease` node.
`releases.ServedDatabase`	One database of a DBMS as `SHOW DATABASES` and its release node describe it.
`releases.ReleaseDiff`	Datasets by what happened to their bytes between two releases.
`releases.DatasetDrift`	A served dataset the local code would serialize under a different contract.
`releases.ReleaseCompatibility`	Whether the local schema surface matches the contracts a release was built under.
`releases.IncompatibleReleaseError`	A served release and the installed package are not a pair.

Functions 

`releases.release_id`	`<YYYY.MM.DD>-<commit[:8]>`: the build date and the generation commit.
`releases.next_version`	`major.minor` after a full build (major + 1, minor 0) or an admission (minor + 1).
`releases.content_sha256`	sha256 of the sorted, newline-joined ids (trailing newline).
`releases.content_hashes_from_csv`	`Dataset.id -> content_sha256` from a BioCypher output directory.
`releases.content_hashes_from_store`	`Dataset.id -> content_sha256` streamed from a running store, one dataset at a time.
`releases.read_release`	The store's release node, or None when the store predates release nodes.
`releases.write_release`	Write (or replace) the store's single release node; the database must be writable.
`releases.release_from_manifest`	The release record a manifest describes, with the measured content hashes.
`releases.stamp_manifest`	Give a manifest its version, release id, package version, and content hashes.
`releases.list_databases`	Every user database of the DBMS at `uri` with its aliases, release, and health.
`releases.list_databases_bounded`	`list_databases()` with a wall-clock bound; a hung host yields one row.
`releases.resolve_database`	The database name to open for `version`.
`releases.datasets`	The datasets a version serves, with record counts and content hashes.
`releases.diff`	Datasets unchanged (byte-identical), changed, added, and removed from `a` to `b`.
`releases.compatibility`	Which served datasets the checkout at `repo_root` would serialize differently.
`releases.compatibility_with_surface`	`compatibility()` against an already-loaded schema surface.
`releases.closure_compatibility`	The compatibility verdict from a release's closures and any schema surface.
`releases.require_paired`	The compatibility report, or `IncompatibleReleaseError`.
`releases.package_checkout`	The checkout this `torchcell` was imported from (the parent of the package).
`releases.commit_index`	Position of `sha` on linear main, `(off-main)`, or `(unknown)`.
`releases.commit_date`	Committer date of `sha` as `YYYY.MM.DD`.
`releases.behind_main`	How far a served commit lags local main.
`releases.package_label`	The PKG cell: the tag when the build ran from one, else the version marked untagged.
`releases.status_rows`	Table rows for one host: HOST DATABASE VERSION RELEASE PKG COMMIT# DATE DATASETS ORGANISMS NODES ALIASES STATUS.
`releases.format_table`	Fixed-width table with the header row.

`subset.load_gene_sets`	Load gene sets from a text file: one set per line, whitespace-separated genes.
`subset.select_indices`	Return the indices of records satisfying `record_filter`, in one LMDB pass.
`subset.subset_dataset`	Return the dataset narrowed to a prefiltered pool, then randomly sampled.

`supported_queries.check`

The supported-query drift check: does each registered query still hold on a release?

Classes 

`supported_queries.check.QueryDrift`	One way a registered query no longer holds on a release.
`supported_queries.check.QueryResult`	One query's verdict, with the issue the CI job files when it fails.
`supported_queries.check.CheckReport`	The check over a whole registry against one release.
`supported_queries.check.GraphLabels`	Node labels and relationship types a release carries, with node properties.

Functions 

`supported_queries.check.pascal_label`	BioCypher's label for a schema class: `fitness phenotype` -> `FitnessPhenotype`.
`supported_queries.check.schema_is_a`	`class -> [parent, ...]` from the `is_a` entries of a BioCypher schema config.
`supported_queries.check.graph_labels`	The labels and types of a release's graph schema under the `is_a` hierarchy.
`supported_queries.check.selected_composite`	`composite_sha256` over the selected datasets the release serves.
`supported_queries.check.query_drifts`	Every drift of `query` against `snapshot`; `deps` None means no file.
`supported_queries.check.run_check`	Check every registered query; exit code 1 when a supported query drifts.
`supported_queries.check.checkout_commit`	`git rev-parse HEAD` of the checkout at `repo_root`.
`supported_queries.check.resolve_snapshot`	The named release's snapshot, or the newest committed one.
`supported_queries.check.check_repo`	`run_check()` with every input read from the checkout at `repo_root`.
`supported_queries.check.validate_query`	Record `release` as the one `query_id` holds on.

`supported_queries.cypher_deps`

What a Cypher query reads from the graph, extracted from its text.

Functions 

`supported_queries.cypher_deps.strip_comments`	The query with `//` and `/* */` comments replaced by spaces (offsets kept).
`supported_queries.cypher_deps.blank_literals`	`strip_comments()`, plus every string literal's contents replaced by spaces.
`supported_queries.cypher_deps.split_blocks`	`(start, end)` offsets of the `UNION`-separated blocks of blanked `text`.
`supported_queries.cypher_deps.extract_dependencies`	The labels, types, properties, literals and parameters `cypher` depends on.

`supported_queries.registry`

The supported-query registry: which shipped .cql queries the project stands behind.

Classes 

`supported_queries.registry.QueryDependencies`	What a query reads from the graph, as extracted from its Cypher text.
`supported_queries.registry.SupportedQuery`	One registered query and where it stands in the release lifecycle.
`supported_queries.registry.QueryRegistry`	Every registered query; `schema_version` versions this file's shape.

Functions 

supported_queries.registry.registry_path

registry.json in the checkout at repo_root.

Not documented 

Submodules left out of this page:

conf (configuration files only)