torchcell.data.compute_experiment_reference_index_parallel

compute_experiment_reference_index_parallel(dataset: Dataset, batch_size: int = 10000, io_workers: int = 1) → list[ExperimentReferenceIndex]

Build the reference index (streaming member-index grouping).

Delegates to the sequential streaming build: the sparse member-index representation requires each record’s TRUE dataset index, but the multiprocessing loader’s workers return batches out of order, so consumption-order positions cannot be trusted. The former hash-parallelism gave marginal benefit (the build is LMDB-I/O-bound, not hash-CPU-bound) and silently depended on in-order delivery; a single streaming pass is both correct and sufficient. Signature kept for API compatibility.