Example data#

larmor.data downloads public datasets for trying Larmor and for the examples. Each fetch_ function downloads a dataset the first time it is called, checks every file against its SHA-256 checksum, and returns the paths of the files; later calls return the cached copy at once.

from larmor.data import fetch_ismrm2015
from larmor.io import load_dwi

files = fetch_ismrm2015()
dwi, gtab = load_dwi(files["dwi"], files["bvals"], files["bvecs"])

Datasets#

Function	Contents	Size	Licence
`fetch_ismrm2015`	The ISMRM 2015 Tractography Challenge data, simulated from bundles traced in a real brain: DWI at 2 mm with one b=0 image and 32 directions at b=1000 s/mm^2, with noise and artefacts, and a T1-like image at 1 mm	41 MB	CC BY 4.0
`fetch_cfin_multishell`	One healthy volunteer at 3 T: DWI at 2.5 mm with one b=0 image and 33 directions at each of 15 b-values from 200 to 3000 s/mm^2 (raw), and a defaced 1 mm T1	197 MB	CC0 1.0
`fetch_stanford_hardi`	One healthy volunteer at 3 T: DWI at 2 mm with 10 b=0 images and 150 directions at b=2000 s/mm^2 (motion corrected), a T1 and a FreeSurfer parcellation on the same grid	93 MB	PDDL 1.0

The datasets store their b-vectors differently. CFIN’s are in the format of FSL bvecs files, load_dwi’s default, so they are read as in the code above, and so are the ISMRM 2015 data’s, after one correction. The published ISMRM 2015 b-vectors have the wrong sign on their x components, as the challenge’s data page notes, so fetch_ismrm2015 changes that sign when it extracts the file: the dwi.bvecs in the cache differs from the Zenodo original in its first row, and is right for any program that reads FSL bvecs files. Read this way, the tensors’ principal directions follow the challenge’s ground-truth bundles; without the correction the fibres are mirrored top to bottom. The corrected file is checked by its SHA-256, so a cache written before the correction is downloaded and corrected again, once. The other files are as published, renamed. Stanford’s are in world coordinates, so they are read with frame="world": load_dwi(files["dwi"], files["bvals"], files["bvecs"], frame="world"). Read as FSL b-vectors they would be mirrored left to right. The direction of the fibres beside the corpus callosum shows which frame is right.

Please cite the dataset you use:

ISMRM 2015 Tractography Challenge: Maier-Hein KH, Neher PF, Houde J-C, et al. The challenge of mapping the human connectome based on diffusion tractography. Nature Communications 8 (2017) 1349. doi:10.1038/s41467-017-01285-x. The data is published on Zenodo, doi:10.5281/zenodo.572345, under CC BY 4.0. The website’s brain images are made from it.
CFIN multi-shell: Hansen B, Jespersen SN. Data for evaluation of fast kurtosis strategies, b-value optimization and exploration of diffusion MRI contrast. Scientific Data 3 (2016) 160072. doi:10.1038/sdata.2016.72. The data is published on Dryad, doi:10.5061/dryad.9bc43; Larmor downloads the copy in the University of Washington’s ResearchWorks archive (hdl:1773/38488), because Dryad’s downloads need a browser.
Stanford HARDI: Rokem A, Yeatman J, Pestilli F, Wandell B. High angular resolution diffusion MRI. Stanford Digital Repository (2013). purl.stanford.edu/yx282xq2090. The repository asks that the data never be used to identify the volunteer.

The licence and citation are also logged (at the INFO level of the larmor logger) when a dataset is downloaded, and saved in a README.txt next to its files.

The cache folder#

The datasets go into the folder in the LARMOR_DATA_DIR environment variable, or else into Larmor’s folder in the user’s cache: ~/Library/Caches/larmor on macOS, ~/.cache/larmor on Linux (or under $XDG_CACHE_HOME) and %LOCALAPPDATA%\larmor\Cache on Windows. larmor.data.data_folder() returns it, and each function also takes a folder. Deleting the folder removes the data; it is downloaded again when needed.

A download that is interrupted starts again from the beginning, the next time or, after a dropped connection, at once (up to three times). A file is moved into the cache only when complete and its checksum matches, so the cache never holds a partial or wrong file.

No dataset offers a tractogram yet: a whole-brain tractogram can be made from any of them with Larmor’s own tracking.

For contributors#

An example that uses larmor.data carries the line # needs: example-data; the tests run it only when LARMOR_DATA_DIR is set, so a plain pytest run downloads nothing. CI sets it to a folder it keeps between runs, keyed on src/larmor/data/files.py, the table of URLs, checksums and sizes. A new dataset needs a licence that allows redistribution and use in documentation (CC0, CC-BY, PDDL or similar), its exact licence and citation, and a stable host from which a script can download it.