Finding a Dataset

emdatabase ships an index of every dataset it knows about, and three functions for querying it: list_datasets(), search() and filter(). All three return dataset objects, so anything they hand back can be downloaded directly.

Nothing here touches the network - the index is metadata that ships with the package, and querying it never opens a file.

import emdatabase

Every dataset in the index.

print(len(emdatabase.list_datasets()), "datasets")

22 datasets

search is the browser widget’s search box, callable. It matches a lowercased blob of every field - name, description, technique, detector, microscope, tags, authors and their affiliations - and every whitespace separated term has to appear somewhere, though not in the same field.

for ds in emdatabase.search("amorphous"):
    techniques = ", ".join(ds.metadata.technique)
    print(f"{type(ds).__name__:24s} {techniques:8s} {ds.size:>9s}")

AmorphousFilm4nm4DSTEM   4D-STEM    1.10 GB
PdCuSiCrystallization    4D-STEM    5.75 GB
PdNiPGlass               4D-STEM   304.6 MB

Because terms may land in different fields, a query can combine what an instrument is with what the technique was.

for ds in emdatabase.search("jeol eels"):
    print(type(ds).__name__)

CuZnEELSMapping
CuZnHAADF
LSMOLineScan
LSMOLineScanLowLoss
LSMOSTOLineScan
LSMOSTOLineScanLowLoss

filter matches named fields instead. Strings compare exactly but case-insensitively, technique, tags, authors and version test membership, and a list means any of its values.

for ds in emdatabase.filter(technique="4D-STEM", tags="Strain"):
    print(ds, "·", ", ".join(ds.metadata.tags))

<LayeredCuNb4DSTEM LayeredCuNb4DSTEM.zspy · 4D-STEM · 720.9 MB> · Nanolaminate, Nanodiffraction, Layered, Orientation Mapping, Strain

for ds in emdatabase.filter(microscope_vendor=["JEOL", "Hitachi"]):
    print(ds)

<CuZnEELSMapping CuZn_EELS_mapping_tutorial.hspy · EELS · 466.1 kB>
<CuZnHAADF CuZn_HAADF.hspy · STEM · 34.0 kB>
<FeAlStripes FeAl_stripes.zspy · 4D-STEM · 382.1 MB>
<LSMOLineScan LSMO_linescan.hspy · EELS · 91.9 kB>
<LSMOLineScanLowLoss LSMO_linescan_low_loss.hspy · EELS · 138.8 kB>
<LSMOSTOLineScan LSMO_STO_linescan.hspy · EELS · 42.5 kB>
<LSMOSTOLineScanLowLoss LSMO_STO_linescan_low_loss.hspy · EELS · 42.0 kB>
<MgONanoCrystals mgo_nano.zspy · 4D-STEM · 104.3 MB>
<SPEDAg SPED-Ag.zspy · 4D-STEM · 214.0 MB>

downloaded and location describe this machine rather than the dataset, which makes “what do I already have” a one-liner.

for ds in emdatabase.filter(downloaded=True):
    print(f"{type(ds).__name__:24s} {ds.filepath()}")

BilayerWS2               /home/runner/.cache/emdatabase/smallPtychography.hspy

An unknown field raises rather than being ignored - a typo that quietly returned the whole index would be worse than an error.

try:
    emdatabase.filter(techniqu="4D-STEM")
except TypeError as error:
    print(error)

unknown filter field(s) 'techniqu'; choose from kind, version, technique, detector, detector_manufacturer, microscope_vendor, microscope_model, camera_length, voltage, license, doi, tags, authors, downloaded, location

The result is a dataset, so it is ready to use. Sizes are integers (size_bytes), so they sort; size is the same number for reading.

smallest = min(emdatabase.filter(technique="EELS"), key=lambda ds: ds.size_bytes or 0)
print(smallest)
print()
print(smallest.metadata)

<LSMOSTOLineScanLowLoss LSMO_STO_linescan_low_loss.hspy · EELS · 42.0 kB>

LSMO_STO_linescan_low_loss.hspy · EELS · 42.0 kB
The low-loss half of the DualEELS pair for LSMOSTOLineScan - a La(0.7)Sr(0.3)MnO3 on
SrTiO3 thin film line scan. 10 probe positions at 3.152 nm with 512 energy channels
covering -50 to 461 eV at 1 eV dispersion, so it contains the zero-loss peak and the
plasmon region. Use it with the core-loss spectrum for relative thickness mapping and
Fourier-ratio deconvolution. Acquired on a JEOL ARM200cF with a Gatan Quantum ER at 200
kV; the low-loss dwell time is 9.44e-05 s against 0.4999 s for the core loss.

               source: https://raw.githubusercontent.com/hyperspy/exspy-demos/927d1f21b3b8aba4e2e622c2e621d3d9d5542d1c/EELS/datasets
             checksum: md5:fd6e922c73e0b1e11229acb5960370cc
detector_manufacturer: Gatan
             detector: Quantum ER
    microscope_vendor: JEOL
     microscope_model: ARM200cF
              voltage: 200 kV
              license: Unspecified
                 tags: Line Scan, Low Loss, Zero Loss Peak, Perovskite, DualEELS,
                       Thickness, Tutorial
              authors: Magnus Nord (Norwegian University of Science and Technology)
           size_bytes: 42038 (42.0 kB)