emdatabase.config

Configuration for emdatabase, in the style of dask’s (and quantem’s) config.

Two keys are shipped, in emdatabase/emdatabase.yaml. locations is a mapping of name to directory: personal is the reserved name for the one writable location, where downloads go (null means pooch’s cache directory); every other entry is a read-only directory, searched before it:

locations:
  example_data: /group/example_data
  personal: /big/disk/emdatabase

check_updates (true by default) is whether downloading a weights family’s latest asks the index on the project’s main branch whether newer weights have been published, and warns if they have.

Add and remove entries with add_location(), locations() and remove_location():

from emdatabase import config

config.add_location("/group/example_data")                       # read-only
config.add_location("/big/disk/emdatabase", name="personal")     # downloads
config.locations()                                               # search order
config.remove_location("example_data")

Each of those persists to the config file unless called with persist=False. A location’s name is also what a download writes into, which is how a shared location is seeded: data.CuZnHAADF().download(destination="example_data").

Or read and write the key directly:

config.get("locations")
config.set({"locations.personal": "/big/disk/emdatabase"})  # for this process
with config.set({"locations.personal": "/scratch"}):        # or for a block
    ...
config.write()                                              # persist to the yaml

Or from the environment, prefix EMDATABASE_, double underscore to nest:

EMDATABASE_LOCATIONS__PERSONAL=/scratch/data
EMDATABASE_LOCATIONS__GROUP=/wigeon/shared/example_data

Files live in ~/.config/emdatabase (or wherever EMDATABASE_CONFIG points); every *.yaml and *.yml in that directory is merged, in name order. Precedence, lowest first: shipped defaults, config files, environment variables, set.

Functions

add_location(path[, name, persist])	Add a location, or repoint one that is already configured.
collect_env([env])	Collect config from environment variables
collect_yaml(path)	Collect configuration from the yaml files in a directory
data_dir()	The directory downloads are written to.
data_search_dirs()	Everywhere to look for an existing dataset: the shared locations, then data_dir().
first_run_notice(directory)	Say where downloads will go, once per process.
get(key[, default])	Get a configuration value.
interpret_value(value)
locations()	Every configured location, in search order.
refresh()	Re-read the configuration: the shipped defaults, then the config files, then the environment.
remove_location(name_or_path[, persist])	Remove a location, or reset personal to the cache directory.
resolve_destination(destination)	The directory a destination= argument names, or None.
update(old, new)	Merge new into old in place; new wins, and a nested mapping is merged key by key rather than replacing the one in old.
write([path])	Write the current configuration to a yaml file.

Classes

Location(name, path, kind)	One place datasets are looked for.
set(arg[, config])	Set configuration values, for the process or, as a context manager, for a block