emdatabase.config

Configuration for emdatabase, in the style of dask’s (and quantem’s) config.

Two keys are shipped, in emdatabase/emdatabase.yaml. locations is a mapping of name to directory: personal is the reserved name for the one writable location, where downloads go (null means pooch’s cache directory); every other entry is a read-only directory, searched before it:

locations:
  example_data: /group/example_data
  personal: /big/disk/emdatabase

check_updates (true by default) is whether downloading a weights family’s latest asks the index on the project’s main branch whether newer weights have been published, and warns if they have.

Add and remove entries with add_location(), locations() and remove_location():

from emdatabase import config

config.add_location("/group/example_data")                       # read-only
config.add_location("/big/disk/emdatabase", name="personal")     # downloads
config.locations()                                               # search order
config.remove_location("example_data")

Each of those persists to the config file unless called with persist=False. A location’s name is also what a download writes into, which is how a shared location is seeded: data.CuZnHAADF().download(destination="example_data").

Or read and write the key directly:

config.get("locations")
config.set({"locations.personal": "/big/disk/emdatabase"})  # for this process
with config.set({"locations.personal": "/scratch"}):        # or for a block
    ...
config.write()                                              # persist to the yaml

Or from the environment, prefix EMDATABASE_, double underscore to nest:

EMDATABASE_LOCATIONS__PERSONAL=/scratch/data
EMDATABASE_LOCATIONS__GROUP=/wigeon/shared/example_data

Files live in ~/.config/emdatabase (or wherever EMDATABASE_CONFIG points); every *.yaml and *.yml in that directory is merged, in name order. Precedence, lowest first: shipped defaults, config files, environment variables, set.

Functions

add_location(path[, name, persist])	Add a location, or repoint one that is already configured.
canonical_name(k, config)	Return the canonical name for a key.
check_key_val(key, val[, deprecations])	Check whether a key has been renamed, removed, or is not one we ship
collect([path, env])	Collect configuration from the config directory and the environment
collect_env([env])	Collect config from environment variables
collect_yaml(path)	Collect configuration from the yaml files in a directory
data_dir()	The directory downloads are written to.
data_search_dirs()	Everywhere to look for an existing dataset: the shared locations, then data_dir().
first_run_notice([directory])	Say where downloads will go, once per process.
get(key[, default, config, override_with])	Get elements from global config
interpret_value(value)
locations()	Every configured location, in search order.
merge(*dicts)	Update a sequence of nested dictionaries
refresh([config, defaults])	Update configuration by re-reading yaml files and env variables
remove_location(name_or_path[, persist])	Remove a location, or reset personal to the cache directory.
resolve_destination(destination)	The directory a destination= argument names, or None.
update(old, new[, priority, defaults, check])	Update a nested dictionary with values from another
update_defaults(new[, config, defaults])	Add a new set of defaults to the configuration
write([path])	Write the current configuration to a yaml file.

Classes

Location(name, path, kind)	One place datasets are looked for.
set([arg, config])	Temporarily set configuration values within a context manager