File-Based Archivers

ArrayArchiver, ZarrArchiver, and JsonArchiver all write payloads to the local filesystem. The output path (or store path) for each event is resolved at runtime by rendering a Jinja2 template against the merged metadata.

Jinja2 path templates

Every file-based archiver accepts one or more template strings. Templates are rendered against a metadata dict that is the merge of default_metadata (static defaults set at configuration time) and the per-event metadata received with each payload. default_metadata values act as fallbacks; per-event values take precedence on key collision.

Any Python expression valid inside a Jinja2 {{ }} block can be used, including arithmetic and filter pipelines:

{{ file_path }}/shot-{{ '%04d' % (frame / 2) | round(0, 'floor') }}-{{ frame % 2 }}.npy

This produces paths such as:

/mnt/storage/shot-0000-0.npy
/mnt/storage/shot-0000-1.npy
/mnt/storage/shot-0001-0.npy

when file_path is /mnt/storage and frame is a running integer supplied by the event source.

Parent directories are created automatically if they do not exist.

ArrayArchiver

Saves each numpy array payload as a .npy file.

Parameters

save_template

Jinja2 template rendered to the output file path.

split_data_array (default: False)

When True, each row of a 2-D array is written as a separate file. The special variable {{ _split_index }} is injected into the metadata before rendering so that row number can appear in the filename.

BOSS configuration example

{
  "_id": "my-camera-capturer",
  "classname": "herostools.actor.archiver.ArrayArchiver",
  "arguments": {
    "object_selector": "my-camera",
    "event_name": "acquisition_data",
    "default_metadata": {
      "file_path": "/mnt/storage/images"
    },
    "save_template": "{{ file_path }}/frame-{{ '%04d' % frame }}.npy"
  }
}

ZarrArchiver

Stores numpy array payloads inside a zarr directory store. A zarr store is a directory (e.g. shot_0001.zarr/) — not a single file. Its internal structure (chunk files, .zarray metadata) is managed by zarr and should be treated as opaque. The store is opened in append mode ('a') so successive writes to the same store path do not erase existing arrays.

Parameters

store_template

Jinja2 template rendered to the zarr store directory path.

array_path_template

Jinja2 template rendered to the array key within the store. Nested keys (e.g. sensors/spectrum) create a group hierarchy inside the store.

BOSS configuration example

{
  "_id": "my-zarr-archiver",
  "classname": "herostools.actor.archiver.ZarrArchiver",
  "arguments": {
    "object_selector": "my-instrument",
    "event_name": "frame_data",
    "default_metadata": {
      "store_root": "/mnt/storage/zarr"
    },
    "store_template": "{{ store_root }}/{{ run_id }}.zarr",
    "array_path_template": "{{ identifier }}/data"
  }
}

With run_id = "run_01" and identifier = "shot_001" this writes the array to /mnt/storage/zarr/run_01.zarr/shot_001/data.

To read the array back:

import zarr
group = zarr.open_group("/mnt/storage/zarr/run_01.zarr", mode="r")
arr = group["shot_001/data"][:]

JsonArchiver

Saves each dict payload as a .json file.

Parameters

save_template

Jinja2 template rendered to the output file path.

merge_metadata (default: False)

When True, the metadata dict is embedded in the saved JSON under the key "metadata".

BOSS configuration example

{
  "_id": "my-json-archiver",
  "classname": "herostools.actor.archiver.JsonArchiver",
  "arguments": {
    "object_selector": "my-device",
    "event_name": "result_data",
    "default_metadata": {
      "output_dir": "/mnt/storage/results"
    },
    "save_template": "{{ output_dir }}/{{ identifier }}.json",
    "merge_metadata": true
  }
}