Core Concepts

pyPhases has four concepts. Everything else in the ecosystem builds on top of them.

Phase

A Phase is a Python class with a main() method. It represents one step in your pipeline (e.g., load data, preprocess, train). Additionally a phase can implement generateData(name) to produce named data artifacts lazily on demand. If the generateData method is not implemented, the complete main() method is executed when any data artifact is requested.

from pyPhases import Phase

class GenerateIt(Phase):
    def generateData(self, name):
        if name == "who":
            who = self.getConfig("who")
            self.project.registerData("who", who)

    def main(self):
        who = self.getData("who", str)
        self.log(f"Hello {who}!")

Key methods available inside a phase:

Method	Description
`self.getConfig("key")`	Read a config value from `project.yaml`
`self.registerData("name", value)`	Persist data via the configured exporter
`self.getData("name", Type)`	Load data (runs the generating phase if not cached)
`self.log("msg")`	Log at INFO level
`self.logSuccess("msg")`	Log at SUCCESS level

Config

Configuration lives in project.yaml under the config: key and is accessible to every phase via self.getConfig("key").

config:
  who: world
  samplerate: 128

sr = self.getConfig("samplerate")  # → 128

Config values are also used to compute a hash suffix for stored data filenames. If you change a config value, any data that dependsOn that key is automatically considered stale and regenerated. For more details on how config values affect caching, see Config Reference.

Exporter

An Exporter handles persistence — writing and reading data from disk. You register exporters in project.yaml:

exporter:
  - PickleExporter         # primitive types, numpy arrays
  - MemmapRecordExporter   # large signal recordings (memmap)

The exporter is chosen automatically based on the Python type of the data being stored.

Available default exporters:

Name	Handles
`PickleExporter`	Any picklable Python object
`PandasExporter`	`pd.DataFrame`
`MemmapRecordExporter`	`RecordSignal` (large numpy memmaps) requires `pyPhasesML` package
`pyPhasesHDF5RecordExporter`	`RecordSignal` (HDF5 backend) requires `pyPhasesHDF5RecordExporter` package
`ModelExporter`	`torch` or `tensorflow` models, depending on whats installed in the environment

For more details on how to use, register and implement exporters, see Exporter Reference.

Project

A Project wires everything together — it holds the config, knows the phase list, and owns the exporters.

Project-based workflow (most common): project.yaml + phases run — see Tutorial: Create a Project.

Flat workflow: DefaultProject.create() in Python — create an in-memory project and call phase methods directly — see Tutorial: Create a Project.

Data lifecycle

sequenceDiagram
    participant CLI as phases run SayIt
    participant SayIt
    participant Project
    participant GenerateIt
    participant Exporter

    CLI->>SayIt: main()
    SayIt->>Project: getData("sentence")
    Project->>Exporter: load("sentence--world--current")
    Exporter-->>Project: not found
    Project->>GenerateIt: generateData("sentence")
    GenerateIt->>Exporter: registerData("sentence", "Hello world!")
    Exporter-->>GenerateIt: saved to disk
    Project-->>SayIt: "Hello world!"

On the second run the exporter finds the file and skips GenerateIt entirely.