Results and Domain Models

Results

result

Inspectable optimization result models.

UsageSummary dataclass

UsageSummary(input_tokens: int, output_tokens: int, exact_cost: float, estimated_cost: float, latency_seconds: float, status: ValueStatus)

Aggregate usage with estimate provenance retained.

input_tokens instance-attribute

input_tokens: int

output_tokens instance-attribute

output_tokens: int

exact_cost instance-attribute

exact_cost: float

estimated_cost instance-attribute

estimated_cost: float

latency_seconds instance-attribute

latency_seconds: float

status instance-attribute

status: ValueStatus

TrialSummary dataclass

TrialSummary(id: str, sequence: int, candidate: Candidate, status: TrialStatus, metrics: tuple[MetricValue, ...], usage: tuple[UsageRecord, ...], objective: ObjectiveResult | None, attempts: int, cached: bool = False, error: str | None = None)

Actionable terminal summary for one candidate trial.

id instance-attribute

id: str

sequence instance-attribute

sequence: int

candidate instance-attribute

candidate: Candidate

status instance-attribute

status: TrialStatus

metrics instance-attribute

metrics: tuple[MetricValue, ...]

usage instance-attribute

usage: tuple[UsageRecord, ...]

objective instance-attribute

objective: ObjectiveResult | None

attempts instance-attribute

attempts: int

cached class-attribute instance-attribute

cached: bool = False

error class-attribute instance-attribute

error: str | None = None

metric

metric(name: str) -> MetricValue | None

Return one named metric, or None when it was not observed.

Source code in src/tunarag/result.py

def metric(self, name: str) -> MetricValue | None:
    """Return one named metric, or None when it was not observed."""

    return next((metric for metric in self.metrics if metric.name == name), None)

OptimizationResult dataclass

OptimizationResult(study_id: str, study_status: StudyStatus, stop_reason: str, trials: tuple[TrialSummary, ...], best_trial: TrialSummary | None, usage: UsageSummary)

Completed study result, including failures and no-success outcomes.

study_id instance-attribute

study_id: str

study_status instance-attribute

study_status: StudyStatus

stop_reason instance-attribute

stop_reason: str

trials instance-attribute

trials: tuple[TrialSummary, ...]

best_trial instance-attribute

best_trial: TrialSummary | None

usage instance-attribute

usage: UsageSummary

best_candidate property

best_candidate: Candidate | None

Return the highest-scoring candidate when one succeeded.

successful_trials property

successful_trials: tuple[TrialSummary, ...]

Return successful trials in execution order.

failed_trials property

failed_trials: tuple[TrialSummary, ...]

Return failed trials in execution order.

ranked_trials property

ranked_trials: tuple[TrialSummary, ...]

Return successful trials by descending objective, then sequence.

metric_names property

metric_names: tuple[str, ...]

Return all observed metric names in lexical order.

metric_leaders

metric_leaders(name: str, *, direction: ObjectiveDirection = MAXIMIZE) -> tuple[TrialSummary, ...]

Rank successful trials that have an available named metric.

Source code in src/tunarag/result.py

def metric_leaders(
    self,
    name: str,
    *,
    direction: ObjectiveDirection = ObjectiveDirection.MAXIMIZE,
) -> tuple[TrialSummary, ...]:
    """Rank successful trials that have an available named metric."""

    if not name.strip():
        raise ValueError("metric name must not be empty")
    trials = [
        trial
        for trial in self.successful_trials
        if (metric := trial.metric(name)) is not None and metric.value is not None
    ]
    multiplier = -1.0 if direction is ObjectiveDirection.MAXIMIZE else 1.0
    return tuple(
        sorted(
            trials,
            key=lambda trial: (
                multiplier * cast(float, cast(MetricValue, trial.metric(name)).value),
                trial.sequence,
            ),
        )
    )

as_dict

as_dict() -> dict[str, Any]

Return a redacted, JSON-compatible report.

Source code in src/tunarag/result.py

def as_dict(self) -> dict[str, Any]:
    """Return a redacted, JSON-compatible report."""

    return {
        "schema_version": 1,
        "study_id": self.study_id,
        "study_status": self.study_status.value,
        "stop_reason": self.stop_reason,
        "best_trial_id": None if self.best_trial is None else self.best_trial.id,
        "counts": {
            "trials": len(self.trials),
            "successful": len(self.successful_trials),
            "failed": len(self.failed_trials),
        },
        "usage": _usage_dict(self.usage),
        "trials": [_trial_dict(trial) for trial in self.trials],
    }

to_json

to_json(*, indent: int | None = 2) -> str

Serialize the redacted report to deterministic JSON.

Source code in src/tunarag/result.py

def to_json(self, *, indent: int | None = 2) -> str:
    """Serialize the redacted report to deterministic JSON."""

    return json.dumps(
        self.as_dict(),
        ensure_ascii=False,
        allow_nan=False,
        indent=indent,
        sort_keys=True,
    )

write_json

write_json(path: str | Path, *, indent: int | None = 2) -> Path

Write a redacted JSON report and return its path.

Source code in src/tunarag/result.py

def write_json(self, path: str | Path, *, indent: int | None = 2) -> Path:
    """Write a redacted JSON report and return its path."""

    target = Path(path)
    target.write_text(self.to_json(indent=indent) + "\n", encoding="utf-8")
    return target

write_csv

write_csv(path: str | Path) -> Path

Write one redacted, flattened row per trial and return its path.

Source code in src/tunarag/result.py

def write_csv(self, path: str | Path) -> Path:
    """Write one redacted, flattened row per trial and return its path."""

    target = Path(path)
    metric_columns = [
        column
        for name in self.metric_names
        for column in (
            f"metric.{name}.value",
            f"metric.{name}.status",
            f"metric.{name}.coverage",
        )
    ]
    fieldnames = [
        "study_id",
        "trial_id",
        "sequence",
        "status",
        "objective_score",
        "objective_status",
        "attempts",
        "cached",
        "error",
        "candidate",
        "input_tokens",
        "output_tokens",
        "exact_cost",
        "estimated_cost",
        "latency_seconds",
        "usage_status",
        *metric_columns,
    ]
    with target.open("w", encoding="utf-8", newline="") as stream:
        writer = csv.DictWriter(stream, fieldnames=fieldnames, extrasaction="raise")
        writer.writeheader()
        for trial in self.trials:
            writer.writerow(_trial_csv_row(self.study_id, trial, self.metric_names))
    return target

Domain values

domain

Immutable domain values shared by the package contracts.

ValueStatus

Bases: str, Enum

Confidence state for an observed or derived value.

EXACT class-attribute instance-attribute

EXACT = 'exact'

ESTIMATED class-attribute instance-attribute

ESTIMATED = 'estimated'

UNAVAILABLE class-attribute instance-attribute

UNAVAILABLE = 'unavailable'

Secret dataclass

Secret(value: str)

A value whose contents are never exposed by representation or hashing.

value instance-attribute

value: str

MetricValue dataclass

MetricValue(name: str, value: float | None, status: ValueStatus = EXACT, coverage: float = 1.0)

A metric observation with its coverage and confidence state.

name instance-attribute

name: str

value instance-attribute

value: float | None

status class-attribute instance-attribute

status: ValueStatus = ValueStatus.EXACT

coverage class-attribute instance-attribute

coverage: float = 1.0

UsageRecord dataclass

UsageRecord(component: str, input_tokens: int | None = None, output_tokens: int | None = None, cost: float | None = None, latency_seconds: float | None = None, status: ValueStatus = EXACT, pricing_version: str | None = None)

Usage observed for one component of a trial.

component instance-attribute

component: str

input_tokens class-attribute instance-attribute

input_tokens: int | None = None

output_tokens class-attribute instance-attribute

output_tokens: int | None = None

cost class-attribute instance-attribute

cost: float | None = None

latency_seconds class-attribute instance-attribute

latency_seconds: float | None = None

status class-attribute instance-attribute

status: ValueStatus = ValueStatus.EXACT

pricing_version class-attribute instance-attribute

pricing_version: str | None = None

Candidate dataclass

Candidate(parameters: Mapping[str, Any])

A typed candidate configuration proposed for a trial.

parameters instance-attribute

parameters: Mapping[str, Any]