Batch Experiment Module

This section documents the batch experiment components of the Nextmv Cloud API.

batch_experiment

This module contains definitions for batch experiments.

CLASS	DESCRIPTION
BatchExperimentInformation	Base class for all batch experiment models containing common information.
BatchExperiment	Class representing a batch experiment that compares two or more instances.
BatchExperimentRun	Class representing a single execution of a batch experiment.
BatchExperimentMetadata	Class containing metadata of a batch experiment.

BatchExperiment

Bases: BatchExperimentInformation

A batch experiment compares two or more instances by executing all the inputs contained in the input set.

You can import the BatchExperiment class directly from cloud:

from nextmv.cloud import BatchExperiment

This class extends BatchExperimentInformation with attributes specific to a full batch experiment.

PARAMETER	DESCRIPTION
input_set_id	ID of the input set used for the experiment. TYPE: str
instance_ids	List of instance IDs used for the experiment. TYPE: list[str]
grouped_distributional_summaries	Grouped distributional summaries of the batch experiment. Defaults to None. TYPE: list[dict[str, Any]]

created_at instance-attribute

created_at: datetime

Creation date of the batch experiment.

description class-attribute instance-attribute

description: Optional[str] = None

Description of the batch experiment.

grouped_distributional_summaries class-attribute instance-attribute

grouped_distributional_summaries: Optional[
    list[dict[str, Any]]
] = None

Grouped distributional summaries of the batch experiment.

id instance-attribute

id: str

ID of the batch experiment.

input_set_id instance-attribute

input_set_id: str

ID of the input set used for the experiment.

instance_ids instance-attribute

instance_ids: list[str]

List of instance IDs used for the experiment.

name instance-attribute

name: str

Name of the batch experiment.

number_of_completed_runs class-attribute instance-attribute

number_of_completed_runs: Optional[int] = None

Number of completed runs in the batch experiment.

number_of_requested_runs class-attribute instance-attribute

number_of_requested_runs: Optional[int] = None

Number of runs requested for the batch experiment.

number_of_runs class-attribute instance-attribute

number_of_runs: Optional[int] = None

Number of runs in the batch experiment.

option_sets class-attribute instance-attribute

option_sets: Optional[dict[str, dict[str, str]]] = None

Option sets used for the experiment.

status class-attribute instance-attribute

status: Optional[str] = None

Status of the batch experiment.

type class-attribute instance-attribute

type: Optional[str] = None

Type of the batch experiment.

updated_at instance-attribute

updated_at: datetime

Last update date of the batch experiment.

BatchExperimentInformation

Bases: BaseModel

Information about a batch experiment.

You can import the BatchExperimentInformation class directly from cloud:

from nextmv.cloud import BatchExperimentInformation

This class serves as a base for all the other batch experiment models and contains common attributes shared by different types of batch experiments.

PARAMETER	DESCRIPTION
id	ID of the batch experiment. TYPE: str
name	Name of the batch experiment. TYPE: str
created_at	Creation date of the batch experiment. TYPE: datetime
updated_at	Last update date of the batch experiment. TYPE: datetime
status	Status of the batch experiment. Defaults to None. TYPE: str
description	Description of the batch experiment. Defaults to None. TYPE: str
number_of_requested_runs	Number of runs requested for the batch experiment. Defaults to None. TYPE: int
number_of_runs	Number of runs in the batch experiment. Defaults to None. TYPE: int
number_of_completed_runs	Number of completed runs in the batch experiment. Defaults to None. TYPE: int
type	Type of the batch experiment. Defaults to None. TYPE: str
option_sets	Option sets used for the experiment. Defaults to None. TYPE: dict[str, dict[str, str]]

Examples:

>>> from datetime import datetime
>>> info = BatchExperimentInformation(
...     id="bexp-123",
...     name="Test Experiment",
...     created_at=datetime.now(),
...     updated_at=datetime.now(),
...     status="running",
...     description="A sample batch experiment."
... )
>>> print(info.id)
bexp-123
>>> print(info.name)
Test Experiment

created_at instance-attribute

created_at: datetime

Creation date of the batch experiment.

description class-attribute instance-attribute

description: Optional[str] = None

Description of the batch experiment.

id instance-attribute

id: str

ID of the batch experiment.

name instance-attribute

name: str

Name of the batch experiment.

number_of_completed_runs class-attribute instance-attribute

number_of_completed_runs: Optional[int] = None

Number of completed runs in the batch experiment.

number_of_requested_runs class-attribute instance-attribute

number_of_requested_runs: Optional[int] = None

Number of runs requested for the batch experiment.

number_of_runs class-attribute instance-attribute

number_of_runs: Optional[int] = None

Number of runs in the batch experiment.

option_sets class-attribute instance-attribute

option_sets: Optional[dict[str, dict[str, str]]] = None

Option sets used for the experiment.

status class-attribute instance-attribute

status: Optional[str] = None

Status of the batch experiment.

type class-attribute instance-attribute

type: Optional[str] = None

Type of the batch experiment.

updated_at instance-attribute

updated_at: datetime

Last update date of the batch experiment.

BatchExperimentMetadata

Bases: BatchExperimentInformation

Metadata of a batch experiment.

You can import the BatchExperimentMetadata class directly from cloud:

from nextmv.cloud import BatchExperimentMetadata

This class extends BatchExperimentInformation with application-specific metadata.

PARAMETER	DESCRIPTION
app_id	ID of the application used for the batch experiment. Defaults to None. TYPE: str

app_id class-attribute instance-attribute

app_id: Optional[str] = None

ID of the application used for the batch experiment.

created_at instance-attribute

created_at: datetime

Creation date of the batch experiment.

description class-attribute instance-attribute

description: Optional[str] = None

Description of the batch experiment.

id instance-attribute

id: str

ID of the batch experiment.

name instance-attribute

name: str

Name of the batch experiment.

number_of_completed_runs class-attribute instance-attribute

number_of_completed_runs: Optional[int] = None

Number of completed runs in the batch experiment.

number_of_requested_runs class-attribute instance-attribute

number_of_requested_runs: Optional[int] = None

Number of runs requested for the batch experiment.

number_of_runs class-attribute instance-attribute

number_of_runs: Optional[int] = None

Number of runs in the batch experiment.

option_sets class-attribute instance-attribute

option_sets: Optional[dict[str, dict[str, str]]] = None

Option sets used for the experiment.

status class-attribute instance-attribute

status: Optional[str] = None

Status of the batch experiment.

type class-attribute instance-attribute

type: Optional[str] = None

Type of the batch experiment.

updated_at instance-attribute

updated_at: datetime

Last update date of the batch experiment.

BatchExperimentRun

Bases: BaseModel

A batch experiment run is a single execution of a batch experiment.

You can import the BatchExperimentRun class directly from cloud:

from nextmv.cloud import BatchExperimentRun

It contains information about the experiment, the input used, and the configuration used for the run.

PARAMETER	DESCRIPTION
input_id	ID of the input used for the experiment. TYPE: str
option_set	Option set used for the experiment. Defaults to None. TYPE: str
instance_id	ID of the instance used for the experiment. Defaults to None. TYPE: str
version_id	ID of the version used for the experiment. Defaults to None. TYPE: str
input_set_id	ID of the input set used for the experiment. Defaults to None. TYPE: str
scenario_id	If the batch experiment is a scenario test, this is the ID of that test. Defaults to None. TYPE: str
repetition	Repetition number of the experiment. Defaults to None. TYPE: int
run_number	Run number of the experiment. Defaults to None. TYPE: str

input_id instance-attribute

input_id: str

ID of the input used for the experiment.

input_set_id class-attribute instance-attribute

input_set_id: Optional[str] = None

ID of the input set used for the experiment.

instance_id class-attribute instance-attribute

instance_id: Optional[str] = None

ID of the instance used for the experiment.

option_set class-attribute instance-attribute

option_set: Optional[str] = None

Option set used for the experiment.

repetition class-attribute instance-attribute

repetition: Optional[int] = None

Repetition number of the experiment.

scenario_id class-attribute instance-attribute

scenario_id: Optional[str] = None

If the batch experiment is a scenario test, this is the ID of that test.

version_id class-attribute instance-attribute

version_id: Optional[str] = None

ID of the version used for the experiment.

to_runs

to_runs(
    instance_ids: list[str], input_set: InputSet
) -> list[BatchExperimentRun]

Translate a legacy batch experiment list of instance ids to runs.

PARAMETER	DESCRIPTION
instance_ids	List of instance IDs to be converted into runs. TYPE: list[str]
input_set	Input set associated with the runs. TYPE: InputSet

RETURNS	DESCRIPTION
list[BatchExperimentRun]	A list of BatchExperimentRun objects created from the instance IDs.

Source code in nextmv/nextmv/cloud/batch_experiment.py

def to_runs(instance_ids: list[str], input_set: InputSet) -> list[BatchExperimentRun]:
    """
    Translate a legacy batch experiment list of instance ids to runs.

    Parameters
    ----------
    instance_ids : list[str]
        List of instance IDs to be converted into runs.
    input_set : InputSet
        Input set associated with the runs.

    Returns
    -------
    list[BatchExperimentRun]
        A list of `BatchExperimentRun` objects created from the instance IDs.
    """

    input_ids = input_set.input_ids
    if len(input_set.input_ids) == 0:
        input_ids = [i.id for i in input_set.inputs]

    runs = []
    for instance_id in instance_ids:
        for input_id in input_ids:
            run = BatchExperimentRun(
                input_id=input_id,
                instance_id=instance_id,
                input_set_id=input_set.id,
            )
            runs.append(run)

    return runs