Batch Experiment Module

This section documents the batch experiment components of the Nextmv Cloud API.

batch_experiment

This module contains definitions for batch experiments.

CLASS	DESCRIPTION
ExperimentStatus	Enum representing the status of an experiment.
BatchExperimentInformation	Base class for all batch experiment models containing common information.
BatchExperiment	Class representing a batch experiment that compares two or more instances.
BatchExperimentRun	Class representing a single execution of a batch experiment.
BatchExperimentMetadata	Class containing metadata of a batch experiment.

BatchExperiment

Bases: BatchExperimentInformation

A batch experiment compares two or more instances by executing all the inputs contained in the input set.

You can import the BatchExperiment class directly from cloud:

from nextmv.cloud import BatchExperiment

This class extends BatchExperimentInformation with attributes specific to a full batch experiment.

PARAMETER	DESCRIPTION
input_set_id	ID of the input set used for the experiment. TYPE: str
instance_ids	List of instance IDs used for the experiment. TYPE: list[str]
grouped_distributional_summaries	Grouped distributional summaries of the batch experiment. Defaults to None. TYPE: list[dict[str, Any]]

created_at instance-attribute

created_at: datetime

Creation date of the batch experiment.

description class-attribute instance-attribute

description: Optional[str] = None

Description of the batch experiment.

grouped_distributional_summaries class-attribute instance-attribute

grouped_distributional_summaries: Optional[
    list[dict[str, Any]]
] = None

Grouped distributional summaries of the batch experiment.

id instance-attribute

id: str

ID of the batch experiment.

input_set_id instance-attribute

input_set_id: str

ID of the input set used for the experiment.

instance_ids instance-attribute

instance_ids: list[str]

List of instance IDs used for the experiment.

name instance-attribute

name: str

Name of the batch experiment.

number_of_completed_runs class-attribute instance-attribute

number_of_completed_runs: Optional[int] = None

Number of completed runs in the batch experiment.

number_of_requested_runs class-attribute instance-attribute

number_of_requested_runs: Optional[int] = None

Number of runs requested for the batch experiment.

number_of_runs class-attribute instance-attribute

number_of_runs: Optional[int] = None

Number of runs in the batch experiment.

option_sets class-attribute instance-attribute

option_sets: Optional[dict[str, dict[str, str]]] = None

Option sets used for the experiment.

status class-attribute instance-attribute

status: Optional[ExperimentStatus] = None

Status of the batch experiment.

type class-attribute instance-attribute

type: Optional[str] = None

Type of the batch experiment.

updated_at instance-attribute

updated_at: datetime

Last update date of the batch experiment.

BatchExperimentInformation

Bases: BaseModel

Information about a batch experiment.

You can import the BatchExperimentInformation class directly from cloud:

from nextmv.cloud import BatchExperimentInformation

This class serves as a base for all the other batch experiment models and contains common attributes shared by different types of batch experiments.

PARAMETER	DESCRIPTION
id	ID of the batch experiment. TYPE: str
name	Name of the batch experiment. TYPE: str
created_at	Creation date of the batch experiment. TYPE: datetime
updated_at	Last update date of the batch experiment. TYPE: datetime
status	Status of the batch experiment. Defaults to None. TYPE: str
description	Description of the batch experiment. Defaults to None. TYPE: str
number_of_requested_runs	Number of runs requested for the batch experiment. Defaults to None. TYPE: int
number_of_runs	Number of runs in the batch experiment. Defaults to None. TYPE: int
number_of_completed_runs	Number of completed runs in the batch experiment. Defaults to None. TYPE: int
type	Type of the batch experiment. Defaults to None. TYPE: str
option_sets	Option sets used for the experiment. Defaults to None. TYPE: dict[str, dict[str, str]]

Examples:

>>> from datetime import datetime
>>> info = BatchExperimentInformation(
...     id="bexp-123",
...     name="Test Experiment",
...     created_at=datetime.now(),
...     updated_at=datetime.now(),
...     status="running",
...     description="A sample batch experiment."
... )
>>> print(info.id)
bexp-123
>>> print(info.name)
Test Experiment

created_at instance-attribute

created_at: datetime

Creation date of the batch experiment.

description class-attribute instance-attribute

description: Optional[str] = None

Description of the batch experiment.

id instance-attribute

id: str

ID of the batch experiment.

name instance-attribute

name: str

Name of the batch experiment.

number_of_completed_runs class-attribute instance-attribute

number_of_completed_runs: Optional[int] = None

Number of completed runs in the batch experiment.

number_of_requested_runs class-attribute instance-attribute

number_of_requested_runs: Optional[int] = None

Number of runs requested for the batch experiment.

number_of_runs class-attribute instance-attribute

number_of_runs: Optional[int] = None

Number of runs in the batch experiment.

option_sets class-attribute instance-attribute

option_sets: Optional[dict[str, dict[str, str]]] = None

Option sets used for the experiment.

status class-attribute instance-attribute

status: Optional[ExperimentStatus] = None

Status of the batch experiment.

type class-attribute instance-attribute

type: Optional[str] = None

Type of the batch experiment.

updated_at instance-attribute

updated_at: datetime

Last update date of the batch experiment.

BatchExperimentMetadata

Bases: BatchExperimentInformation

Metadata of a batch experiment.

You can import the BatchExperimentMetadata class directly from cloud:

from nextmv.cloud import BatchExperimentMetadata

This class extends BatchExperimentInformation with application-specific metadata.

PARAMETER	DESCRIPTION
app_id	ID of the application used for the batch experiment. Defaults to None. TYPE: str

app_id class-attribute instance-attribute

app_id: Optional[str] = None

ID of the application used for the batch experiment.

created_at instance-attribute

created_at: datetime

Creation date of the batch experiment.

description class-attribute instance-attribute

description: Optional[str] = None

Description of the batch experiment.

id instance-attribute

id: str

ID of the batch experiment.

name instance-attribute

name: str

Name of the batch experiment.

number_of_completed_runs class-attribute instance-attribute

number_of_completed_runs: Optional[int] = None

Number of completed runs in the batch experiment.

number_of_requested_runs class-attribute instance-attribute

number_of_requested_runs: Optional[int] = None

Number of runs requested for the batch experiment.

number_of_runs class-attribute instance-attribute

number_of_runs: Optional[int] = None

Number of runs in the batch experiment.

option_sets class-attribute instance-attribute

option_sets: Optional[dict[str, dict[str, str]]] = None

Option sets used for the experiment.

status class-attribute instance-attribute

status: Optional[ExperimentStatus] = None

Status of the batch experiment.

type class-attribute instance-attribute

type: Optional[str] = None

Type of the batch experiment.

updated_at instance-attribute

updated_at: datetime

Last update date of the batch experiment.

BatchExperimentRun

Bases: BaseModel

A batch experiment run is a single execution of a batch experiment.

You can import the BatchExperimentRun class directly from cloud:

from nextmv.cloud import BatchExperimentRun

It contains information about the experiment, the input used, and the configuration used for the run.

PARAMETER	DESCRIPTION
input_id	ID of the input used for the experiment. TYPE: str
option_set	Option set used for the experiment. Defaults to None. TYPE: str
instance_id	ID of the instance used for the experiment. Defaults to None. TYPE: str
version_id	ID of the version used for the experiment. Defaults to None. TYPE: str
input_set_id	ID of the input set used for the experiment. Defaults to None. TYPE: str
scenario_id	If the batch experiment is a scenario test, this is the ID of that test. Defaults to None. TYPE: str
repetition	Repetition number of the experiment. Defaults to None. TYPE: int
run_number	Run number of the experiment. Defaults to None. TYPE: str

input_id instance-attribute

input_id: str

ID of the input used for the experiment.

input_set_id class-attribute instance-attribute

input_set_id: Optional[str] = None

ID of the input set used for the experiment.

instance_id class-attribute instance-attribute

instance_id: Optional[str] = None

ID of the instance used for the experiment.

option_set class-attribute instance-attribute

option_set: Optional[str] = None

Option set used for the experiment.

repetition class-attribute instance-attribute

repetition: Optional[int] = None

Repetition number of the experiment.

scenario_id class-attribute instance-attribute

scenario_id: Optional[str] = None

If the batch experiment is a scenario test, this is the ID of that test.

version_id class-attribute instance-attribute

version_id: Optional[str] = None

ID of the version used for the experiment.

ExperimentStatus

Bases: str, Enum

Status of an experiment.

You can import the ExperimentStatus class directly from cloud:

python from nextmv.cloud import ExperimentStatus

This enum represents the comprehensive set of possible states for an experiment in Nextmv Cloud.

Attributes

STARTED : str Experiment started. COMPLETED : str Experiment completed. FAILED : str Experiment failed. DRAFT : str Experiment is a draft. CANCELED : str Experiment was canceled. STOPPING : str Experiment is stopping. DELETING : str Experiment is being deleted. DELETE_FAILED : str Experiment deletion failed. UNKNOWN : str Experiment status is unknown.

Examples

from nextmv.cloud import ExperimentStatus status = ExperimentStatus.STARTED print(f"The status is: {status.value}") The status is: started

if status == ExperimentStatus.COMPLETED: ... print("Processing complete.") ... elif status in [ExperimentStatus.STARTED, ExperimentStatus.STOPPING]: ... print("Processing in progress.") ... else: ... print("Processing has not started or has ended with issues.") Processing in progress.

CANCELED class-attribute instance-attribute

CANCELED = 'canceled'

Experiment was canceled.

COMPLETED class-attribute instance-attribute

COMPLETED = 'completed'

Experiment completed.

DELETE_FAILED class-attribute instance-attribute

DELETE_FAILED = 'delete-failed'

Experiment deletion failed.

DELETING class-attribute instance-attribute

DELETING = 'deleting'

Experiment is being deleted.

DRAFT class-attribute instance-attribute

DRAFT = 'draft'

Experiment is a draft.

FAILED class-attribute instance-attribute

FAILED = 'failed'

Experiment failed.

STARTED class-attribute instance-attribute

STARTED = 'started'

Experiment started.

STOPPING class-attribute instance-attribute

STOPPING = 'stopping'

Experiment is stopping.

UNKNOWN class-attribute instance-attribute

UNKNOWN = 'unknown'

Experiment status is unknown.

to_runs

to_runs(
    instance_ids: list[str], input_set: InputSet
) -> list[BatchExperimentRun]

Translate a legacy batch experiment list of instance ids to runs.

PARAMETER	DESCRIPTION
instance_ids	List of instance IDs to be converted into runs. TYPE: list[str]
input_set	Input set associated with the runs. TYPE: InputSet

RETURNS	DESCRIPTION
list[BatchExperimentRun]	A list of BatchExperimentRun objects created from the instance IDs.

Source code in nextmv/nextmv/cloud/batch_experiment.py

def to_runs(instance_ids: list[str], input_set: InputSet) -> list[BatchExperimentRun]:
    """
    Translate a legacy batch experiment list of instance ids to runs.

    Parameters
    ----------
    instance_ids : list[str]
        List of instance IDs to be converted into runs.
    input_set : InputSet
        Input set associated with the runs.

    Returns
    -------
    list[BatchExperimentRun]
        A list of `BatchExperimentRun` objects created from the instance IDs.
    """

    input_ids = input_set.input_ids
    if len(input_set.input_ids) == 0:
        input_ids = [i.id for i in input_set.inputs]

    runs = []
    for instance_id in instance_ids:
        for input_id in input_ids:
            run = BatchExperimentRun(
                input_id=input_id,
                instance_id=instance_id,
                input_set_id=input_set.id,
            )
            runs.append(run)

    return runs