Input Module¶

This section documents the input components of the Nextmv Python SDK.

input ¶

Module for handling input sources and data.

This module provides classes and functions for loading and handling input data in various formats for decision problems. It supports JSON and multi-file formats and can load data from standard input or files.

CLASS	DESCRIPTION
`Input`	Container for input data with format specification and options.
`InputLoader`	Base class for loading inputs from various sources.
`LocalInputLoader`	Class for loading inputs from local files or stdin.

FUNCTION	DESCRIPTION
`load`	Load input data using a specified loader.

ATTRIBUTE	DESCRIPTION
`INPUTS_KEY`	Key used for identifying inputs in the run. TYPE: `str`

DataFile `dataclass` ¶

DataFile(
    name: str,
    loader: Callable[[str], Any],
    loader_kwargs: dict[str, Any] | None = None,
    loader_args: list[Any] | None = None,
    input_data_key: str | None = None,
)

Represents data to be read from a file.

You can import the DataFile class directly from nextmv:

from nextmv import DataFile

This class is used to define data that will be read from a file in the filesystem. It includes the name of the file, and the reader function that will handle the loading, and deserialization of the data from the file. This DataFile class is typically used in the Input, when the Input.input_format is set to ContentFormat.MULTI_FILE. Given that it is difficul to handle every edge case of how data is deserialized, and read from a file, this class exists so that the user can implement the reader callable of their choice and provide it with any reader_args and reader_kwargs they might need.

PARAMETER DESCRIPTION

`name` ¶

Name of the data (input) file. The file extension should be included in the name.

TYPE: str

`reader` ¶

Callable that reads the data from the file. This should be a function implemented by the user. There are convenience functions that you can use as a reader as well. The reader must receive, at the very minimum, the following arguments:

file_path: a str argument which is the location where this data will be read from. This includes the dir and name of the file. As such, the name parameter of this class is going to be passed to the reader function, joined with the directory where the file will be read from.

The reader can also receive additional arguments, and keyword arguments. The reader_args and reader_kwargs parameters of this class can be used to provide those additional arguments.

The reader function should return the data that will be used in the model.

TYPE: Callable[[str], Any]

Returned by:

nextmv ✏️ Modeling Reference Input Module input

Used by:

nextmv ✏️ Modeling Reference Input Module input
- LocalInputLoader load
- load

input_data_key `class-attribute` `instance-attribute` ¶

input_data_key: str | None = None

Use this parameter to set a custom key to represent your file.

When using ContentFormat.MULTI_FILE as the input_format of the Input, the data from the file is loaded to the .data parameter of the Input. In that case, the type of .data is dict[str, Any], where each key represents the file name (with extension) and the value is the data that is actually loaded from the file using the loader function. You can set a custom key to represent your file by using this attribute.

loader `instance-attribute` ¶

loader: Callable[[str], Any]

Callable that reads (loads) the data from the file. This should be a function implemented by the user. There are convenience functions that you can use as a loader as well. The loader must receive, at the very minimum, the following arguments:

file_path: a str argument which is the location where this data will be read from. This includes the dir and name of the file. As such, the name parameter of this class is going to be passed to the loader function, joined with the directory where the file will be read from.

The loader can also receive additional arguments, and keyword arguments. The loader_args and loader_kwargs parameters of this class can be used to provide those additional arguments.

The loader function should return the data that will be used in the model.

loader_args `class-attribute` `instance-attribute` ¶

loader_args: list[Any] | None = None

Optional positional arguments to pass to the loader function. This can be used to customize the behavior of the loader.

loader_kwargs `class-attribute` `instance-attribute` ¶

loader_kwargs: dict[str, Any] | None = None

Optional keyword arguments to pass to the loader function. This can be used to customize the behavior of the loader.

name `instance-attribute` ¶

name: str

Name of the data (input) file. The file extension should be included in the name.

INPUTS_KEY `module-attribute` ¶

INPUTS_KEY = 'inputs'

Inputs key constant used for identifying inputs in the run.

Input `dataclass` ¶

Input(
    data: dict[str, Any]
    | Any
    | str
    | list[dict[str, Any]]
    | dict[str, list[dict[str, Any]]]
    | dict[str, Any],
    input_format: ContentFormat | None = JSON,
    options: Options | None = None,
)

Input for a decision problem.

You can import the Input class directly from nextmv:

from nextmv import Input

The data's type must match the input_format:

ContentFormat.JSON: the data is Union[dict[str, Any], Any]. This just means that the data must be JSON-deserializable, which includes dicts and lists.
ContentFormat.MULTI_FILE: the data is dict[str, Any], where for each item, the key is the file name (with the extension) and the actual data from the file is the value. When working with multi-file, data is loaded from one or more files in a specific directory. Given that each file can be of different types (JSON, CSV, Excel, etc...), the data captured from each might vary. To reflect this, the data is loaded as a dict of items. You can have a custom key for the data, that is not the file name, if you use the input_data_key parameter of the DataFile class.

PARAMETER	DESCRIPTION
`data` ¶	The actual data. TYPE: `Union[dict[str, Any], Any, dict[str, Any]]`
`input_format` ¶	Format of the input data. Default is `ContentFormat.JSON`. TYPE: `ContentFormat` DEFAULT: `JSON`
`options` ¶	Options that the input was created with. TYPE: `Options` DEFAULT: `None`

RAISES	DESCRIPTION
`ValueError`	If the data type doesn't match the expected type for the given format.
`ValueError`	If the `input_format` is not one of the supported formats.

Returned by:

nextmv ✏️ Modeling Reference Input Module input

data `instance-attribute` ¶

data: (
    dict[str, Any]
    | Any
    | str
    | list[dict[str, Any]]
    | dict[str, list[dict[str, Any]]]
    | dict[str, Any]
)

The actual data.

The data can be of various types, depending on the input format:

For ContentFormat.JSON: Union[dict[str, Any], Any]
For ContentFormat.MULTI_FILE: dict[str, Any]

input_format `class-attribute` `instance-attribute` ¶

input_format: ContentFormat | None = JSON

Format of the input data.

Default is ContentFormat.JSON.

options `class-attribute` `instance-attribute` ¶

options: Options | None = None

Options that the Input was created with.

A copy of the options is made during initialization, ensuring the original options remain unchanged even if modified later.

to_dict ¶

to_dict() -> dict[str, Any]

Convert the input to a dictionary.

This method serializes the Input object to a dictionary format that can be easily converted to JSON or other serialization formats. When the input_type is set to ContentFormat.MULTI_FILE, it will not include the data field, as it is uncertain how data is deserialized from the file.

RETURNS	DESCRIPTION
`dict[str, Any]`	A dictionary containing the input data, format, and options. The structure is: `{ "data": <the input data>, "input_format": <the input format as a string>, "options": <the options as a dictionary or None> }`

Examples:

>>> from nextmv.input import Input, ContentFormat
>>> input_obj = Input(data={"key": "value"}, input_format=ContentFormat.JSON)
>>> input_dict = input_obj.to_dict()
>>> print(input_dict)
{'data': {'key': 'value'}, 'input_format': 'json', 'options': None}

Source code in nextmv/nextmv/input.py

def to_dict(self) -> dict[str, Any]:
    """
    Convert the input to a dictionary.

    This method serializes the Input object to a dictionary format that can
    be easily converted to JSON or other serialization formats. When the
    `input_type` is set to `ContentFormat.MULTI_FILE`, it will not include
    the `data` field, as it is uncertain how data is deserialized from the file.

    Returns
    -------
    dict[str, Any]
        A dictionary containing the input data, format, and options.

        The structure is:
        ```python
        {
            "data": <the input data>,
            "input_format": <the input format as a string>,
            "options": <the options as a dictionary or None>
        }
        ```

    Examples
    --------
    >>> from nextmv.input import Input, ContentFormat
    >>> input_obj = Input(data={"key": "value"}, input_format=ContentFormat.JSON)
    >>> input_dict = input_obj.to_dict()
    >>> print(input_dict)
    {'data': {'key': 'value'}, 'input_format': 'json', 'options': None}
    """

    input_dict = {
        "input_format": self.input_format.value,
        "options": self.options.to_dict() if self.options is not None else None,
    }

    if self.input_format == ContentFormat.MULTI_FILE:
        return input_dict

    input_dict["data"] = self.data

    return input_dict

InputLoader ¶

Base class for loading inputs.

You can import the InputLoader class directly from nextmv:

from nextmv import InputLoader

This abstract class defines the interface for input loaders. Subclasses must implement the load method to provide concrete input loading functionality.

Subclassed by:

nextmv ✏️ Modeling Reference Input Module input LocalInputLoader

Used by:

nextmv ✏️ Modeling Reference Input Module input load

load ¶

load(
    input_format: ContentFormat = JSON,
    options: Options | None = None,
    *args,
    **kwargs,
) -> Input

Read the input data. This method should be implemented by subclasses.

PARAMETER	DESCRIPTION
`input_format` ¶	Format of the input data. Default is `ContentFormat.JSON`. TYPE: `ContentFormat` DEFAULT: `JSON`
`options` ¶	Options for loading the input data. TYPE: `Options` DEFAULT: `None`
`*args` ¶	Additional positional arguments. DEFAULT: `()`
`**kwargs` ¶	Additional keyword arguments. DEFAULT: `{}`

RETURNS	DESCRIPTION
`Input`	The input data.

RAISES	DESCRIPTION
`NotImplementedError`	If the method is not implemented.

Source code in nextmv/nextmv/input.py

def load(
    self,
    input_format: ContentFormat = ContentFormat.JSON,
    options: Options | None = None,
    *args,
    **kwargs,
) -> Input:
    """
    Read the input data. This method should be implemented by
    subclasses.

    Parameters
    ----------
    input_format : ContentFormat, optional
        Format of the input data. Default is `ContentFormat.JSON`.
    options : Options, optional
        Options for loading the input data.
    *args
        Additional positional arguments.
    **kwargs
        Additional keyword arguments.

    Returns
    -------
    Input
        The input data.

    Raises
    ------
    NotImplementedError
        If the method is not implemented.
    """

    raise NotImplementedError

LocalInputLoader ¶

Bases: InputLoader

Class for loading local inputs.

You can import the LocalInputLoader class directly from nextmv:

from nextmv import LocalInputLoader

This class can load input data from the local filesystem, by using stdin, a file, or a directory, where applicable. It supports JSON and multi-file formats.

Call the load method to read the input data.

Examples:

>>> from nextmv.input import LocalInputLoader, ContentFormat
>>> loader = LocalInputLoader()
>>> # Load JSON from stdin or file
>>> input_obj = loader.load(input_format=ContentFormat.JSON, path="data.json")

FILE_READERS `class-attribute` `instance-attribute` ¶

FILE_READERS = {
    JSON: _read_json,
    TEXT: _read_text,
    "CSV": _read_csv,
}

Dictionary of functions to read from files.

Each key is a ContentFormat, and each value is a function that reads from a file in that format.

STDIN_READERS `class-attribute` `instance-attribute` ¶

STDIN_READERS = {
    JSON: lambda _: load(stdin),
    TEXT: lambda _: rstrip("\n"),
}

Dictionary of functions to read from standard input.

Each key is a ContentFormat, and each value is a function that reads from standard input in that format.

load ¶

load(
    input_format: ContentFormat | None = None,
    options: Options | None = None,
    path: str | None = None,
    csv_configurations: dict[str, Any] | None = None,
    data_files: list[DataFile] | None = None,
    manifest: Manifest | None = None,
) -> Input

Load the input data. For ContentFormat.JSON, the data can be streamed from stdin or read from a file. When the path argument is provided (and valid), the input data is read from the file specified by path, otherwise, it is streamed from stdin.

The Input that is returned contains the data attribute. This data can be of different types, depending on the provided input_format:

ContentFormat.JSON: the data is a dict[str, Any].
ContentFormat.MULTI_FILE: the data is a dict[str, Any], where each key is the file name (with extension) and the value is the data read from the file. The data can be of any type, depending on the file type and the reader function provided in the DataFile instances.

PARAMETER	DESCRIPTION
`input_format` ¶	Format of the input data. Default is `ContentFormat.JSON`. TYPE: `ContentFormat` DEFAULT: `None`
`options` ¶	Options for loading the input data. TYPE: `Options` DEFAULT: `None`
`path` ¶	Path to the input data. TYPE: `str` DEFAULT: `None`
`csv_configurations` ¶	Configurations for loading CSV files. The default `DictReader` is used when loading a CSV file, so you have the option to pass in a dictionary with custom kwargs for the `DictReader`. TYPE: `dict[str, Any]` DEFAULT: `None`
`data_files` ¶	List of `DataFile` instances to read from. This is used when the `input_format` is set to `ContentFormat.MULTI_FILE`. Each `DataFile` instance should have a `name` (the file name with extension) and a `loader` function that reads the data from the file. The `loader` function should accept the file path as its first argument and return the data read from the file. The `loader` can also accept additional positional and keyword arguments, which can be provided through the `loader_args` and `loader_kwargs` attributes of the `DataFile` instance. TYPE: `list[DataFile]` DEFAULT: `None`
`manifest` ¶	The manifest to use for loading the input data. If not provided, the method will look for an `app.yaml` file in the current working directory and use it as the manifest if it exists. The manifest can provide default values for `input_format`, `path`, and `options`, which can be overridden by the corresponding arguments if they are provided directly to the method. TYPE: `Manifest` DEFAULT: `None`

RETURNS	DESCRIPTION
`Input`	The input data.

RAISES	DESCRIPTION
`ValueError`	If the path is not a valid directory.

Source code in nextmv/nextmv/input.py

def load(  # noqa: C901
    self,
    input_format: ContentFormat | None = None,
    options: Options | None = None,
    path: str | None = None,
    csv_configurations: dict[str, Any] | None = None,
    data_files: list[DataFile] | None = None,
    manifest: Manifest | None = None,
) -> Input:
    """
    Load the input data. For `ContentFormat.JSON`, the data can be streamed
    from stdin or read from a file. When the `path` argument is provided
    (and valid), the input data is read from the file specified by `path`,
    otherwise, it is streamed from stdin.

    The `Input` that is returned contains the `data` attribute. This data
    can be of different types, depending on the provided `input_format`:

    - `ContentFormat.JSON`: the data is a `dict[str, Any]`.
    - `ContentFormat.MULTI_FILE`: the data is a `dict[str, Any]`, where
      each key is the file name (with extension) and the value is the data
      read from the file. The data can be of any type, depending on the
      file type and the reader function provided in the `DataFile`
      instances.

    Parameters
    ----------
    input_format : ContentFormat, optional
        Format of the input data. Default is `ContentFormat.JSON`.
    options : Options, optional
        Options for loading the input data.
    path : str, optional
        Path to the input data.
    csv_configurations : dict[str, Any], optional
        Configurations for loading CSV files. The default `DictReader` is
        used when loading a CSV file, so you have the option to pass in a
        dictionary with custom kwargs for the `DictReader`.
    data_files : list[DataFile], optional
        List of `DataFile` instances to read from. This is used when the
        `input_format` is set to `ContentFormat.MULTI_FILE`. Each
        `DataFile` instance should have a `name` (the file name with
        extension) and a `loader` function that reads the data from the
        file. The `loader` function should accept the file path as its
        first argument and return the data read from the file. The `loader`
        can also accept additional positional and keyword arguments, which
        can be provided through the `loader_args` and `loader_kwargs`
        attributes of the `DataFile` instance.
    manifest : Manifest, optional
        The manifest to use for loading the input data. If not provided,
        the method will look for an `app.yaml` file in the current working
        directory and use it as the manifest if it exists. The manifest can
        provide default values for `input_format`, `path`, and `options`,
        which can be overridden by the corresponding arguments if they are
        provided directly to the method.

    Returns
    -------
    Input
        The input data.

    Raises
    ------
    ValueError
        If the path is not a valid directory.
    """

    manifest = resolve_manifest(manifest)
    content_format = self.__resolve_content_format(input_format, manifest)
    path = self.__resolve_path(content_format=content_format, path=path, manifest=manifest)

    data: Any = None
    if csv_configurations is None:
        csv_configurations = {}

    if content_format == ContentFormat.JSON:
        data = self.__load_utf8_encoded(
            path=path,
            input_format=content_format,
            csv_configurations=csv_configurations,
        )
    elif content_format == ContentFormat.MULTI_FILE:
        if data_files is None:
            raise ValueError("`data_files` must be provided when input_format is `ContentFormat.MULTI_FILE`")

        if not isinstance(data_files, list):
            raise ValueError("`data_files` must be a list of `DataFile` instances")

        data = self.__load_multi_file(data_files=data_files, path=path)
    elif type(content_format) is InputFormat and content_format == InputFormat.TEXT:
        data = self.__load_utf8_encoded(
            path=path,
            input_format=content_format,
            csv_configurations=csv_configurations,
        )
    elif type(content_format) is InputFormat and content_format == InputFormat.CSV_ARCHIVE:
        data = self.__load_archive(path=path, csv_configurations=csv_configurations)

    options = self.__resolve_options(options, manifest)

    return Input(data=data, input_format=content_format, options=options)

csv_data_file ¶

csv_data_file(
    name: str,
    csv_configurations: dict[str, Any] | None = None,
    input_data_key: str | None = None,
) -> DataFile

This is a convenience function to create a DataFile that reads CSV data.

You can import the csv_data_file function directly from nextmv:

from nextmv import csv_data_file

PARAMETER	DESCRIPTION
`name` ¶	Name of the data file. You don't need to include the `.csv` extension. TYPE: `str`
`csv_configurations` ¶	CSV-specific configurations for reading the data. TYPE: `dict[str, Any]` DEFAULT: `None`
`input_data_key` ¶	A custom key to represent the data from this file. When using `ContentFormat.MULTI_FILE` as the `input_format` of the `Input`, the data from the file is loaded to the `.data` parameter of the `Input`. In that case, the type of `.data` is `dict[str, Any]`, where each key represents the file name (with extension) and the value is the data that is actually loaded from the file using the `loader` function. You can set a custom key to represent your file by using this attribute. TYPE: `str` DEFAULT: `None`

RETURNS	DESCRIPTION
`DataFile`	A `DataFile` instance that reads CSV data from a file with the given name.

Examples:

>>> from nextmv import csv_data_file
>>> data_file = csv_data_file("my_data")
>>> data = data_file.read()
>>> print(data)
[
    {"column1": "value1", "column2": "value2"},
    {"column1": "value3", "column2": "value4"}
]

Source code in nextmv/nextmv/input.py

def csv_data_file(
    name: str,
    csv_configurations: dict[str, Any] | None = None,
    input_data_key: str | None = None,
) -> DataFile:
    """
    This is a convenience function to create a `DataFile` that reads CSV data.

    You can import the `csv_data_file` function directly from `nextmv`:

    ```python
    from nextmv import csv_data_file
    ```

    Parameters
    ----------
    name : str
        Name of the data file. You don't need to include the `.csv` extension.
    csv_configurations : dict[str, Any], optional
        CSV-specific configurations for reading the data.
    input_data_key : str, optional
        A custom key to represent the data from this file.

        When using `ContentFormat.MULTI_FILE` as the `input_format` of the `Input`,
        the data from the file is loaded to the `.data` parameter of the `Input`.
        In that case, the type of `.data` is `dict[str, Any]`, where each key
        represents the file name (with extension) and the value is the data that is
        actually loaded from the file using the `loader` function. You can set a
        custom key to represent your file by using this attribute.

    Returns
    -------
    DataFile
        A `DataFile` instance that reads CSV data from a file with the given
        name.

    Examples
    --------
    >>> from nextmv import csv_data_file
    >>> data_file = csv_data_file("my_data")
    >>> data = data_file.read()
    >>> print(data)
    [
        {"column1": "value1", "column2": "value2"},
        {"column1": "value3", "column2": "value4"}
    ]
    """

    if not name.endswith(".csv"):
        name += ".csv"

    csv_configurations = csv_configurations or {}

    def loader(file_path: str) -> list[dict[str, Any]]:
        with open(file_path, encoding="utf-8") as f:
            return list(csv.DictReader(f, **csv_configurations))

    return DataFile(
        name=name,
        loader=loader,
        input_data_key=input_data_key,
    )

json_data_file ¶

json_data_file(
    name: str,
    json_configurations: dict[str, Any] | None = None,
    input_data_key: str | None = None,
) -> DataFile

This is a convenience function to create a DataFile that reads JSON data.

You can import the json_data_file function directly from nextmv:

from nextmv import json_data_file

PARAMETER	DESCRIPTION
`name` ¶	Name of the data file. You don't need to include the `.json` extension. TYPE: `str`
`json_configurations` ¶	JSON-specific configurations for reading the data. TYPE: `dict[str, Any]` DEFAULT: `None`
`input_data_key` ¶	A custom key to represent the data from this file. When using `ContentFormat.MULTI_FILE` as the `input_format` of the `Input`, the data from the file is loaded to the `.data` parameter of the `Input`. In that case, the type of `.data` is `dict[str, Any]`, where each key represents the file name (with extension) and the value is the data that is actually loaded from the file using the `loader` function. You can set a custom key to represent your file by using this attribute. TYPE: `str` DEFAULT: `None`

RETURNS	DESCRIPTION
`DataFile`	A `DataFile` instance that reads JSON data from a file with the given name.

Examples:

>>> from nextmv import json_data_file
>>> data_file = json_data_file("my_data")
>>> data = data_file.read()
>>> print(data)
{
    "key": "value",
    "another_key": [1, 2, 3]
}

Source code in nextmv/nextmv/input.py

def json_data_file(
    name: str,
    json_configurations: dict[str, Any] | None = None,
    input_data_key: str | None = None,
) -> DataFile:
    """
    This is a convenience function to create a `DataFile` that reads JSON data.

    You can import the `json_data_file` function directly from `nextmv`:

    ```python
    from nextmv import json_data_file
    ```

    Parameters
    ----------
    name : str
        Name of the data file. You don't need to include the `.json` extension.
    json_configurations : dict[str, Any], optional
        JSON-specific configurations for reading the data.
    input_data_key : str, optional
        A custom key to represent the data from this file.

        When using `ContentFormat.MULTI_FILE` as the `input_format` of the `Input`,
        the data from the file is loaded to the `.data` parameter of the `Input`.
        In that case, the type of `.data` is `dict[str, Any]`, where each key
        represents the file name (with extension) and the value is the data that is
        actually loaded from the file using the `loader` function. You can set a
        custom key to represent your file by using this attribute.

    Returns
    -------
    DataFile
        A `DataFile` instance that reads JSON data from a file with the given
        name.

    Examples
    --------
    >>> from nextmv import json_data_file
    >>> data_file = json_data_file("my_data")
    >>> data = data_file.read()
    >>> print(data)
    {
        "key": "value",
        "another_key": [1, 2, 3]
    }
    """

    if not name.endswith(".json"):
        name += ".json"

    json_configurations = json_configurations or {}

    def loader(file_path: str) -> dict[str, Any] | Any:
        with open(file_path, encoding="utf-8") as f:
            return json.load(f, **json_configurations)

    return DataFile(
        name=name,
        loader=loader,
        input_data_key=input_data_key,
    )

load ¶

load(
    input_format: ContentFormat | None = None,
    options: Options | None = None,
    path: str | None = None,
    csv_configurations: dict[str, Any] | None = None,
    loader: InputLoader | None = _LOCAL_INPUT_LOADER,
    data_files: list[DataFile] | None = None,
    manifest: Manifest | None = None,
) -> Input

Load input data using the specified loader.

You can import the load function directly from nextmv:

from nextmv import load

This is a convenience function for loading an Input object. By default, it uses the LocalInputLoader to load data from local sources like files or stdin. When working with the local filesystem, please consider the following.

There are two main ways in which you can use this function:

Having an app.yaml (app manifest) present in the current working directory (recommended). In this case, you just use the function as load(), and it will automatically resolve the input_format, path, and options from the manifest. Alternatively, you can also provide the manifest explicitly as the manifest argument.
Providing the input_format, path, and options arguments directly to the function.

When both a manifest can be used (via the current working directory or the manifest argument) and the input_format, path, or options arguments are provided directly, the function will prioritize using the provided arguments over the manifest values. This allows you to override the manifest values when needed, while still having the convenience of using the manifest for default configurations.

The input data can be in various formats and can be loaded from different sources depending on the loader:

ContentFormat.JSON: the data is a dict[str, Any]
ContentFormat.MULTI_FILE: the data is a dict[str, Any], where each key is the file name (with extension) and the value is the data read from the file. This is used for loading multiple files in a single input, where each file can be of different types (JSON, CSV, Excel, etc.). The data is loaded as a dict of items, where each item corresponds to a file and its content.

When working with input_format as ContentFormat.MULTI_FILE, the data_files argument must be provided. This argument is a list of DataFile instances, each representing a file to be read. Each DataFile instance should have a name (the file name with extension) and a loader function that reads the data from the file. The loader function should accept the file path as its first argument and return the data read from the file. The loader can also accept additional positional and keyword arguments, which can be provided through the loader_args and loader_kwargs attributes of the DataFile instance.

There are convenience functions that can be used to create DataFile classes, such as:

json_data_file: Creates a DataFile that reads JSON data.
csv_data_file: Creates a DataFile that reads CSV data.
text_data_file: Creates a DataFile that reads utf-8 encoded text data.

When working with data in other formats, such as Excel files, you are encouraged to create your own DataFile objects with your own implementation of the loader function. This allows you to read data from files in a way that suits your needs, while still adhering to the DataFile interface.

PARAMETER	DESCRIPTION
`input_format` ¶	Format of the input data. Default is `ContentFormat.JSON`. TYPE: `ContentFormat` DEFAULT: `None`
`options` ¶	Options for loading the input data. TYPE: `Options` DEFAULT: `None`
`path` ¶	Path to the input data. For file-based loaders: - If provided, reads from the specified file or directory - If None, reads from stdin (for JSON) or uses a default directory (for multi-file) TYPE: `str` DEFAULT: `None`
`csv_configurations` ¶	Configurations for loading CSV files. Custom kwargs for Python's `csv.DictReader`. TYPE: `dict[str, Any]` DEFAULT: `None`
`loader` ¶	The loader to use for loading the input data. Default is an instance of `LocalInputLoader`. TYPE: `InputLoader` DEFAULT: `_LOCAL_INPUT_LOADER`
`data_files` ¶	List of `DataFile` instances to read from. This is used when the `input_format` is set to `ContentFormat.MULTI_FILE`. Each `DataFile` instance should have a `name` (the file name with extension) and a `loader` function that reads the data from the file. The `loader` function should accept the file path as its first argument and return the data read from the file. The `loader` can also accept additional positional and keyword arguments, which can be provided through the `loader_args` and `loader_kwargs` attributes of the `DataFile` instance. There are convenience functions that can be used to create `DataFile` classes, such as `json_data_file`, `csv_data_file`, and `text_data_file`. When working with data in other formats, such as Excel files, you are encouraged to create your own `DataFile` objects with your own implementation of the `loader` function. This allows you to read data from files in a way that suits your needs, while still adhering to the `DataFile` interface. TYPE: `list[DataFile]` DEFAULT: `None`
`manifest` ¶	The manifest to use for loading the input data. If not provided, the function will look for an `app.yaml` file in the current working directory and use it as the manifest if it exists. The manifest can provide default values for `input_format`, `path`, and `options`, which can be overridden by the corresponding arguments if they are provided directly to the function. TYPE: `Manifest` DEFAULT: `None`

RETURNS	DESCRIPTION
`Input`	The loaded input data in an Input object.

RAISES	DESCRIPTION
`ValueError`	If one of many validations fails.

Examples:

>>> import nextmv
>>> input = nextmv.load()

Source code in nextmv/nextmv/input.py

def load(
    input_format: ContentFormat | None = None,
    options: Options | None = None,
    path: str | None = None,
    csv_configurations: dict[str, Any] | None = None,
    loader: InputLoader | None = _LOCAL_INPUT_LOADER,
    data_files: list[DataFile] | None = None,
    manifest: Manifest | None = None,
) -> Input:
    """
    Load input data using the specified loader.

    You can import the `load` function directly from `nextmv`:

    ```python
    from nextmv import load
    ```

    This is a convenience function for loading an `Input` object. By default,
    it uses the `LocalInputLoader` to load data from local sources like files
    or stdin. When working with the local filesystem, please consider the
    following.

    There are two main ways in which you can use this function:

    - Having an `app.yaml` (app manifest) present in the current working
      directory (recommended). In this case, you just use the function as
      `load()`, and it will automatically resolve the `input_format`, `path`,
      and `options` from the manifest. Alternatively, you can also provide the
      manifest explicitly as the `manifest` argument.
    - Providing the `input_format`, `path`, and `options` arguments directly to
      the function.

    When both a manifest can be used (via the current working directory or the
    `manifest` argument) and the `input_format`, `path`, or `options` arguments
    are provided directly, the function will prioritize using the provided
    arguments over the manifest values. This allows you to override the
    manifest values when needed, while still having the convenience of using
    the manifest for default configurations.

    The input data can be in various formats and can be loaded from different
    sources depending on the loader:

    - `ContentFormat.JSON`: the data is a `dict[str, Any]`
    - `ContentFormat.MULTI_FILE`: the data is a `dict[str, Any]`, where each
      key is the file name (with extension) and the value is the data read from
      the file. This is used for loading multiple files in a single input,
      where each file can be of different types (JSON, CSV, Excel, etc.). The
      data is loaded as a dict of items, where each item corresponds to a file
      and its content.

    When working with `input_format` as `ContentFormat.MULTI_FILE`, the
    `data_files` argument must be provided. This argument is a list of
    `DataFile` instances, each representing a file to be read. Each `DataFile`
    instance should have a `name` (the file name with extension) and a `loader`
    function that reads the data from the file. The `loader` function should
    accept the file path as its first argument and return the data read from
    the file. The `loader` can also accept additional positional and keyword
    arguments, which can be provided through the `loader_args` and
    `loader_kwargs` attributes of the `DataFile` instance.

    There are convenience functions that can be used to create `DataFile`
    classes, such as:

    - `json_data_file`: Creates a `DataFile` that reads JSON data.
    - `csv_data_file`: Creates a `DataFile` that reads CSV data.
    - `text_data_file`: Creates a `DataFile` that reads utf-8 encoded text
      data.

    When working with data in other formats, such as Excel files, you are
    encouraged to create your own `DataFile` objects with your own
    implementation of the `loader` function. This allows you to read data
    from files in a way that suits your needs, while still adhering to the
    `DataFile` interface.

    Parameters
    ----------
    input_format : ContentFormat, optional
        Format of the input data. Default is `ContentFormat.JSON`.
    options : Options, optional
        Options for loading the input data.
    path : str, optional
        Path to the input data. For file-based loaders:
        - If provided, reads from the specified file or directory
        - If None, reads from stdin (for JSON) or uses a default directory
          (for multi-file)
    csv_configurations : dict[str, Any], optional
        Configurations for loading CSV files. Custom kwargs for
        Python's `csv.DictReader`.
    loader : InputLoader, optional
        The loader to use for loading the input data.
        Default is an instance of `LocalInputLoader`.
    data_files : list[DataFile], optional
        List of `DataFile` instances to read from. This is used when the
        `input_format` is set to `ContentFormat.MULTI_FILE`. Each `DataFile`
        instance should have a `name` (the file name with extension) and a
        `loader` function that reads the data from the file. The `loader`
        function should accept the file path as its first argument and return
        the data read from the file. The `loader` can also accept additional
        positional and keyword arguments, which can be provided through the
        `loader_args` and `loader_kwargs` attributes of the `DataFile`
        instance.

        There are convenience functions that can be used to create `DataFile`
        classes, such as `json_data_file`, `csv_data_file`, and
        `text_data_file`. When working with data in other formats, such as
        Excel files, you are encouraged to create your own `DataFile` objects
        with your own implementation of the `loader` function. This allows you
        to read data from files in a way that suits your needs, while still
        adhering to the `DataFile` interface.
    manifest : Manifest, optional
        The manifest to use for loading the input data. If not provided, the
        function will look for an `app.yaml` file in the current working
        directory and use it as the manifest if it exists. The manifest can
        provide default values for `input_format`, `path`, and `options`, which
        can be overridden by the corresponding arguments if they are provided
        directly to the function.

    Returns
    -------
    Input
        The loaded input data in an Input object.

    Raises
    ------
    ValueError
        If one of many validations fails.

    Examples
    --------
    >>> import nextmv
    >>> input = nextmv.load()
    """

    return loader.load(
        input_format,
        options,
        path,
        csv_configurations,
        data_files,
        manifest,
    )

text_data_file ¶

text_data_file(
    name: str, input_data_key: str | None = None
) -> DataFile

This is a convenience function to create a DataFile that reads utf-8 encoded text data.

You can import the text_data_file function directly from nextmv:

from nextmv import text_data_file

You must provide the extension as part of the name parameter.

PARAMETER DESCRIPTION

`name` ¶

Name of the data file. The file extension must be provided in the name.

TYPE: str

`input_data_key` ¶

A custom key to represent the data from this file.

When using ContentFormat.MULTI_FILE as the input_format of the Input, the data from the file is loaded to the .data parameter of the Input. In that case, the type of .data is dict[str, Any], where each key represents the file name (with extension) and the value is the data that is actually loaded from the file using the loader function. You can set a custom key to represent your file by using this attribute.

TYPE: str DEFAULT: None

RETURNS	DESCRIPTION
`DataFile`	A `DataFile` instance that reads text data from a file with the given name.

Examples:

>>> from nextmv import text_data_file
>>> data_file = text_data_file("my_data")
>>> data = data_file.read()
>>> print(data)
This is some text data.

Source code in nextmv/nextmv/input.py

def text_data_file(name: str, input_data_key: str | None = None) -> DataFile:
    """
    This is a convenience function to create a `DataFile` that reads utf-8
    encoded text data.

    You can import the `text_data_file` function directly from `nextmv`:

    ```python
    from nextmv import text_data_file
    ```

    You must provide the extension as part of the `name` parameter.

    Parameters
    ----------
    name : str
        Name of the data file. The file extension must be provided in the name.
    input_data_key : str, optional
        A custom key to represent the data from this file.

        When using `ContentFormat.MULTI_FILE` as the `input_format` of the `Input`,
        the data from the file is loaded to the `.data` parameter of the `Input`.
        In that case, the type of `.data` is `dict[str, Any]`, where each key
        represents the file name (with extension) and the value is the data that is
        actually loaded from the file using the `loader` function. You can set a
        custom key to represent your file by using this attribute.

    Returns
    -------
    DataFile
        A `DataFile` instance that reads text data from a file with the given
        name.

    Examples
    --------
    >>> from nextmv import text_data_file
    >>> data_file = text_data_file("my_data")
    >>> data = data_file.read()
    >>> print(data)
    This is some text data.
    """

    def loader(file_path: str) -> str:
        with open(file_path, encoding="utf-8") as f:
            return f.read().rstrip("\n")

    return DataFile(
        name=name,
        loader=loader,
        input_data_key=input_data_key,
    )

Input Module¶

input ¶

DataFile dataclass ¶

name ¶

reader ¶

input_data_key class-attribute instance-attribute ¶

loader instance-attribute ¶

loader_args class-attribute instance-attribute ¶

loader_kwargs class-attribute instance-attribute ¶

name instance-attribute ¶

INPUTS_KEY module-attribute ¶

Input dataclass ¶

data ¶

input_format ¶

options ¶

data instance-attribute ¶

input_format class-attribute instance-attribute ¶

options class-attribute instance-attribute ¶

to_dict ¶

InputLoader ¶

load ¶

input_format ¶

options ¶

*args ¶

**kwargs ¶

LocalInputLoader ¶

FILE_READERS class-attribute instance-attribute ¶

STDIN_READERS class-attribute instance-attribute ¶

load ¶

input_format ¶

options ¶

path ¶

csv_configurations ¶

data_files ¶

manifest ¶

csv_data_file ¶

name ¶

csv_configurations ¶

input_data_key ¶

json_data_file ¶

name ¶

json_configurations ¶

input_data_key ¶

load ¶

input_format ¶

options ¶

path ¶

csv_configurations ¶

loader ¶

data_files ¶

manifest ¶

text_data_file ¶

name ¶

input_data_key ¶

DataFile `dataclass` ¶

`name` ¶

`reader` ¶

input_data_key `class-attribute` `instance-attribute` ¶

loader `instance-attribute` ¶

loader_args `class-attribute` `instance-attribute` ¶

loader_kwargs `class-attribute` `instance-attribute` ¶

name `instance-attribute` ¶

INPUTS_KEY `module-attribute` ¶

Input `dataclass` ¶

`data` ¶

`input_format` ¶

`options` ¶

data `instance-attribute` ¶

input_format `class-attribute` `instance-attribute` ¶

options `class-attribute` `instance-attribute` ¶

`input_format` ¶

`options` ¶

`*args` ¶

`**kwargs` ¶

FILE_READERS `class-attribute` `instance-attribute` ¶

STDIN_READERS `class-attribute` `instance-attribute` ¶

`input_format` ¶

`options` ¶

`path` ¶

`csv_configurations` ¶

`data_files` ¶

`manifest` ¶

`name` ¶

`csv_configurations` ¶

`input_data_key` ¶

`name` ¶

`json_configurations` ¶

`input_data_key` ¶

`input_format` ¶

`options` ¶

`path` ¶

`csv_configurations` ¶

`loader` ¶

`data_files` ¶

`manifest` ¶

`name` ¶

`input_data_key` ¶