Models

RxInferServer provides a flexible system for loading, managing, and exposing RxInfer probabilistic models through the API. This section explains the technical implementation details of how models work in the server, including the model dispatcher, loading process, and API integration.

For information about how to create and add new models, please refer to the How to Add a Model manual.

Model System Overview

The model system in RxInferServer consists of several key components:

RxInferServer.Models.ModelsDispatcher: Manages model discovery, loading, and access
RxInferServer.Models.LoadedModel: Represents a loaded model with its configuration and implementation
Model registry: Maintains a collection of available models
Hot-reloading system: Enables dynamic model updates during development

Model Discovery and Loading

The server discovers and loads models at startup using this process:

It scans all directories specified in RxInferServer.Models.RXINFER_SERVER_MODELS_LOCATIONS
For each directory, it looks for subdirectories that might contain models
In each subdirectory, it checks for model.jl and config.yaml files
If both files exist, it loads the model's configuration and code
The model is added to the server's model registry if successful

Models are accessed through a RxInferServer.Models.ModelsDispatcher which provides methods to retrieve models by name or list all available (non-private) models.

Hot-Reloading System

RxInferServer supports hot-reloading of models during development. When model files are modified:

The server detects the changes automatically
It reloads all models from their directories
The updated models become immediately available through the API

This feature is enabled by default during development and can be disabled through the server's configuration. See Hot-Reloading for more details.

API Integration

Models are exposed through the API endpoints defined in the OpenAPI specification. When a client requests model information or executes a model, the server:

Looks up the requested model by name using the dispatcher
If found, returns the model's metadata
Returns appropriate error responses if the model is not found or other issues occur

API Reference

RxInferServer.Models — Module

Models

Module responsible for loading, managing, and accessing RxInfer probabilistic models in the server. Handles model discovery, loading, and provides access to models through a dispatcher.

RxInferServer.Models.ModelsDispatcher — Type

ModelsDispatcher

A dispatcher that manages loaded models from specified locations. Responsible for model discovery, loading, and providing access to models.

Fields

locations::L: The locations where models are stored
models::M: Dictionary mapping model names to loaded model instances

RxInferServer.Models.LoadedModel — Type

LoadedModel

Represents a loaded RxInfer probabilistic model with its metadata and implementation.

Fields

path::String: Path to the model directory
name::String: Name of the model
description::String: Description of the model's purpose and functionality
author::String: Author or organization that created the model
roles::Vector{String}: List of roles that can access the model
config::Dict{String, Any}: Configuration parameters for the model
mod::Module: Julia module containing the model's implementation

RxInferServer.Models.get_models — Function

get_models(dispatcher::ModelsDispatcher; role = nothing)

Get all non-private models from the given dispatcher.

Arguments

dispatcher::ModelsDispatcher: The dispatcher to get models from
roles::Union{Vector{String}, Nothing}: The roles to filter models by (optional)

Returns

A collection of all non-private loaded models

get_models()

Get all non-private models using the current dispatcher.

Returns

A collection of all non-private loaded models

RxInferServer.Models.get_model — Function

get_model(dispatcher::ModelsDispatcher, model_name::String)

Get a specific model by name from the dispatcher.

Arguments

dispatcher::ModelsDispatcher: The dispatcher to get the model from
model_name::String: The name of the model to retrieve

Returns

LoadedModel or nothing: The requested model if found, otherwise nothing

get_model(model_name::String)

Get a specific model by name from the current dispatcher.

Arguments

model_name::String: The name of the model to retrieve

Returns

LoadedModel or nothing: The requested model if found, otherwise nothing

RxInferServer.Models.load_models! — Function

load_models!(models, locations)

Load models from the specified locations into the models dictionary.

Arguments

models: Dictionary to populate with loaded models (name => LoadedModel)
locations: List of directories to scan for models

Throws

ErrorException: If a location does not exist or if duplicate model names are found

RxInferServer.Models.reload! — Function

reload!(dispatcher::ModelsDispatcher)

Reload all models from the dispatcher's locations, updating the dispatcher's models. Used for hot-reloading models when their files change.

Arguments

dispatcher::ModelsDispatcher: The dispatcher to reload models for

Warning

This function completely replaces the models dictionary with newly loaded models, allowing for model updates, additions, and removals to be recognized. Indented for interactive use only.

RxInferServer.Models.with_models — Function

with_models(f::Function; locations = RXINFER_SERVER_MODELS_LOCATIONS())

Execute function f with an initialized models dispatcher for the given locations. Creates a scoped context where models can be accessed via the dispatcher.

Arguments

f::Function: The function to execute within the models context
locations: The locations to scan for models, defaults to RXINFER_SERVER_MODELS_LOCATIONS()

RxInferServer.Models.get_models_dispatcher — Function

get_models_dispatcher()::ModelsDispatcher

Get the current active models dispatcher. Must be called within a with_models context.

Returns

ModelsDispatcher: The active models dispatcher

Throws

ErrorException: If called outside of a with_models context

RxInferServer.Models.serialize_parameters — Function

serialize_parameters(parameters)

Serialize the given parameters to an opaque binary format.

RxInferServer.Models.serialize_state — Function

serialize_state(state)

Serialize the given state to an opaque binary format.

RxInferServer.Models.deserialize_state — Function

deserialize_state(state_buffer)

Deserialize the given state from an opaque binary format.

RxInferServer.Models.deserialize_parameters — Function

deserialize_parameters(parameters_buffer)

Deserialize the given parameters from an opaque binary format.

RxInferServer.Models.validate_model_config_header — Function

validate_model_config_header(config)

Validate the model config header. This includes checking that the config satisfies the model config schema. The function checks the existence of the following named keys:

name must be a string
description must be a string
author must be a string
roles must be an array of strings

Arguments

config: The model configuration

Returns

nothing: If the model configuration is valid
RxInferServer.Models.ModelConfigurationValidationError: If the model configuration is invalid

RxInferServer.Models.validate_model_config_arguments — Function

validate_model_config_required_arguments(config, arguments)

Validate the arguments from the model configuration. This includes checking that

the required arguments are present

Arguments

config: The model configuration
arguments: The arguments to validate

RxInferServer.Models.parse_model_config_default_arguments — Function

parse_model_config_default_arguments(config)

Parse the default arguments from the model configuration.

Arguments

config: The model configuration

Returns

Dict{String, Any}: The default arguments

RxInferServer.Models.ModelConfigurationValidationError — Type

ModelConfigurationValidationError

A custom error type for model configuration validation errors.