GitHub

Configuration

This section describes the configuration options for the RxInferServer.jl package. All configuration is done through environment variables, which can be set before starting the server. These settings are runtime configurations that do not require recompilation of the project.

Environment Configuration with .env files

The server supports loading environment variables from .env files. The specific files loaded depend on the server environment setting:

RxInferServer.RXINFER_SERVER_ENV — Function
RXINFER_SERVER_ENV() -> String

Controls which .env files are loaded by the server.

Returns the value of the RXINFER_SERVER_ENV environment variable, which determines the environment-specific configuration files to load. Defaults to an empty string when the environment variable is not set.

By default, values already set in the environment variable are not overridden by the .env files. Set RXINFER_SERVER_ENV_OVERRIDE to true to override existing values.

Files Loaded Based on Environment Value

Environment Value.env Files Loaded
"" (default).env.local, .env
"production".env.production.local, .env.production, .env
"development".env.development.local, .env.development, .env
"test".env.test.local, .env.test, .env
"local".env.local.local, .env.local, .env
Custom value "xyz".env.xyz.local, .env.xyz, .env

If the same key is defined in multiple files, the first occurrence of the key is used (e.g the .env.local file takes precedence over the .env file). However, if the RXINFER_SERVER_ENV_OVERRIDE environment variable is set to true, the last occurrence of the key is used instead (e.g the .env file takes precedence over the .env.local file).

All files are assumed to be in the directory specified by the RXINFER_SERVER_ENV_PWD environment variable. If this variable is not set, the current working directory (pwd()) is used instead.

Warning

Never store sensitive information in the .env files. The .env files are included in the repository by default to simplify the setup process. In a production environment, sensitive information should be stored in a more secure manner, such as in environment variables or in a secrets manager.

Note

RXINFER_SERVER_ENV, RXINFER_SERVER_ENV_PWD and RXINFER_SERVER_ENV_OVERRIDE cannot be set via the .env files as they are accessed before the .env files are loaded.

See also: RXINFER_SERVER_ENV_PWD, RXINFER_SERVER_ENV_OVERRIDE.

source
RxInferServer.RXINFER_SERVER_ENV_OVERRIDE — Function
RXINFER_SERVER_ENV_OVERRIDE() -> Bool

Returns the value of the RXINFER_SERVER_ENV_OVERRIDE environment variable. Defaults to false when the environment variable is not set.

If RXINFER_SERVER_ENV_OVERRIDE environment variable is set to true, values already set in the environment variable are overridden by the .env files.

Note

RXINFER_SERVER_ENV, RXINFER_SERVER_ENV_PWD and RXINFER_SERVER_ENV_OVERRIDE cannot be set via the .env files as they are accessed before the .env files are loaded.

See also: RXINFER_SERVER_ENV, RXINFER_SERVER_ENV_PWD.

source

Port Configuration

The server port can be configured using the following environment variable:

RxInferServer.RXINFER_SERVER_PORT — Function

The port on which the RxInfer server will run. This can be configured using the RXINFER_SERVER_PORT environment variable. Defaults to 8000 if not specified.

# Set port via environment variable
ENV["RXINFER_SERVER_PORT"] = 9000
RxInferServer.serve()
source

Models Configuration

The server supports models configuration. The following environment variables can be set to configure the models:

RxInferServer.Models.RXINFER_SERVER_MODELS_LOCATIONS — Function
RXINFER_SERVER_MODELS_LOCATIONS

The directories where the models are stored. Colon-separated list of directories. This can be configured using the RXINFER_SERVER_MODELS_LOCATIONS environment variable. Defaults to "models:custom_models" if not specified. Note that the custom_models directory is git-ignored by default. Use the custom_models directory to experiment with custom models without committing them to the repository.

ENV["RXINFER_SERVER_MODELS_LOCATIONS"] = "/path/to/models1:/path/to/models2"
RxInferServer.serve()
source
RxInferServer.Models.RXINFER_SERVER_LOAD_TEST_MODELS — Function
RXINFER_SERVER_LOAD_TEST_MODELS

Environment variable to determine whether to load test models. Can be either true or false. The test models are located under the test directory of the project.

Note

pkgdir(@__MODULE__) is used to locate the project's directory and load the test models from there. This means that the test models can be loaded only when the package is installed in the development mode. Do not use this variable in the production environment.

source

For detailed information about model configuration and implementation, including the structure of config.yaml files and model implementation requirements, please refer to the How to Add a Model manual.

Logging Configuration

The server implements a comprehensive logging system that writes logs both to the terminal and to files. Logs are organized by functional groups (e.g., Server, Authentication) and stored in separate files. The configurable options include:

RxInferServer.Logging.RXINFER_SERVER_LOGS_LOCATION — Function

The directory where server logs will be stored. This can be configured using the RXINFER_SERVER_LOGS_LOCATION environment variable. Defaults to ".server-logs" in the current working directory if not specified. The server automatically creates this directory if it doesn't exist.

The logging system uses:

  • Terminal output with formatted, human-readable logs
  • File-based logging with separate files for different functional groups
# Set logs directory via environment variable
ENV["RXINFER_SERVER_LOGS_LOCATION"] = "/path/to/logs"
RxInferServer.serve()
source
RxInferServer.Logging.RXINFER_SERVER_ENABLE_DEBUG_LOGGING — Function

Whether to enable debug logging. This can be configured using the RXINFER_SERVER_ENABLE_DEBUG_LOGGING environment variable. Defaults to "false" if not specified. Note that this is a string variable, not a boolean. If enabled, writes to debug.log in the RXINFERSERVERLOGS_LOCATION directory. Note that the debug logs are overwritten each time the server is restarted.

ENV["RXINFER_SERVER_ENABLE_DEBUG_LOGGING"] = "true"
RxInferServer.serve()
source
Note

For production deployments, consider setting a persistent, absolute path for your log files to ensure they are preserved and easily accessible for monitoring and debugging.

Note

make serve command runs the server with debug logging enabled.

MongoDB Configuration

The MongoDB connection can be configured using the following environment variables:

RxInferServer.Database.RXINFER_SERVER_MONGODB_URL — Function

The MongoDB connection URL for the server. This can be configured using the RXINFER_SERVER_MONGODB_URL environment variable. Defaults to mongodb://localhost:27017 if not specified.

# Set MongoDB URL via environment variable
ENV["RXINFER_SERVER_MONGODB_URL"] = "mongodb://user:password@host:port/database"
RxInferServer.serve()
source
RxInferServer.Database.RXINFER_SERVER_MONGODB_DATABASE — Function

The MongoDB database name to use. This can be configured using the RXINFER_SERVER_MONGODB_DATABASE environment variable. Defaults to rxinferserver if not specified.

# Set MongoDB database name via environment variable
ENV["RXINFER_SERVER_MONGODB_DATABASE"] = "my-database"
RxInferServer.serve()
source

The default connection URL for the Docker development environment is mongodb://database:27017, which connects to the MongoDB Atlas Local instance running in the Docker Compose environment. When deploying to production, you should set this to your actual MongoDB Atlas connection string or other MongoDB instance.

SSL Certificates for MongoDB

When connecting to MongoDB servers with SSL/TLS enabled, you'll need to provide appropriate certificate files. RxInferServer includes functionality to help find SSL certificates on your system.

Automatic Certificate Discovery

For remote MongoDB connections (not localhost), RxInferServer can automatically find and use SSL certificates from standard system locations. This behavior is controlled by the RXINFER_SERVER_SSL_CA_FILE environment variable:

RxInferServer.Database.RXINFER_SERVER_SSL_CA_FILE — Function

The SSL/TLS CA file path to use for MongoDB connections. This can be configured using the RXINFER_SERVER_SSL_CA_FILE environment variable. If not specified, the system will try to automatically find a suitable CA certificate.

# Set SSL CA file path via environment variable
ENV["RXINFER_SERVER_SSL_CA_FILE"] = "/path/to/ca.pem"
RxInferServer.serve()
source

See more details in the Database section.

Using MongoDB Compass

If you're using MongoDB Compass to connect to and manage your MongoDB database during development:

  1. Always connect to localhost:27017 from your host machine (except when using the RxInferServer.Database.RXINFER_SERVER_MONGODB_URL environment variable)
  2. This is because Docker maps the container's port 27017 to your host's port 27017
  3. No authentication is required for the development database by default

For production MongoDB Atlas connections in Compass, you would use the standard Atlas connection string format.

CORS Configuration

The server supports CORS configuration. The following environment variables can be set to configure the CORS settings:

Authentication Configuration

The server implements standard Bearer token authentication using the HTTP Authorization header. Most of the endpoints except for /generate-token and /ping require authentication by default.

When implementing client applications, you must include the token in the Authorization header with the Bearer prefix:

Authorization: Bearer your-token-here

A simplified authentication for development can be configured through the environment variable:

RxInferServer.RXINFER_SERVER_ENABLE_DEV_TOKEN — Function

An environment variable that can be used to enable development token authentication. By default, the development token is disabled and the RXINFER_SERVER_ENABLE_DEV_TOKEN environment variable set to "false". Set to "true" to enable development token authentication (do not use in production!). Note that RxInferServer checks this environment variable only once before starting the server.

# Use a specific development token
ENV["RXINFER_SERVER_ENABLE_DEV_TOKEN"] = "true"

# Disable development token (production mode)
ENV["RXINFER_SERVER_ENABLE_DEV_TOKEN"] = "false"

If enabled use dev-token as the development token. The development token has ["user"] roles by default. This, however, can be overriden by appending a different comma-separated list of roles directly into the development token after the : symbol. For example:

ENV["RXINFER_SERVER_ENABLE_DEV_TOKEN"] = "true"

# ...
# token used for making requests with extra roles
token = "dev-token:user,admin"
Warning

In production environments, you should always set RXINFER_SERVER_ENABLE_DEV_TOKEN=false. Failure to do so will make the development token available and usable by anyone, which leads to a huge potential security risk.

See also: check_dev_token

source
RxInferServer.check_dev_token — Function
check_dev_token(token::String)

Checks if the token provided is the development token. Additionally, parses the token to extract the roles from the token. Returns both the token and the roles if the token is the development token. Returns nothing if the token is not the development token.

See also: RXINFER_SERVER_ENABLE_DEV_TOKEN

source

This is particularly useful for development and testing.

Warning

Do not use the development token in production environments.

Server Edition

The server edition can be configured using the following environment variable:

RxInferServer.RXINFER_SERVER_EDITION — Function

The server edition identifier. This can be configured using the RXINFER_SERVER_EDITION environment variable. This setting is used to identify the server edition in the server information endpoint and has no functional impact on server behavior. Defaults to "CommunityEdition" if not specified.

# Set server edition via environment variable
ENV["RXINFER_SERVER_EDITION"] = "EnterpriseEdition"
RxInferServer.serve()
source

Other Configuration

RxInferServer.RXINFER_SERVER_SHOW_BANNER — Function

Whether to show the welcome banner. This can be configured using the RXINFER_SERVER_SHOW_BANNER environment variable. Defaults to "true" if not specified.

ENV["RXINFER_SERVER_SHOW_BANNER"] = "false"
RxInferServer.serve()
source
RxInferServer.RXINFER_SERVER_LISTEN_KEYBOARD — Function

Whether to listen for keyboard input to quit the server. This can be configured using the RXINFER_SERVER_LISTEN_KEYBOARD environment variable. Defaults to "true" if not specified. Defaults to "false" if "CI" environment variable is set to "true".

ENV["RXINFER_SERVER_LISTEN_KEYBOARD"] = "false"
RxInferServer.serve()
source

Hot Reloading

The server supports hot reloading, which automatically updates endpoints when code changes are detected. This feature is controlled by the following environment variable:

RxInferServer.RXINFER_SERVER_ENABLE_HOT_RELOAD — Function
RXINFER_SERVER_ENABLE_HOT_RELOAD()

Whether to enable hot reloading or not. This can be configured using the RXINFER_SERVER_ENABLE_HOT_RELOAD environment variable. Defaults to "true" if not specified, but Revise.jl must be loaded in the current Julia session for this functionality to work. If the setting is set to "true", but Revise.jl is not loaded, the server will not hot reload the source code and models when the source code changes and print a warning instead. Set to "false" to disable hot reloading even if Revise.jl is loaded.

using Revise

# Check current setting
ENV["RXINFER_SERVER_ENABLE_HOT_RELOAD"] = "true"

RxInferServer.serve()

See also: RxInferServer.is_hot_reload_enabled

source

Hot reloading consists of two separate mechanisms:

  • Source Code Hot-Reloading: Updates API endpoints and server code when changes are detected
  • Models Hot-Reloading: Refreshes models when their files change

Both mechanisms monitor files for changes using Revise.jl and automatically apply updates when detected. Console logs with the [HOT-RELOAD] prefix indicate reloading activity.

Note

Hot reloading requires Revise.jl to be loaded in the current Julia session. If Revise.jl is not loaded, hot reloading will be disabled even if enabled through the environment variable.

Warning

Hot reloading should be disabled in production environments as it can impact performance and may cause unexpected behavior.