Reinforcement Learning (RL)

Overview

The Reinforcement Learning (RL) module (nala.athomic.ai.rl) provides a robust, production-ready framework for implementing adaptive decision engines. It allows the AI system to learn from interactions, optimize choices based on feedback (rewards), and dynamically adjust its behavior over time.

Unlike traditional heuristic rules, this module leverages mathematical policies to balance Exploration (trying new actions to discover better outcomes) and Exploitation (choosing the best-known action to maximize immediate reward).

Scope — read this first. The RLAlgorithmProtocol abstraction is RL-generic, but the algorithms shipped today are contextual / multi-armed bandits, not classical Reinforcement Learning. EpsilonGreedyBandit is a stateless multi-armed bandit; ContextualBandit is an online linear model trained via SGD on the context vector. Neither models temporal dynamics — there is no Bellman equation, discount factor γ, TD-error, state-dependent value/Q-function, episodes, or bootstrapping. In practice this module is an Adaptive Decision Engine based on contextual bandits and online incremental learning, which is the right tool for recommendation, routing, tool/agent selection, and controlled exploration. Evolving to full RL (Q-Learning → Double Q-Learning → DQN → PPO) is additive on the existing abstraction. See ADR-139.

Key Features

Stateless Architecture: Ensures absolute thread safety and zero race conditions in high-concurrency asynchronous environments. The mathematical core is pure and decoupled from persistence.
Pluggable Algorithms: Easily switch between multi-armed bandit strategies (e.g., Epsilon-Greedy, Contextual Bandit) via configuration without altering business logic.
Dual-Strategy Training: Supports both Online (synchronous, real-time weight updates) and Batch (asynchronous, offline Experience Replay) learning strategies to optimize latency versus accuracy.
Dual-Store Memory Facade: Intelligently splits persistence—fast Key-Value stores (Redis) for mathematical weights, and Vector/Document stores (Qdrant) for rich Experience Replay traces.
Full Observability: Deeply instrumented with OpenTelemetry traces and Prometheus metrics tracking exploration rates, reward distributions, and operation latency.

Architecture

The RL module strictly adheres to the Strategy and Template Method patterns, isolating mathematics, persistence, and execution flow.

Service Layer (ReinforcementLearningService): The orchestrator. It manages the lifecycle and delegates operations between the algorithm, the memory, and the trainer. It holds no mathematical state.
Mathematical Layer (RLAlgorithmProtocol): Pure, stateless implementations of RL policies. They receive their state (weights) via arguments, compute the next action or the updated weights, and return them.
- Epsilon-Greedy: Simple, effective multi-armed bandit.
- Contextual Bandit: Uses Stochastic Gradient Descent (SGD) and state vectors to make context-aware decisions.
Persistence Layer (RLMemoryService): A facade over the database manager.
- Saves/Loads weights using the KV Store (KVStoreProtocol).
- Stores/Samples experiences (State, Action, Reward, Next State) using the Vector Store (VectorStoreProtocol).
Trainer Layer (RLTrainingStrategy): Defines when and how the algorithm updates its weights.
- Online: Updates weights immediately during the user request (blocks until saved).
- Batch: Saves the experience instantly (low latency) and defers weight updates to an offline background job via train_offline.

Usage Example

Basic Online Learning (Epsilon-Greedy)

This example demonstrates how to use the RL service to make a decision and immediately provide feedback.

```python import uuid from nala.athomic.ai.rl.factory import ReinforcementLearningFactory

async def optimize_ui_layout(): # 1. Initialize the RL Service (automatically wires DBs based on settings) rl_service = ReinforcementLearningFactory.create() await rl_service.connect()

# The unique identifier for this specific learning context
model_id = "homepage_banner_optimization"

# 2. Get an Action (Predict)
# State vectors are optional for Epsilon-Greedy but required for Contextual Bandits
action, metadata = await rl_service.get_action(
    model_id=model_id,
    state_vector=[1.0], 
    available_actions=[1, 2, 3] # e.g., 3 different banner designs
)

print(f"Selected Banner ID: {action} (Explored: {metadata.get('exploration_triggered')})")

# ... application displays banner and waits for user interaction ...

# 3. Process Feedback (Update)
# E.g., The user clicked the banner (Reward = 1.0) or ignored it (Reward = 0.0)
user_clicked = True 
reward = 1.0 if user_clicked else 0.0

await rl_service.process_feedback(
    model_id=model_id,
    state_vector=[1.0],
    action_index=action,
    reward=reward,
    next_state_vector=[1.0]
)

```

Offline Batch Training (Experience Replay)

For high-throughput systems, updating mathematical weights on every request adds unacceptable latency. Use the Batch trainer to simply record interactions and train later.

```python

Assuming settings.toml defines trainer_type = "batch"

During the live user request:

await rl_service.process_feedback( model_id="recommendation_engine", state_vector=[0.5, 0.8, 0.1], # User context action_index=5, # Recommended item reward=10.0, # User bought the item next_state_vector=[0.9, 0.8, 0.1] )

Returns almost instantly. Weights are NOT updated yet.

... later, in a background Celery/Taskiq worker ...

async def nightly_model_training(): await rl_service.trigger_offline_training( model_id="recommendation_engine", batch_size=1024 # Sample a random batch from the Vector Store and apply SGD ) ```

Configuration

RL is configured under [ai.rl] in settings.toml. It relies on the global database configurations for KV and Vector stores.

```toml [default.ai.rl] enabled = true

The mathematical policy. Options: "epsilon_greedy", "contextual_bandit"

default_algorithm = "epsilon_greedy"

The execution strategy. Options: "online", "batch"

trainer_type = "online"

--- Algorithm Hyperparameters ---

[default.ai.rl.algorithm_parameters]

How heavily new rewards override old knowledge (0.0 to 1.0)

learning_rate = 0.1

The probability of choosing a random action to discover new paths (0.0 to 1.0)

exploration_rate = 0.2 discount_factor = 0.9

--- Infrastructure Bindings ---

[default.ai.rl.memory]

Must match a connection defined in [database.kvstore]

kv_store_connection_name = "default_redis"

Must match a connection defined in [database.vector]

vector_store_connection_name = "default_qdrant"

Namespaces to isolate RL data from other application caches

weights_namespace = "nala:ai:rl:weights" experience_collection_name = "nala_rl_experiences" ```

Observability

The RL Service automatically exports metrics to Prometheus to monitor the agent's learning progress and system health.

Metric Name	Type	Labels	Description
`rl_exploration_total`	Counter	`algorithm`	Number of times the agent chose a random action (Exploration) vs known best (Exploitation).
`rl_reward_distribution`	Histogram	`algorithm`	The distribution of rewards received, useful to detect if the environment is too sparse.
`rl_operations_total`	Counter	`algorithm`, `operation`, `status`	Total throughput (predict, update) and success/failure rates.
`rl_operation_duration_seconds`	Histogram	`algorithm`, `operation`, `status`	Latency overhead of predictions and weight updates.
`ai_rl_model_decisions_total`	Counter	`model_id`, `outcome`	Decisions served per model, split by `explore`/`exploit`. Exploration ratio = `explore / (explore + exploit)`. Emitted by the serving plane.
`ai_rl_model_reward`	Histogram	`model_id`	Per-model reward stream. `_sum` = cumulative reward (learning-progress proxy); `rate(_sum)/rate(_count)` = reward moving average; bucket spread ≈ variance. Emitted by the training plane.

On cumulative regret: true regret needs the optimal arm's expected reward, which is not available at runtime, so it is intentionally not emitted as a fabricated metric. Cumulative reward (ai_rl_model_reward_sum) is the honest proxy; a real regret baseline is deferred to shadow-mode execution (backlog RL-006).

Useful PromQL

# Per-model exploration ratio over 5m
sum(rate(ai_rl_model_decisions_total{outcome="explore"}[5m])) by (model_id)
  / sum(rate(ai_rl_model_decisions_total[5m])) by (model_id)

# Per-model reward moving average over 5m (is the agent learning?)
sum(rate(ai_rl_model_reward_sum[5m])) by (model_id)
  / sum(rate(ai_rl_model_reward_count[5m])) by (model_id)

API Reference

`nala.athomic.ai.rl.service.ReinforcementLearningService`

Bases: BaseService

Lifecycle owner and composition root for the RL module.

Bootstraps the memory backend and exposes three focused planes: - serving: hot-path action prediction - training: feedback ingestion and offline training - releases: weight version lifecycle (list / current / rollback) — RL-007

Backward-compat delegation methods keep existing callers working unchanged.

`nala.athomic.ai.rl.factory.ReinforcementLearningFactory`

Bases: FactoryProtocol[ReinforcementLearningService, RLSettings]

Root factory for the Reinforcement Learning module.

Responsible for assembling the mathematical algorithm, the memory facade, and the training strategy into the unified ReinforcementLearningService.

`create(settings, *, lock_provider=None, **kwargs)` `classmethod`

Instantiates the complete Reinforcement Learning orchestration service.

Parameters:

Name	Type	Description	Default
`settings`	`RLSettings`	Configuration for the RL module. Must be provided by the caller.	required
`lock_provider`	`Optional[LockingProtocol]`	Distributed lock guarding the trainers' weight load→update→save cycle (RL-001). Injected by the composition root; when `None` the cycle runs unguarded (single-process fallback). Kept explicit — not bundled into `**kwargs` — so it routes to the trainer and never reaches the memory provider.	`None`
`**kwargs`	`Any`	Dependencies forwarded to the memory provider's `build`. Storage: `connection_manager` (or pre-built `kv_store`/`vector_store`) — required by the default `kv_vector` provider, whose `build` raises without them. Resilience: optional `kv_resilience`/`vector_resilience` (`ResilienceOrchestrator`) guarding KV/Vector persistence round-trips with circuit breaker + exponential-backoff retry (RL-002). These ride `kwargs` — not explicit params — because, unlike `lock_provider`, they belong to the memory provider, which is exactly where `kwargs` is routed.	`{}`

Returns:

Name	Type	Description
`ReinforcementLearningService`	`ReinforcementLearningService`	The assembled RL orchestrator.

`nala.athomic.ai.rl.algorithms.protocol.RLAlgorithmProtocol`

Bases: Protocol

Defines the strict mathematical interface for Reinforcement Learning algorithms.

This protocol ensures that the learning core remains isolated from domain-specific business logic. Implementations must operate exclusively on numerical vectors and discrete indices in a completely stateless manner, receiving their execution state (weights) per invocation.

`evaluate(weights, transitions)`

Returns a read-only offline score of weights over a held-out batch of transitions — higher is better (RL-007 pre-promote eval gate).

weights is typed as the opaque JSON blob it is (Dict[str, object], not Dict[str, Any]): the algorithm narrows the few entries it reads.

The score is a value-calibration proxy: how well weights predict the observed rewards of the held-out transitions. Like weight_health, the algorithm owns it — it is the only component that knows how to read a prediction out of its weight layout. The computation is pure and must not mutate weights or fit on the batch (it is an evaluation, not a training step), so the same batch can score two weight sets fairly.

Parameters:

Name	Type	Description	Default
`weights`	`Dict[str, object]`	The weight set to score.	required
`transitions`	`Sequence[Transition]`	The held-out `(s, a, r, s')` batch to score against.	required

Returns:

Name	Type	Description
`float`	`float`	The score (higher is better). An empty batch yields `0.0`
	`float`	(no evidence ⇒ a neutral score).

`predict(weights, state_vector, available_actions)` `async`

Calculates the probability distribution over the action space and selects the optimal action index according to the underlying mathematical policy.

Parameters:

Name	Type	Description	Default
`weights`	`Dict[str, Any]`	The current mathematical state of the algorithm.	required
`state_vector`	`List[float]`	A numerical embedding representing the current environment state.	required
`available_actions`	`List[int]`	A list of integer indices representing valid choices.	required

Returns:

Type	Description
`int`	Tuple[int, Dict[str, Any]]: The selected action index and an extensible
`Dict[str, Any]`	metadata dictionary for algorithm-specific telemetry (e.g., confidence,
`Tuple[int, Dict[str, Any]]`	exploration flags).

`update(weights, state_vector, action_index, reward, next_state_vector)` `async`

Calculates the new internal weights or action-value approximations based on the observed scalar reward and the previous state.

Parameters:

Name	Type	Description	Default
`weights`	`Dict[str, Any]`	The current mathematical state of the algorithm before the update.	required
`state_vector`	`List[float]`	The state numerical representation before the action was taken.	required
`action_index`	`int`	The index of the action executed.	required
`reward`	`float`	The scalar feedback signal received from the environment.	required
`next_state_vector`	`List[float]`	The state numerical representation after the action execution — the `s'` of the persisted `(s, a, r, s')` replay record. The shipped bandit algorithms do not bootstrap, so they ignore it; it is kept on the contract (CR-DESIGN-03 seam) as the `s'` a temporal-difference learner will consume, so adding one needs no re-plumbing or experience re-collection.	required

Returns:

Type	Description
`Dict[str, Any]`	Tuple[Dict[str, Any], float]: the newly calculated weights, and the
`float`	scalar TD-error of this transition (e.g. `reward - prediction`).
`Tuple[Dict[str, Any], float]`	The error is surfaced — not discarded — so prioritized experience
`Tuple[Dict[str, Any], float]`	replay can weight this transition by how surprising it was (RL-004).

`validate_weights(weights)`

Validates a raw weights blob against this algorithm's structural schema and returns a normalized, JSON-safe copy (RL-008).

Called at the (de)serialization boundary of the weight read-modify-write cycle — right after a blob is loaded from storage and right before it is persisted — so a structurally corrupt blob (wrong value types, ragged contextual vectors, …) can never be silently written, snapshotted, or fed into a learning step. Like weight_health, the algorithm owns this: it is the only component that knows which keys its weights carry and what shape they must have. Sibling keys the blob carries beyond the learnable parameters are preserved.

Parameters:

Name	Type	Description	Default
`weights`	`Dict[str, Any]`	The raw weights mapping to validate.	required

Returns:

Type	Description
`Dict[str, Any]`	Dict[str, Any]: A normalized, JSON-safe copy of `weights`.

Raises:

Type	Description
`WeightSchemaError`	If the blob violates the algorithm's schema.

`weight_health(weights)`

Returns a structure-aware health snapshot of the learnable parameters held in weights (RL-005).

The algorithm is the only component that knows which entries of its weights dict are learnable parameters (e.g. a Q-table's values, a bandit's per-action vectors) versus bookkeeping (e.g. visit counts), so it owns this computation. It stays a pure, model-agnostic function — the trainer attaches the model_id and exports the snapshot as metrics.

Parameters:

Name	Type	Description	Default
`weights`	`Dict[str, Any]`	The algorithm's current mathematical state.	required

Returns:

Name	Type	Description
`WeightHealth`	`WeightHealth`	L2 norm, max absolute weight, and parameter count of
	`WeightHealth`	the learnable parameters (all-zero for a cold-start, parameter-less
	`WeightHealth`	model).

`nala.athomic.ai.rl.trainers.protocol.RLTrainingStrategy`

Bases: Protocol

Defines the Strategy pattern contract for Reinforcement Learning training cycles.

This protocol establishes the required methods for any training orchestration strategy. Implementations are responsible for bridging the persistence layer (Memory) with the stateless mathematical layer (Algorithm), ensuring that weights are loaded, updated, and saved without retaining internal state.

`process_experience(model_id, algorithm, memory, state_vector, action_index, reward, next_state_vector)` `async`

Processes a single interaction step received from the environment.

Implementations must retrieve the current weights from memory, compute the new weights via the stateless algorithm, and persist the updated state and experience back to the memory facade.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`algorithm`	`RLAlgorithmProtocol`	The stateless mathematical learning engine protocol.	required
`memory`	`RLMemoryProtocol`	The persistence facade protocol for weights and experiences.	required
`state_vector`	`List[float]`	The numerical state representation before the action.	required
`action_index`	`int`	The executed action index.	required
`reward`	`float`	The scalar feedback signal received.	required
`next_state_vector`	`List[float]`	The numerical state representation after the action.	required

`train_offline(model_id, algorithm, memory, batch_size=32)` `async`

Triggers an offline training cycle using historical experiences.

Implementations must load the current weights and sample experiences from memory, iteratively update the weights using the stateless algorithm, and save the final computed weights.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`algorithm`	`RLAlgorithmProtocol`	The stateless mathematical learning engine protocol.	required
`memory`	`RLMemoryProtocol`	The persistence facade protocol for weights and experiences.	required
`batch_size`	`int`	The number of historical experiences to sample and process.	`32`

Returns:

Name	Type	Description
`bool`	`bool`	Whether new weights were promoted this cycle — `True` when a
	`bool`	candidate was saved + snapshotted, `False` when there was nothing to
	`bool`	train on or the pre-promote eval gate rejected the candidate (RL-007).
	`bool`	The post-deploy reward monitor uses this to anchor its baseline at a
	`bool`	real promotion.

`nala.athomic.ai.rl.memory.protocol.RLMemoryProtocol`

Bases: BaseServiceProtocol, Protocol

Defines the persistence and lifecycle contract for RL memory backends.

Inherits BaseServiceProtocol so callers can manage lifecycle (connect, wait_ready, close, is_ready) without defensive hasattr checks.

`get_current_version(model_id)` `async`

Returns the id of the model's current (active) weight version (RL-007).

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required

Returns:

Type	Description
`Optional[str]`	Optional[str]: The current version id, or `None` if unversioned.

`list_weight_versions(model_id)` `async`

Returns the retained weight versions for a model, newest first (RL-007).

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required

Returns:

Type	Description
`List[WeightVersionInfo]`	List[WeightVersionInfo]: The retained version index (empty when none).

`load_norm_stats(model_id)` `async`

Retrieves the persisted observation-normalization statistics (RL-003).

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required

Returns:

Type	Description
`Optional[Dict[str, Any]]`	Optional[Dict[str, Any]]: The serialized normalization state if
`Optional[Dict[str, Any]]`	present, otherwise `None` (cold start — accumulate from scratch).

`load_weights(model_id)` `async`

Retrieves the previously saved parameters for a given model.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required

Returns:

Type	Description
`Optional[Dict[str, Any]]`	Optional[Dict[str, Any]]: The learned parameters if found, otherwise None.

`rollback_weights(model_id, version=None)` `async`

Restores a previous weight version as the model's active weights (RL-007).

Loads the target version's weights and writes them back as the active weights, then repoints current to it (the version rolled back from becomes previous, so a forward roll is possible).

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`version`	`Optional[str]`	The version id to restore. Defaults to `previous` (the last-known-good before the most recent snapshot).	`None`

Returns:

Type	Description
`Optional[str]`	Optional[str]: The version restored, or `None` when there is no
`Optional[str]`	valid target (no previous version / unknown version id).

`sample_experiences(model_id, batch_size)` `async`

Retrieves a batch of historical experiences for offline training.

Sampling contract (CR-DESIGN-02). The default semantics are a uniform i.i.d. draw — experience replay exists to decorrelate the samples a learner trains on. A backend may offer non-uniform modes (prioritized, or semantic similarity on vector backends), but only when explicitly configured via RLMemorySettings.sampling_mode; a backend must never silently substitute a biased draw for the uniform default. Implementations also honor max_experience_age_seconds (stale transitions are excluded from the batch without being evicted).

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`batch_size`	`int`	The number of experiences to retrieve.	required

Returns:

Type	Description
`List[StoredExperience]`	List[StoredExperience]: The sampled batch (empty when no experiences
`List[StoredExperience]`	exist). Each record carries its `id`/`timestamp`/`priority` so
`List[StoredExperience]`	callers can replay the transition and (PR ③) write a TD-error back.

`save_norm_stats(model_id, stats)` `async`

Persists the observation-normalization statistics for a model (RL-003).

Co-located with weights here because both are per-model_id learned state that must survive restarts and be shared across horizontally-scaled pods. The payload is the serialized NormalizationState (running moments + extrema); it is opaque to the backend.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`stats`	`Dict[str, Any]`	The serialized normalization state (JSON-safe mapping).	required

`save_weights(model_id, weights)` `async`

Persists the current state of the algorithm's learned parameters.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`weights`	`Dict[str, Any]`	A dictionary containing the Q-Table or neural network weights.	required

`snapshot_weights(model_id, weights, metadata)` `async`

Captures weights as a new immutable, retained version (RL-007).

Called after a training cycle so a model can later be rolled back to a known-good state. Assigns a monotonic version id (v1, v2, …), promotes it to current (the prior current becomes previous), and evicts the oldest versions beyond max_weight_versions. A no-op returning None when weight versioning is disabled (max_weight_versions <= 0).

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`weights`	`Dict[str, Any]`	The weight blob to snapshot (an immutable copy is retained).	required
`metadata`	`Optional[Dict[str, Any]]`	Optional JSON-safe annotations (e.g. weight-health, batch size) stored alongside the version.	required

Returns:

Type	Description
`Optional[str]`	Optional[str]: The new version id, or `None` when versioning is off.

`store_experience(model_id, state_vector, action_index, reward, next_state_vector)` `async`

Records a single interaction step for experience replay or batch training.

Parameters:

Name	Type	Description	Default
`model_id`	`str`	The unique identifier for the specific model instance.	required
`state_vector`	`List[float]`	The numerical state before the action.	required
`action_index`	`int`	The selected action.	required
`reward`	`float`	The resulting scalar reward.	required
`next_state_vector`	`List[float]`	The numerical state after the action.	required

`update_priorities(model_id, updates)` `async`

Updates the replay priority of specific stored experiences (RL-004 / PR ③).

updates is a list of (experience_id, priority) pairs — typically the absolute TD-error the trainer just computed for each sampled transition — so prioritized sampling revisits surprising transitions more often. Ids no longer in the buffer (aged out / evicted) are ignored. A no-op for backends that do not support prioritized replay.

Reinforcement Learning (RL)

Overview

Key Features

Architecture

Usage Example

Basic Online Learning (Epsilon-Greedy)

Offline Batch Training (Experience Replay)

Assuming settings.toml defines trainer_type = "batch"

During the live user request:

Returns almost instantly. Weights are NOT updated yet.

... later, in a background Celery/Taskiq worker ...

Configuration

The mathematical policy. Options: "epsilon_greedy", "contextual_bandit"

The execution strategy. Options: "online", "batch"

--- Algorithm Hyperparameters ---

How heavily new rewards override old knowledge (0.0 to 1.0)

The probability of choosing a random action to discover new paths (0.0 to 1.0)

--- Infrastructure Bindings ---

Must match a connection defined in [database.kvstore]

Must match a connection defined in [database.vector]

Namespaces to isolate RL data from other application caches

Observability

Useful PromQL

API Reference

nala.athomic.ai.rl.service.ReinforcementLearningService

nala.athomic.ai.rl.factory.ReinforcementLearningFactory

create(settings, *, lock_provider=None, **kwargs) classmethod

nala.athomic.ai.rl.algorithms.protocol.RLAlgorithmProtocol

evaluate(weights, transitions)

predict(weights, state_vector, available_actions) async

update(weights, state_vector, action_index, reward, next_state_vector) async

validate_weights(weights)

weight_health(weights)

nala.athomic.ai.rl.trainers.protocol.RLTrainingStrategy

process_experience(model_id, algorithm, memory, state_vector, action_index, reward, next_state_vector) async

train_offline(model_id, algorithm, memory, batch_size=32) async

nala.athomic.ai.rl.memory.protocol.RLMemoryProtocol

get_current_version(model_id) async

list_weight_versions(model_id) async

load_norm_stats(model_id) async

load_weights(model_id) async

rollback_weights(model_id, version=None) async

sample_experiences(model_id, batch_size) async

save_norm_stats(model_id, stats) async

save_weights(model_id, weights) async

snapshot_weights(model_id, weights, metadata) async

store_experience(model_id, state_vector, action_index, reward, next_state_vector) async

update_priorities(model_id, updates) async

`nala.athomic.ai.rl.service.ReinforcementLearningService`

`nala.athomic.ai.rl.factory.ReinforcementLearningFactory`

`create(settings, *, lock_provider=None, **kwargs)` `classmethod`

`nala.athomic.ai.rl.algorithms.protocol.RLAlgorithmProtocol`

`evaluate(weights, transitions)`

`predict(weights, state_vector, available_actions)` `async`

`update(weights, state_vector, action_index, reward, next_state_vector)` `async`

`validate_weights(weights)`

`weight_health(weights)`

`nala.athomic.ai.rl.trainers.protocol.RLTrainingStrategy`

`process_experience(model_id, algorithm, memory, state_vector, action_index, reward, next_state_vector)` `async`

`train_offline(model_id, algorithm, memory, batch_size=32)` `async`

`nala.athomic.ai.rl.memory.protocol.RLMemoryProtocol`

`get_current_version(model_id)` `async`

`list_weight_versions(model_id)` `async`

`load_norm_stats(model_id)` `async`

`load_weights(model_id)` `async`

`rollback_weights(model_id, version=None)` `async`

`sample_experiences(model_id, batch_size)` `async`

`save_norm_stats(model_id, stats)` `async`

`save_weights(model_id, weights)` `async`

`snapshot_weights(model_id, weights, metadata)` `async`

`store_experience(model_id, state_vector, action_index, reward, next_state_vector)` `async`

`update_priorities(model_id, updates)` `async`