AI & Semantic Memory Usage Guide

Overview

This guide provides practical recipes for leveraging the athomic.ai module. It covers common scenarios such as generating text with LLMs, extracting structured data, building RAG (Retrieval-Augmented Generation) pipelines using Semantic Memory, and enforcing governance policies.

---

Recipe 1: Chat & Text Generation

This recipe demonstrates the standard way to invoke an LLM (like OpenAI GPT-4 or Google Gemini) for text generation.

from nala.athomic.ai.llm.factory import LLMFactory
from nala.athomic.config import get_settings

async def generate_marketing_copy(product_name: str):
    # 1. Load configuration
    # Assumes 'default' is configured in settings.toml
    settings = get_settings().ai.llm.connections["default"]

    # 2. Create the provider instance via Factory
    # This automatically handles authentication and connection setup
    llm = LLMFactory.create(settings)

    # 3. Generate content
    # The 'system_message' sets the behavior of the model
    response = await llm.generate_content(
        prompt=f"Write a catchy tagline for {product_name}",
        system_message="You are a creative marketing expert.",
        temperature=0.9 # Higher creativity
    )

    return response

Key Concepts

• LLMFactory: Decouples your code from the specific provider class (OpenAI vs Vertex).
• generate_content: The unified method for unstructured text, automatically traced and measured.

---

Recipe 2: Structured Data Extraction

Extracting reliable JSON from an LLM is a common requirement. Athomic standardizes this using Pydantic models.

from pydantic import BaseModel, Field
from nala.athomic.ai.llm.factory import LLMFactory
from nala.athomic.config import get_settings

# 1. Define the desired output schema
class SupportTicketAnalysis(BaseModel):
    sentiment: str = Field(..., description="Customer sentiment: positive, neutral, negative")
    urgency: int = Field(..., description="Urgency level from 1 to 5")
    category: str = Field(..., description="Category: billing, technical, feature_request")

async def analyze_ticket(ticket_body: str):
    settings = get_settings().ai.llm.connections["default"]
    llm = LLMFactory.create(settings)

    # 2. Call generate_structured
    # This guarantees the return type is SupportTicketAnalysis or raises StructuredOutputError
    analysis = await llm.generate_structured(
        prompt=f"Analyze this ticket:n{ticket_body}",
        response_model=SupportTicketAnalysis,
        max_retries=2
    )

    if analysis.urgency >= 4:
        print(f"High urgency ticket detected! Category: {analysis.category}")

---

Recipe 3: Implementing Semantic Memory (RAG)

The SemanticMemoryService orchestrates the entire RAG flow: embedding text, storing it in the vector DB, and retrieving relevant context.

Storing Information (Indexing)

from nala.athomic.ai.memory.factory import SemanticMemoryFactory

async def learn_user_preferences(user_id: str, notes: list[str]):
    # 1. Initialize the service
    # 'user_context' is the logical collection name in the Vector DB
    memory_service = SemanticMemoryFactory.create(collection_name="user_context")

    # 2. Add memories in batch for efficiency
    # The service automatically handles embedding generation (Document Mode)
    await memory_service.add_batch(
        contents=notes,
        metadatas=[{"user_id": user_id, "source": "onboarding"}] * len(notes)
    )
    print("Preferences stored.")

Recalling Information (Retrieval)

async def chat_with_context(user_id: str, user_question: str):
    memory_service = SemanticMemoryFactory.create(collection_name="user_context")
    llm = LLMFactory.create(get_settings().ai.llm.connections["default"])

    # 1. Recall relevant memories
    # The service automatically embeds the query (Query Mode) and searches the DB
    results = await memory_service.recall(
        query=user_question,
        limit=3,
        filters={"user_id": user_id} # Tenancy isolation
    )

    # 2. Construct the context block
    context_str = "n".join([f"- {res.memory.content}" for res in results])

    # 3. Generate answer using context
    prompt = f"""
    Context information is below:
    {context_str}
    ---------------------
    Given the context, answer the question: {user_question}
    """

    return await llm.generate_content(prompt)

---

Recipe 4: Enforcing AI Governance (Rate Limits)

To prevent cost overruns or abuse, you can attach governance guards to your LLM provider.

from nala.athomic.ai.llm.factory import LLMFactory
from nala.athomic.ai.governance.guards import RateLimitGuard
from nala.athomic.config import get_settings

async def protected_generation(user_id: str, prompt: str):
    settings = get_settings().ai.llm.connections["default"]

    # 1. Define the guard
    # 'ai_standard_user' must be a policy defined in your RateLimiter configuration
    rate_limiter = RateLimitGuard(policy_name="ai_standard_user")

    # 2. Initialize LLM with the guard
    llm = LLMFactory.create(settings)
    llm.guards.append(rate_limiter)

    try:
        # 3. The check is executed automatically before generation
        # Pass 'user_id' in kwargs so the guard knows the scope
        return await llm.generate_content(prompt, user_id=user_id)

    except Exception as e:
        # Captures QuotaExceededError if limit is hit
        print(f"Request blocked by governance: {e}")

---

Best Practices

1. Asymmetric Embeddings

When working with embeddings directly (via EmbeddingFactory), always distinguish between documents and queries. * Use embed_documents() for text you are storing. * Use embed_query() for the search term. * This distinction is critical for providers like Vertex AI (text-embedding-gecko) to produce high-quality vectors.

2. Error Handling

Always wrap AI calls in try/except blocks to handle standard Athomic exceptions: * ContextWindowExceededError: Use a fallback strategy (e.g., summarize context). * StructuredOutputError: The model failed to produce JSON. You might want to retry or return a default value.

3. Tracing

The AI module is fully instrumented with OpenTelemetry. Use the trace_id from your application logs to visualize the full chain: Request -> Controller -> Memory Recall -> Vector Search -> LLM Generation.