server

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 2, 2026 License: MIT Imports: 21 Imported by: 0

README

Server Package

Purpose & Responsibilities

The server package provides HTTP server lifecycle management for the LLM Proxy. It handles:

  • HTTP server creation, configuration, and startup
  • Graceful shutdown with resource cleanup
  • Route registration for health, management, and proxy endpoints
  • Management API endpoints for projects, tokens, and audit
  • Request logging and middleware composition
  • Integration with event bus, audit logging, and proxy components

Architecture

graph TB
    subgraph Server["Server Package"]
        S[Server]
        H[Health Handlers]
        M[Management API]
        MW[Middleware]
    end
    
    subgraph Dependencies
        P[Proxy]
        T[TokenStore]
        PS[ProjectStore]
        DB[Database]
        EB[EventBus]
        AL[AuditLogger]
    end
    
    Client --> S
    S --> H
    S --> M
    S --> MW
    MW --> P
    S --> T
    S --> PS
    S --> DB
    S --> EB
    S --> AL

Key Types & Interfaces

Type Description
Server Main HTTP server struct managing lifecycle, routes, and dependencies
HealthResponse JSON response for health check endpoint
Metrics Runtime metrics including request/error counts and start time
Constructor Functions
Function Description
New(cfg, tokenStore, projectStore) Creates server without database
NewWithDatabase(cfg, tokenStore, projectStore, db) Creates server with database for audit logging

Server Lifecycle

stateDiagram-v2
    [*] --> Created: New() / NewWithDatabase()
    Created --> Starting: Start()
    Starting --> Running: ListenAndServe
    Running --> ShuttingDown: Shutdown()
    ShuttingDown --> [*]: Cleanup Complete
    
    note right of ShuttingDown
        1. Stop cache stats aggregator
        2. Close audit logger
        3. Stop accepting connections
        4. Wait for in-flight requests
    end note

Configuration

The server accepts configuration via config.Config:

Field Description Default
ListenAddr Address to listen on :8080
RequestTimeout Maximum request duration 30s
ManagementToken Token for management API auth Required
EnableMetrics Enable metrics endpoint false
MetricsPath Path for metrics endpoint /metrics
AuditEnabled Enable audit logging false
AuditLogFile Path to audit log file -
EventBusBackend Event bus type (redis, in-memory) in-memory
Environment Variables
Variable Description
MANAGEMENT_TOKEN Token for management API authentication
LISTEN_ADDR Server listen address
HTTP_CACHE_ENABLED Enable HTTP cache (true/false)
HTTP_CACHE_BACKEND Cache backend (redis or in-memory)
REDIS_ADDR Redis server address (shared with event bus)
REDIS_DB Redis database number

API Routes

Health Endpoints
Endpoint Method Description
/health GET Full health check with status and version
/ready GET Readiness probe for k8s
/live GET Liveness probe for k8s
Management API (requires MANAGEMENT_TOKEN)
Endpoint Method Description
/manage/projects GET, POST List or create projects
/manage/projects/{id} GET, PUT, DELETE Project CRUD by ID
/manage/tokens GET, POST List or create tokens
/manage/tokens/{id} GET, PUT, DELETE Token CRUD by ID
/manage/audit GET Query audit events
/manage/audit/{id} GET Get audit event by ID
/manage/cache/purge POST Purge HTTP cache
Proxy Endpoint

All unmatched routes are handled by the transparent proxy, forwarding requests to the configured upstream API (e.g., OpenAI).

Testing Guidance

  • Use httptest.NewServer for integration tests
  • Use in-memory database (database.Config{Path: ":memory:"}) for unit tests
  • Mock TokenStore and ProjectStore interfaces for isolated testing
  • See existing tests in server_test.go for patterns

Troubleshooting

Server Won't Start
Symptom Cause Solution
"address already in use" Port conflict Change ListenAddr or stop conflicting process
"failed to initialize logger" Invalid log config Check LogLevel and LogFile path
"failed to initialize audit logger" Invalid audit path Ensure AuditLogFile directory exists
"unknown event bus backend" Invalid config Use redis or in-memory
Management API Returns 401
  • Ensure MANAGEMENT_TOKEN environment variable is set
  • Include Authorization: Bearer <token> header in requests
  • Token must match exactly (case-sensitive)
Health Check Fails
  • Check if the server started successfully (logs)
  • Verify the listen address is accessible
  • Check for database connectivity issues
Slow Shutdown
  • Increase shutdown timeout if requests need more time
  • Check for stuck connections or long-running requests
  • Review audit logger flush behavior
Package Relationship
proxy Transparent proxy handler for API requests
token Token validation for authentication
database Token and project storage
middleware Request instrumentation and observability
eventbus Async event publishing
audit Audit event logging
config Server configuration

Files

File Description
server.go Main HTTP server implementation and lifecycle management
routes.go Route registration and middleware composition
handlers.go HTTP handler implementations for management API

Documentation

Overview

Package server implements the HTTP server for the LLM Proxy. It handles request routing, lifecycle management, and provides health check endpoints and core API functionality.

Index

Constants

View Source 
const Version = "0.1.0"

Version is the application version, following semantic versioning.

Variables

This section is empty.

Functions

This section is empty.

Types

type CachePurgeRequest 

type CachePurgeRequest struct {
	Method string `json:"method" binding:"required"`
	URL    string `json:"url" binding:"required"`
	Prefix string `json:"prefix,omitempty"`
}

CachePurgeRequest represents the request body for cache purge operations

type CachePurgeResponse 

type CachePurgeResponse struct {
	Deleted interface{} `json:"deleted"` // bool for exact purge, int for prefix purge
}

CachePurgeResponse represents the response body for cache purge operations

type HealthResponse 

type HealthResponse struct {
	Status    string    `json:"status"`    // Service status, "ok" for a healthy system
	Timestamp time.Time `json:"timestamp"` // Current server time
	Version   string    `json:"version"`   // Application version number
}

HealthResponse is the response body for the health check endpoint. It provides basic information about the server status and version.

type Metrics 

type Metrics struct {
	StartTime    time.Time
	RequestCount int64
	ErrorCount   int64
}

Metrics holds runtime metrics for the server.

type ProjectResponse 

type ProjectResponse struct {
	ID            string     `json:"id"`
	Name          string     `json:"name"`
	APIKey        string     `json:"api_key"` // Obfuscated for security
	IsActive      bool       `json:"is_active"`
	DeactivatedAt *time.Time `json:"deactivated_at,omitempty"`
	CreatedAt     time.Time  `json:"created_at"`
	UpdatedAt     time.Time  `json:"updated_at"`
}

ProjectResponse is the sanitized project response with obfuscated API key

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server represents the HTTP server for the LLM Proxy. It encapsulates the underlying http.Server along with application configuration and handles request routing and server lifecycle management.

func New 

func New(cfg *config.Config, tokenStore token.TokenStore, projectStore proxy.ProjectStore) (*Server, error)

New creates a new HTTP server with the provided configuration and store implementations. It initializes the server with appropriate timeouts and registers all necessary route handlers. The server is not started until the Start method is called.

func NewWithDatabase 

func NewWithDatabase(cfg *config.Config, tokenStore token.TokenStore, projectStore proxy.ProjectStore, db *database.DB) (*Server, error)

NewWithDatabase creates a new HTTP server with database support for audit logging. This allows the server to store audit events in both file and database backends.

func (*Server) EventBus 

func (s *Server) EventBus() eventbus.EventBus

EventBus returns the event bus used by the server (may be nil if observability is disabled)

func (*Server) Shutdown 

func (s *Server) Shutdown(ctx context.Context) error

Shutdown gracefully shuts down the server without interrupting active connections. It waits for all connections to complete or for the provided context to be canceled, whichever comes first.

The context should typically include a timeout to prevent the shutdown from blocking indefinitely.

func (*Server) Start 

func (s *Server) Start() error

Start initializes all required components and starts the HTTP server. This method blocks until the server is shut down or an error occurs.

It returns an error if the server fails to start or encounters an unrecoverable error during operation.

type TokenListResponse 

type TokenListResponse struct {
	ID            string     `json:"id"`    // Token UUID (for API operations)
	Token         string     `json:"token"` // Obfuscated token string (for display only)
	ProjectID     string     `json:"project_id"`
	ExpiresAt     *time.Time `json:"expires_at"`
	IsActive      bool       `json:"is_active"`
	RequestCount  int        `json:"request_count"`
	MaxRequests   *int       `json:"max_requests"`
	CreatedAt     time.Time  `json:"created_at"`
	LastUsedAt    *time.Time `json:"last_used_at"`
	CacheHitCount int        `json:"cache_hit_count"`
}

TokenListResponse matches the sanitized token response schema (shared for tests and production)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.