gleamunison/llm_middleware

LLM handler middleware (gap analysis concept A, decomplected).

BAML complects retry/fallback/mocking into a client registry + hosted studio. Here retry is plain handler composition: a middleware is just an OpHandler wrapping another OpHandler. Same contract as llm.gleam: continuation receives {ok, CompletionJsonBin} | {error, ReasonBin}.

The budget/fallback/recording compositions that duplicated llm.complete_env’s inline settlement path were deleted (bk-baac): complete_env is the one live admit → release → meter → report authority (session_refresh and gleamunison_repl_ffi call it), and the shipped CLI path is intentionally budget-free — retry only.

pub type RetryPolicy {
  RetryPolicy(max_attempts: Int, base_delay_ms: Int)
}

pub fn retry_handler(
  inner: fn(
    List(dynamic.Dynamic),
    fn(dynamic.Dynamic) -> dynamic.Dynamic,
  ) -> dynamic.Dynamic,
  policy: RetryPolicy,
) -> fn(
  List(dynamic.Dynamic),
  fn(dynamic.Dynamic) -> dynamic.Dynamic,
) -> dynamic.Dynamic