feat: integrate Ollama inference with API and readiness checks

- Add an Ollama client for Qwen3 8B generation with request timeouts and HTTP error handling
- Implement POST /generate with request validation and sanitized failure responses
- Update GET /ready to report Ollama availability and return HTTP 503 when unavailable
- Add unit and integration tests for generation, dependency health, and failure handling
- Document implemented API endpoints, setup instructions, scripts, and remaining planned capabilities in README

README.md

@@ -1,12 +1,12 @@
 # Local AI Platform
 
-A self-hosted AI application designed to demonstrate production-grade backend engineering, local LLM integration, retrieval-augmented generation (RAG), observability, security, and reliability.
+A self-hosted AI application built to demonstrate production-minded backend engineering, local LLM integration, retrieval-augmented generation (RAG), and reliable API design.
 
-The project emphasizes measurable behavior, reproducible deployments, automated testing, and documented architectural trade-offs.
+The project emphasizes measurable behavior, reproducible development, automated testing, and documented architectural trade-offs.
 
 ## Project Goals
 
-- Build and expose a well-defined API for local LLM inference.
+- Build a well-defined API for local LLM inference.
 - Integrate retrieval-augmented generation with source-grounded responses.
 - Measure latency, throughput, errors, and retrieval quality.
 - Implement structured logging, metrics, and distributed tracing.
@@ -15,30 +15,45 @@ The project emphasizes measurable behavior, reproducible deployments, automated
 - Automate testing and deployment.
 - Document architectural decisions and operational procedures.
 
+These goals will be implemented incrementally.
+
 ## Technology Stack
 
-Planned technologies:
+**Currently implemented**
 
 - **Runtime:** Node.js
 - **Language:** TypeScript
 - **API framework:** Fastify
-- **Validation:** Zod
+- **Request validation:** Zod
 - **Local inference:** Ollama
+- **Testing:** Node.js test runner with `tsx`
+
+**Planned**
+
 - **Database and retrieval:** PostgreSQL with pgvector
 - **Containers:** Docker and Docker Compose
-- **Testing:** Automated unit, integration, and failure-injection tests
 - **Observability:** OpenTelemetry and compatible monitoring tools
 - **CI/CD:** Automated build, test, and deployment workflows
+- **Evaluation:** Performance benchmarks and retrieval-quality tests
 
-Technologies will be introduced incrementally as project requirements justify them.
+Technologies will be introduced as project requirements justify them.
 
 ## Current Status
 
-**Status:** Initial development
+**Status:** Initial API foundation and local inference integration implemented.
 
-The initial milestone is to establish a working API with health and readiness endpoints, then integrate local model inference and baseline performance measurements.
+The API currently provides:
 
-Features listed in the project goals are planned capabilities, not claims that they have already been implemented.
+- `GET /health` — reports API health.
+- `GET /ready` — checks whether Ollama is available.
+- `POST /echo` — demonstrates request validation and API behavior.
+- `POST /generate` — sends a prompt to a local Ollama model and returns generated text.
+- Centralized error handling and consistent responses for unknown routes.
+- Automated tests for API behavior, validation, inference responses, and failure handling.
+
+The inference client currently uses the `qwen3:8b` model and connects to Ollama at `http://127.0.0.1:11434`.
+
+**Not yet implemented:** RAG, PostgreSQL persistence, authentication, production deployment, metrics, distributed tracing, and CI/CD.
 
 ## Getting Started
 
@@ -46,15 +61,14 @@ Features listed in the project goals are planned capabilities, not claims that t
 
 - Node.js and npm
 - Git
-
+- [Ollama](https://ollama.com/) for local inference
-Additional dependencies, including Ollama and PostgreSQL, will be introduced as the project evolves.
 
 ### Installation
 
 Clone the repository:
 
 ```bash
-git clone <repository-url>
+git clone https://git.brandonwolfgang.com/brandon/local-ai-platform.git
 cd local-ai-platform
 ```
 
@@ -64,28 +78,85 @@ Install dependencies:
 npm ci
 ```
 
+Ensure Ollama is running and the configured model is available:
+
+```bash
+ollama pull qwen3:8b
+```
+
 ### Development
 
-The development server and available scripts will be documented here as they are implemented.
+Start the development server:
+
+```bash
+npm run dev
+```
+
+The API listens on `http://127.0.0.1:3000`.
+
+Check API health:
+
+```bash
+curl http://127.0.0.1:3000/health
+```
+
+Check inference-service readiness:
+
+```bash
+curl http://127.0.0.1:3000/ready
+```
+
+Generate a response:
+
+```bash
+curl -X POST http://127.0.0.1:3000/generate \
+  -H "Content-Type: application/json" \
+  -d '{"message":"Explain what a REST API does in one sentence."}'
+```
+
+### Available Scripts
+
+| Command | Purpose |
+|---|---|
+| `npm run dev` | Start the development server with automatic restarts |
+| `npm run build` | Compile TypeScript into `dist/` |
+| `npm start` | Run the compiled application |
+| `npm test` | Run the automated test suite |
 
 ## Architecture
 
-The initial architecture will consist of an HTTP API and a local model inference service.
+The current architecture consists of a TypeScript/Fastify HTTP API and a local Ollama inference service.
 
-Subsequent iterations will introduce retrieval, persistence, observability, security controls, and deployment automation.
+The API validates incoming requests, delegates generation to Ollama, and returns generated text. Readiness checks verify the inference service is reachable, while error handling prevents internal exception details from being returned in API responses.
+
+Future iterations will introduce document ingestion, retrieval, persistence, security controls, observability, and deployment automation.
 
 Architecture diagrams and significant design decisions will be maintained alongside the code.
 
 ## Testing and Quality
 
-The project will progressively introduce:
+The automated test suite currently covers:
 
-- Automated unit and integration tests
+- API health and readiness behavior
-- API contract validation
+- Request validation and input limits
-- Performance benchmarks
+- Successful inference responses
-- Retrieval and answer-quality evaluations
+- Inference failure handling
-- Security regression tests
+- Unknown routes and generic server errors
-- Failure-recovery tests
+- Ollama client response handling
+
+Run the test suite:
+
+```bash
+npm test
+```
+
+Verify the production build:
+
+```bash
+npm run build
+```
+
+Performance benchmarks, retrieval evaluations, security regression tests, and recovery tests remain planned work.
 
 Performance and quality claims will be backed by reproducible measurements.
 

src/ollama.test.ts

@@ -0,0 +1,47 @@
+import assert from "node:assert/strict";
+import { it } from "node:test";
+
+import { generateResponse } from "./ollama.js";
+
+it("returns generated text from Ollama", async () => {
+  const originalFetch = globalThis.fetch;
+
+  globalThis.fetch = async () =>
+    new Response(
+      JSON.stringify({
+        response: "A REST API lets software systems communicate.",
+        thinking: "Internal reasoning that should not be returned",
+      }),
+      {
+        status: 200,
+        headers: {
+          "Content-Type": "application/json",
+        },
+      },
+    );
+
+  try {
+    const result = await generateResponse("Explain a REST API.");
+
+    assert.equal(result, "A REST API lets software systems communicate.");
+  } finally {
+    globalThis.fetch = originalFetch;
+  }
+});
+
+it("throws an error when Ollama returns a non-OK status", async () => {
+  const originalFetch = globalThis.fetch;
+
+  globalThis.fetch = async () =>
+    new Response("Service unavailable", {
+      status: 503,
+    });
+
+  try {
+    await assert.rejects(generateResponse("Explain a REST API."), {
+      message: "Ollama request failed with status 503",
+    });
+  } finally {
+    globalThis.fetch = originalFetch;
+  }
+});

src/ollama.ts

@@ -0,0 +1,40 @@
+const OLLAMA_BASE_URL = "http://127.0.0.1:11434";
+
+interface OllamaGenerateResponse {
+  response: string;
+}
+
+export const generateResponse = async (prompt: string): Promise<string> => {
+  const response = await fetch(`${OLLAMA_BASE_URL}/api/generate`, {
+    method: "POST",
+    headers: {
+      "Content-Type": "application/json",
+    },
+    body: JSON.stringify({
+      model: "qwen3:8b",
+      prompt,
+      stream: false,
+    }),
+    signal: AbortSignal.timeout(30_000),
+  });
+
+  if (!response.ok) {
+    throw new Error(`Ollama request failed with status ${response.status}`);
+  }
+
+  const result = (await response.json()) as OllamaGenerateResponse;
+
+  return result.response;
+};
+
+export const checkOllamaHealth = async (): Promise<boolean> => {
+  try {
+    const response = await fetch(`${OLLAMA_BASE_URL}/api/tags`, {
+      signal: AbortSignal.timeout(3_000),
+    });
+
+    return response.ok;
+  } catch {
+    return false;
+  }
+};

src/server.test.ts

@@ -24,6 +24,9 @@ describe("API endpoints", () => {
   });
 
   it("GET /ready returns 200 and dependency status", async () => {
+    const app = buildApp(undefined, async () => true);
+
+    try {
       const response = await app.inject({
         method: "GET",
         url: "/ready",
@@ -33,9 +36,33 @@ describe("API endpoints", () => {
       assert.deepEqual(response.json(), {
         status: "ready",
         dependencies: {
-        ollama: "not-integrated",
+          ollama: "available",
         },
       });
+    } finally {
+      await app.close();
+    }
+  });
+
+  it("GET /ready returns 503 when Ollama is unavailable", async () => {
+    const app = buildApp(undefined, async () => false);
+
+    try {
+      const response = await app.inject({
+        method: "GET",
+        url: "/ready",
+      });
+
+      assert.equal(response.statusCode, 503);
+      assert.deepEqual(response.json(), {
+        status: "not-ready",
+        dependencies: {
+          ollama: "unavailable",
+        },
+      });
+    } finally {
+      await app.close();
+    }
   });
 
   it("POST /echo returns the validated message", async () => {
@@ -66,6 +93,54 @@ describe("API endpoints", () => {
     assert.deepEqual(response.json(), { message });
   });
 
+  it("POST /generate returns generated text", async () => {
+    const app = buildApp(async (prompt) => {
+      assert.equal(prompt, "Explain a REST API.");
+      return "A REST API enables software systems to communicate.";
+    });
+
+    try {
+      const response = await app.inject({
+        method: "POST",
+        url: "/generate",
+        payload: {
+          message: "Explain a REST API.",
+        },
+      });
+
+      assert.equal(response.statusCode, 200);
+      assert.deepEqual(response.json(), {
+        response: "A REST API enables software systems to communicate.",
+      });
+    } finally {
+      await app.close();
+    }
+  });
+
+  it("POST /generate returns 503 when generation fails", async () => {
+    const app = buildApp(async () => {
+      throw new Error("Ollama is unavailable");
+    });
+
+    try {
+      const response = await app.inject({
+        method: "POST",
+        url: "/generate",
+        payload: {
+          message: "Explain a REST API.",
+        },
+      });
+
+      assert.equal(response.statusCode, 503);
+      assert.deepEqual(response.json(), {
+        error: "Generation service unavailable",
+      });
+      assert.equal(response.body.includes("Ollama is unavailable"), false);
+    } finally {
+      await app.close();
+    }
+  });
+
   it("POST /echo rejects an empty message", async () => {
     const response = await app.inject({
       method: "POST",

src/server.ts

@@ -1,11 +1,16 @@
 import Fastify, { type FastifyError } from "fastify";
 import { z } from "zod";
 
+import { checkOllamaHealth, generateResponse } from "./ollama.js";
+
 const echoSchema = z.object({
   message: z.string().min(1).max(500),
 });
 
-export const buildApp = () => {
+export const buildApp = (
+  generate = generateResponse,
+  checkHealth = checkOllamaHealth,
+) => {
   const app = Fastify({
     logger: true,
   });
@@ -34,12 +39,25 @@ export const buildApp = () => {
     status: "ok",
   }));
 
-  app.get("/ready", async () => ({
+  app.get("/ready", async (_request, reply) => {
+    const ollamaReady = await checkHealth();
+
+    if (!ollamaReady) {
+      return reply.code(503).send({
+        status: "not-ready",
+        dependencies: {
+          ollama: "unavailable",
+        },
+      });
+    }
+
+    return {
       status: "ready",
       dependencies: {
-      ollama: "not-integrated",
+        ollama: "available",
       },
-  }));
+    };
+  });
 
   app.post("/echo", async (request, reply) => {
     const result = echoSchema.safeParse(request.body);
@@ -55,6 +73,28 @@ export const buildApp = () => {
     };
   });
 
+  app.post("/generate", async (request, reply) => {
+    const result = echoSchema.safeParse(request.body);
+
+    if (!result.success) {
+      return reply.code(400).send({
+        error: "Invalid request body",
+      });
+    }
+
+    try {
+      const response = await generate(result.data.message);
+
+      return { response };
+    } catch (error) {
+      request.log.error({ err: error }, "Ollama generation failed");
+
+      return reply.code(503).send({
+        error: "Generation service unavailable",
+      });
+    }
+  });
+
   app.get("/test-error", async () => {
     throw new Error("Sensitive internal detail");
   });