115 lines
3.6 KiB
Markdown
115 lines
3.6 KiB
Markdown
# Local AI Platform
|
|
|
|
A self-hosted AI application designed to demonstrate production-grade backend engineering, local LLM integration, retrieval-augmented generation (RAG), observability, security, and reliability.
|
|
|
|
The project emphasizes measurable behavior, reproducible deployments, automated testing, and documented architectural trade-offs.
|
|
|
|
## Project Goals
|
|
|
|
- Build and expose a well-defined API for local LLM inference.
|
|
- Integrate retrieval-augmented generation with source-grounded responses.
|
|
- Measure latency, throughput, errors, and retrieval quality.
|
|
- Implement structured logging, metrics, and distributed tracing.
|
|
- Apply authentication, authorization, input validation, and secure configuration.
|
|
- Test failure handling, retries, idempotency, and recovery.
|
|
- Automate testing and deployment.
|
|
- Document architectural decisions and operational procedures.
|
|
|
|
## Technology Stack
|
|
|
|
Planned technologies:
|
|
|
|
- **Runtime:** Node.js
|
|
- **Language:** TypeScript
|
|
- **API framework:** Fastify
|
|
- **Validation:** Zod
|
|
- **Local inference:** Ollama
|
|
- **Database and retrieval:** PostgreSQL with pgvector
|
|
- **Containers:** Docker and Docker Compose
|
|
- **Testing:** Automated unit, integration, and failure-injection tests
|
|
- **Observability:** OpenTelemetry and compatible monitoring tools
|
|
- **CI/CD:** Automated build, test, and deployment workflows
|
|
|
|
Technologies will be introduced incrementally as project requirements justify them.
|
|
|
|
## Current Status
|
|
|
|
**Status:** Initial development
|
|
|
|
The initial milestone is to establish a working API with health and readiness endpoints, then integrate local model inference and baseline performance measurements.
|
|
|
|
Features listed in the project goals are planned capabilities, not claims that they have already been implemented.
|
|
|
|
## Getting Started
|
|
|
|
### Prerequisites
|
|
|
|
- Node.js and npm
|
|
- Git
|
|
|
|
Additional dependencies, including Ollama and PostgreSQL, will be introduced as the project evolves.
|
|
|
|
### Installation
|
|
|
|
Clone the repository:
|
|
|
|
```bash
|
|
git clone <repository-url>
|
|
cd local-ai-platform
|
|
```
|
|
|
|
Install dependencies:
|
|
|
|
```bash
|
|
npm ci
|
|
```
|
|
|
|
### Development
|
|
|
|
The development server and available scripts will be documented here as they are implemented.
|
|
|
|
## Architecture
|
|
|
|
The initial architecture will consist of an HTTP API and a local model inference service.
|
|
|
|
Subsequent iterations will introduce retrieval, persistence, observability, security controls, and deployment automation.
|
|
|
|
Architecture diagrams and significant design decisions will be maintained alongside the code.
|
|
|
|
## Testing and Quality
|
|
|
|
The project will progressively introduce:
|
|
|
|
- Automated unit and integration tests
|
|
- API contract validation
|
|
- Performance benchmarks
|
|
- Retrieval and answer-quality evaluations
|
|
- Security regression tests
|
|
- Failure-recovery tests
|
|
|
|
Performance and quality claims will be backed by reproducible measurements.
|
|
|
|
## Security
|
|
|
|
- Never commit credentials, access tokens, private keys, or production data.
|
|
- Keep local configuration and secrets outside version control.
|
|
- Validate untrusted input and enforce authorization at the API boundary.
|
|
- Treat retrieved documents and model-generated output as untrusted.
|
|
- Do not expose the API publicly without appropriate authentication and security controls.
|
|
|
|
Report suspected vulnerabilities privately to the repository owner.
|
|
|
|
## Documentation
|
|
|
|
As the project develops, documentation will cover:
|
|
|
|
- Local setup and deployment
|
|
- Architecture decision records
|
|
- Performance and evaluation results
|
|
- Threat models and security controls
|
|
- Operational runbooks and recovery procedures
|
|
- Known limitations and future improvements
|
|
|
|
## License
|
|
|
|
[MIT License](LICENSE)
|