Loreframe 🎬

An autonomous, multi-agent AI video generation platform that turns a single prompt about a historic event into a full-length, synchronized documentary video.

5
AI Agents (LangGraph)
100%
Local AI Native
16:9 HD
ComfyUI Rendering
FFmpeg
Video Assembly

The Vision & Architecture

Creating historical documentary videos traditionally requires hours of chronological research, script writing, professional voiceover recording, visual asset curation, and precise video-audio timing synchronization in non-linear editing software.

Loreframe automates this entire production pipeline from end to end. Given a simple natural language prompt (e.g. "The Fall of Constantinople in 1453" or "The Apollo 11 Moon Landing"), Loreframe orchestrates a graph of specialized AI agents that collaborate to research, write, synthesize voice narration, generate styled visual art scenes, and assemble a broadcast-ready MP4 documentary video.

Core System Architecture
  • One-Prompt Generation: Converts simple text input directly into an MP4 documentary.
  • Local & Offline Privacy: Driven by Ollama, Kokoro TTS, and ComfyUI.
  • Resilient Fallback Engines: Zero-failure fallback to Sharp vector posters & PCM audio buffers.
  • Real-Time Telemetry: WebSockets status stream into Next.js 19 Studio UI.

Autonomous Multi-Agent Pipeline

LangGraph state machine orchestrating 5 continuous execution nodes.

Agent 01
Director / Research

Deconstructs historical topics, builds chronological context, and structures multi-chapter timeline schemas.

Agent 02
ScriptWriter

Generates narrator voiceover scripts and detailed scene prompts using local LLMs (Ollama / Gemini).

Agent 03
TTS Voiceover

Synthesizes audio narration WAV/MP3 files via Kokoro TTS with exact time duration tracking.

Agent 04
ImageGenerator

Renders 16:9 historical scene visuals using ComfyUI (SD 1.5/SDXL + LoRAs) or dynamic Sharp vector posters.

Agent 05
Video Assembler

FFmpeg static engine handles audio-visual synchronization, motion panning/zooming, and MP4 video rendering.

Engineering & Resilience Highlights

Local-First AI with Cloud Fallbacks Runs completely offline via Ollama (Llama 3.1 8B), Kokoro TTS (FastAPI Docker), and ComfyUI. Includes automatic cloud fallbacks to Google Gemini API.
Zero-Failure Render Pipeline If ComfyUI hits GPU memory limits, dynamic 16:9 vector posters are created via Sharp. Silent PCM tone buffers handle TTS restarts so timing sync never breaks.
Asynchronous Task Queue (BullMQ + Redis) Video rendering requests are queued using BullMQ over Redis. Segment state updates, job logs, and database sync (Prisma + SQLite) run asynchronously.
Real-Time Studio Control UI Next.js 19 studio interface featuring dynamic split-screen layout resizer, WebSockets status telemetry, live progress steps, and video playback.

Technology Stack

LangGraph Next.js 19 Node.js TypeScript BullMQ / Redis Prisma / SQLite Kokoro TTS ComfyUI FFmpeg Sharp WebSockets

Architected with clean decoupling across agent nodes, resilient queue workers, and real-time state synchronization.

Quick Setup Commands

# 1. Install Ollama & Pull Model
# Download Ollama from https://ollama.com/
ollama pull llama3.1:8b

# 2. Clone & Start Loreframe
git clone https://github.com/bibinantony1998/Loreframe.git
cd Loreframe
cp .env.example .env
npm start

# Note: npm start installs all dependencies, sets up the database, 
# and launches the UI, Server, Kokoro TTS, and ComfyUI all at once.
# You can update the LLM model name in the generated .env file.

Explore the Loreframe Repository

Loreframe is completely open source. Check out the unified repository containing the backend agent orchestration server and the Next.js studio control interface on GitHub.

Loreframe Repository