Home
Portfolio
Productionizing a Vibe-Coded AI Course Creation Platform Built on Replit
AI Course Creation Platform
AI Course Creation Platform

Productionizing a Vibe-Coded AI Course Creation Platform Built on Replit

4 Weeks
4 Weeks
to first production release
16 Releases
16 Releases
during active Development
12 Third-Party Integrations
12 Third-Party Integrations
across core services
Productionizing a Vibe-Coded AI Course Creation Platform Built on Replit
Design adapted for confidentiality. Visuals are representative and differ from the final product under NDA.
Industry
EdTech / Enterprise Learning & Development
Project Duration
4.5 months of active development
The first production release went live about 3.5 weeks after active development began.
Tech Stack
React / TypeScript / Vite / Python / FastAPI / Celery / PostgreSQL / pgvector / Valkey-Redis / AWS ECS Fargate / Amazon S3 / CloudFront / GitHub Actions

[ client & product overview/ ]

An AI Platform for Turning Technical Documentation into Training

The client is building a SaaS platform for enterprise technical training. Users upload technical documentation such as manuals, standard operating procedures, and presentations, and the platform transforms it into structured courses with lessons, quizzes, slides, diagrams, narration, and video.

A core requirement is traceability. Key points, figures, and quiz questions maintain direct links to their source documentation, allowing subject-matter experts to verify accuracy.

Before partnering with MEV, the client had built a working prototype using Replit and AI-assisted development, validating the concept on real-world training materials.

Executive Summary

The client approached MEV with a functional Replit prototype, but it lacked a production-grade foundation. It had no automated tests, CI/CD pipelines, or role-based authentication; database migrations were manual, user management was incomplete, and long-running generation tasks ran synchronously on the web server.
MEV transitioned the product to AWS while incrementally hardening the inherited codebase. The team introduced automated testing, versioned database migrations, a robust authentication model, CI/CD pipelines, and production observability, while offloading long-running AI generation into recoverable background queues.

Beyond the infrastructure migration, MEV worked alongside the client to expand the product's capabilities. Key additions included source-grounded retrieval, selective regeneration for URL-sourced documentation, structured instructional design workflows, collaborative course editing, learner features, SCORM 1.2 export, subscriptions, and per-course AI cost tracking.

The platform went live less than a month after MEV joined and remained in production throughout the rest of the 4.5-month active development period.

[ productionization scope/ ]

Jobs MEV Solved in This Project
01:

Productionize a Replit AI application without a full rewrite

MEV stabilized the existing codebase and overhauled the prototype-era infrastructure without slowing down the product roadmap.
02:

Make long-running AI generation recoverable

The team separated generation into independent Celery worker processes and five Valkey-backed queues with provider-specific rate limits, checkpoints, and stalled-job recovery.
03:

Make AI-generated training traceable to its source

MEV hardened the existing source-traceability pipeline, including parsing, grounding images, reference validation, retrieval fixes, and reference evaluation, while the client team extended retrieval with hybrid BM25 + semantic search.
04:

Keep courses up to date when source documentation changes

For courses built from online documentation, the platform updates only lessons affected by source changes. MEV added controls to prevent overlapping jobs and recover stuck ones.
05:

Add subscription billing with API-enforced plan limits

MEV built three plan tiers, two paid through Stripe, with API-enforced limits for draft courses, published courses, and invited learners.
06:

Scaled the prototype into an enterprise training SaaS product

MEV added roles and access, invitations, admin tools, comments and permissions, learner progress persistence, SCORM 1.2 export, subscription controls, and a stepped course-creation wizard.
07:

Keep rapid Replit experimentation connected to controlled production development

MEV established a two-track Git workflow so the client could continue experimenting while every production change passed engineering review and quality gates.

Starting Point: Replit Prototype vs. AWS Production

The move to AWS did not require stopping the product for a ground-up rewrite. The production foundation was introduced first, while functionality continued to evolve.
Area Prototype, February 2026 Production, July 2026
Automated testing None 1,920 unit and API tests across 128 files
Database schema SQL scripts and a bash file, applied manually 35 versioned migrations
Course generation Background task inside the web server Dedicated worker processes, five rate-limited queues, automatic recovery
Authentication Firebase, no application roles Clerk with Creator, Learner, and Administrator roles, invitations, and access moderation
Deployment Rapid-development environment with no CI AWS ECS with automatic Dev deployment and approved Production deployment
Observability AI-call tracing only Error monitoring, infrastructure monitoring, LLM tracing, and product analytics
Codebase 209 files / 32K lines 755 files / 146K lines (roughly one-third client-authored)

[ how we did it/ ]

Solution & Implementation
01:
Moving the Replit prototype onto a production foundation

MEV first stabilized the high-risk operational areas: deployment, authentication, database management, testing, and runtime infrastructure.

The team replaced the prototype authentication with Clerk and added Creator, Learner, and Administrator roles, plus registration moderation, invitations, and per-course access controls. Database changes moved to SQLAlchemy and Alembic, resulting in 35 versioned migrations.

MEV also introduced automated testing and CI. At hand-off, the server test suite contained 1,920 unit and API tests across 128 files, with about 60% written by MEV engineers. External services are mocked in tests, and the suite runs on every pull request.

The production environment moved to AWS using ECS Fargate, S3, CloudFront, ElastiCache, and CloudWatch. GitHub Actions handles automated builds and deployments to AWS. Development deploys automatically, while production releases are deployed manually after founder approval.

02:
Rebuilding long-running AI generation around queues and recovery

Course generation can take up to an hour, with document analysis adding another 10 to 20 minutes. In the prototype, this work ran inside the web server, so a restart could terminate an active generation job.

MEV moved generation into three separate processes and split the main workflow across five queues, each configured around the limits of the external services it uses.

The system saves progress throughout the workflow, allowing interrupted jobs to resume instead of starting over. Stalled jobs are requeued automatically, workers scale separately from the web app, and provider-specific limits prevent one slow service from blocking the rest.

03:
Making generated content verifiable and maintainable

Source traceability starts when a document enters the system. The core pipeline already existed in the Replit prototype, and MEV focused on making it reliable for production.

Landing.ai splits uploaded documents into structured fragments and preserves page and position data for headings, text, tables, formulas, and figures. Cohere Embed via AWS Bedrock creates embeddings for those fragments, which are stored in PostgreSQL 16 with pgvector

MEV improved retrieval reliability through retrieval fixes, reference evaluation, and pipeline validation.

As a result, key points, figures, and quiz questions can stay linked to the source fragments they were generated from, allowing users to open and verify the original material.

MEV hardened selective regeneration for web-sourced courses by adding concurrency controls and stale-job recovery.

04:
Building the course-creation and learning product around the AI pipeline

Much of the instructional-design logic already existed in the prototype. MEV rebuilt the Creator experience as a step-by-step flow for defining the audience, goals, course length, and learning objectives before generation. The workflow uses established instructional-design frameworks, including Dick and Carey, ABCD, and Bloom's taxonomy. Assessments are defined before the corresponding lessons, and the platform also flags gaps or contradictions in the source material that could affect the course.

  • Source Materials and Course Generation

Creators can work with PDF, DOC, DOCX, PPT, and PPTX files, with Office documents converted on the server using LibreOffice. Supported documentation websites can also be used as source material. Generated courses can include 10 slide types and 11 diagram types, plus quizzes, inline exercises, narration, narrated slideshow video, and optional AI-avatar introductions. Numeric examples also receive a programmatic arithmetic check rather than relying only on the language model.

  • Creator Tools

Creators can edit generated text, replace figures with assets from the original source document, and regenerate individual content blocks without rebuilding the whole course. The platform also supports in-course comments and discussions, with permissions controlling who can view or participate.

  • Learner Experience

Learners get lesson-level progress tracking, saved quiz results, and resume support. A built-in course assistant answers questions using course materials, cites relevant lessons, recommends what to review next, and accepts PDF and image attachments as additional context.

  • Publishing and Enterprise Delivery

Courses can be published to the web or exported as SCORM 1.2 packages for use in an existing LMS, including quiz-result reporting back to the LMS. The platform also supports downloadable ZIP archives and embedding source-document fragments on external sites.

  • Subscription and Usage Controls

The SaaS product includes three plan tiers, two paid through Stripe. Course limits and invited-learner limits are enforced at the API layer rather than only in the interface.

05:
Tracking AI Generation Costs and Course Quality

The platform tracks AI-generation costs across five paid providers and shows cost by course and provider. The client-side developer built this cost-accounting capability.

Quality control combines automated and human review. Before releases, MEV's QA engineer runs the full generation flow against real documents and reviews the resulting courses. The client team also built a Claude-based evaluator that scores finished courses against defined quality criteria.

06:
Keeping Replit as an Experimentation Track Without Letting It Bypass Production Controls

Replit remained the client's sandbox for AI-assisted experimentation after the production environment moved to AWS.

MEV kept that experimentation separate from the production path. The founder continued working in Replit, while the client's second developer used regular feature branches and pull requests. All changes went through review before reaching AWS Dev and production.

The repository also included AI-agent instruction files such as replit.md and AGENTS.md, which defined rules for migrations, authentication, API conventions, and other coding standards.
Across the engagement, the project recorded 334 merged pull requests and 16 production releases.

[ technology/ ]

Application, infrastructure, and AI technology stack

Frontend

React 18 / TypeScript / Vite / Tailwind CSS / shadcn/ui / TanStack Query / PDF.js / React Flow / Recharts / KaTeX

Backend

Python 3.12 / FastAPI / Celery / SQLAlchemy / Alembic / Playwright / FFmpeg / LibreOffice

Data & AI

PostgreSQL 16 / pgvector / Valkey-Redis / Anthropic Claude / Cohere Embed via AWS Bedrock / BM25 + semantic retrieval

Infrastructure

AWS ECS Fargate / Amazon S3 / Amazon CloudFront / AWS ElastiCache / Amazon ECR / AWS CloudWatch / Docker / GitHub Actions

Third-Party Integrations

AI and document-processing providers use provider-specific rate limits, retry policies, and cooldown handling.
Service Role in the Product
Anthropic Claude Course structure, lessons, quizzes, slides, diagrams, and course assistant
Landing.ai Structured document parsing into positioned fragments
AWS Bedrock Vector embeddings for semantic retrieval
Clerk Authentication, account management, invitations
Stripe Subscriptions, payments, plan management
Amazon Web Services Hosting, object storage, database infrastructure, CDN
ElevenLabs Speech synthesis for slide narration
Synthesia AI-avatar video for lesson introductions
Sentry Error and performance tracking
PostHog Product analytics and feature usage
LangFuse AI-call tracing: cost, latency, quality
OpenAI Persona chat

[ portfolio/ ]

Related Case Studies

[ contact us/ ]

No pitch-happy salesperson here.

1 file up to 10 MB.
Max file size 10MB.
Uploading...
fileuploaded.jpg
Upload failed. Max size for files is 10 MB.
Before we discuss any details about your project you can request to sign a Non Disclosure Agreement

Thank You!

The ball's in our court now.
We'll get back to you within 24 hours.

By submitting, you agree to our Privacy Policy

Preferences

Privacy is important to us, so you have the option of disabling certain types of storage that may not be necessary for the basic functioning of the website. Blocking categories may impact your experience on the website. More information

Accept all cookies