diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..3c4bc670f8 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,298 @@ +# MIT Learn - Agent Instructions + +This is a Django/Next.js application for browsing and searching MIT learning resources aggregated from multiple platforms (OCW, MITx, xPRO, etc.). + +## Architecture Overview + +MIT Learn is a **hybrid Django + Next.js monorepo** with these major components: + +### Backend (Django + Celery) + +- **Django REST API** using DRF + drf-spectacular for OpenAPI generation +- **ETL Pipelines** in `learning_resources/etl/` extract, transform, and load data from external sources (OCW, edX, xPRO, MITx Online, Canvas, YouTube, podcasts, etc.) +- **OpenSearch** for full-text search with embeddings and hybrid search support +- **Celery** for async task processing (ETL, indexing, notifications) +- **Vector Search** using Qdrant for AI-powered recommendations and semantic search +- **Keycloak + APISIX** for authentication (see README-keycloak.md) + +Key Django apps: + +- `learning_resources` - Core models (LearningResource, Course, Program, ContentFile, etc.) and ETL pipelines +- `learning_resources_search` - OpenSearch indexing and search API +- `channels` - Community channels/groups +- `articles` - User-created articles with CKEditor +- `authentication` - Auth middleware and user management +- `vector_search` - Qdrant integration for embeddings +- `webhooks` - External webhook handlers (OCW, YouTube, etc.) + +### Frontend (Next.js App Router) + +Located in `frontends/` as a **Yarn workspaces monorepo**: + +- `main/` - Next.js 14+ app using App Router (NOT pages router) +- `api/` - Generated TypeScript API client from OpenAPI spec +- `ol-components/` - Shared React components +- `ol-search-ui/`, `ol-forms/`, `ol-utilities/` - Reusable packages +- Uses `@mitodl/smoot-design` for design system components +- Root-relative imports via `@/` in the `main` workspace + +## Build, Test, and Lint Commands + +### Backend (Python) + +Run inside Docker containers with `docker compose`: + +```bash +# Run all tests (parallel) +docker compose run --rm web poetry run pytest -n logical + +# Run specific test file +docker compose run --rm web poetry run pytest learning_resources/models_test.py + +# Run specific test +docker compose run --rm web poetry run pytest learning_resources/models_test.py::test_name -v + +# Lint and format with ruff +docker compose run --rm web poetry run ruff format . +docker compose run --rm web poetry run ruff check . --fix + +# Run Django management commands +docker compose run --rm web python manage.py + +# Create migrations +docker compose run --rm web python manage.py makemigrations + +# Run migrations +docker compose run --rm web python manage.py migrate + +# Create superuser +docker compose run --rm web python manage.py createsuperuser +``` + +### Frontend (TypeScript/React) + +From project root (not `frontends/` directory): + +```bash +# Run all tests +yarn test + +# Run tests for specific file +yarn test path/to/file.test.tsx + +# Watch mode +yarn test-watch + +# Lint +yarn workspace frontends run lint-check +yarn workspace frontends run lint-fix + +# Type checking +yarn workspace frontends run typecheck + +# Style linting (CSS/SCSS) +yarn workspace frontends run style-lint + +# Format with Prettier +yarn workspace frontends run fmt-check +yarn workspace frontends run fmt-fix + +# Build frontend +yarn build + +# Run dev server (inside Docker) +docker compose up + +# Run dev server (on host) +yarn watch +``` + +### E2E Tests (Playwright) + +```bash +# Run Playwright tests +yarn playwright + +# UI mode +yarn playwright:ui + +# View report +yarn playwright:report +``` + +### Pre-commit Hooks + +```bash +# Install pre-commit +pip install pre-commit +pre-commit install + +# Run all checks +pre-commit run --all-files +``` + +## Code Generation + +### OpenAPI Client Generation + +The TypeScript API client in `frontends/api/src/generated/` is auto-generated from the Django API using drf-spectacular: + +```bash +# Regenerate OpenAPI spec and TypeScript client +./scripts/generate_openapi.sh +``` + +**CI will fail if generated code is out of sync** - always regenerate after changing Django views/serializers. + +## Key Conventions + +### Python/Django + +- Use **Ruff** for linting and formatting (configured in `pyproject.toml`) +- **Factory Boy** for test data - all factories in `/factories.py` +- **Named Enums** - use `named_enum.ExtendedEnum` for constants (see `learning_resources/constants.py`) +- **Serializers** - use DRF serializers with `COMMON_IGNORED_FIELDS` exclusion pattern +- **Permissions** - use Django Guardian for object-level permissions +- **Views** - use DRF ViewSets with drf-spectacular decorators (`@extend_schema`) +- **Tests** - pytest-django with `conftest.py`, fixture-based setup, auto-mock external requests +- **Migrations** - all migrations must be non-auto and tested (see `scripts/test/no_auto_migrations.sh`) +- Never import `django.contrib.auth.models.User` directly - use `get_user_model()` or `settings.AUTH_USER_MODEL` + +### TypeScript/React/Next.js + +- **Next.js App Router** (NOT pages router) - routes in `frontends/main/src/app/` +- **React Query** - API calls via generated hooks from `api/` package +- **Test factories** - mock API responses with `setMockResponse` from `api/test-utils` +- **Styling** - CSS Modules or components from `@mitodl/smoot-design` and `ol-components` +- **Root imports** - use `@/` prefix in `main` workspace (e.g., `@/components`, `@/test-utils`) +- **Yarn workspaces** - run commands with `yarn workspace run