Spotlight

A full-stack web application that resurfaces overlooked and under-covered global news. The core UX pairs trusted longform articles with curated social media eyewitness content, inspired by Ground News.

Architecture

• Frontend: Next.js 14 (App Router) + TailwindCSS + shadcn/ui
• Backend: FastAPI + PostgreSQL + Elasticsearch
• Payments: Stripe subscriptions
• Data Ingestion: Twitter API, Apify (TikTok), Instagram Graph API

Features

MVP Features

1. Discovery Feed: List of topics with filters (region, category, recency)
2. Topic Pages: Split-view layout with article (left) and curated social posts (right)
3. Social Content Ingestion: Automated scraping from Twitter, TikTok, Instagram
4. Curation Dashboard: Admin CRUD for topics, articles, and social posts
5. Reliability Scoring: Automated scoring system for social posts
6. Search: Full-text search across topics and posts (Elasticsearch)
7. Auth & Subscriptions: Email/password auth with Stripe $5/month subscription
8. Ads: Ethical ad slots for nonprofits

Getting Started

Prerequisites

• Node.js 18+
• Python 3.11+
• PostgreSQL 14+
• Elasticsearch 8+
• Redis (optional, for background jobs)

Frontend Setup

cd web
npm install
cp .env.example .env.local
# Edit .env.local with your API URL
npm run dev

Backend Setup

cd api
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -e .
cp env.example .env
# Edit .env with your database, Stripe, and API keys

# Run migrations
alembic upgrade head

# Start server
uvicorn app.main:app --reload

Environment Variables

Frontend (.env.local)

NEXT_PUBLIC_API_URL=http://localhost:8000

Backend (.env)

# Database
DATABASE_URL=postgresql://user:pass@localhost/spotlight
ASYNC_DATABASE_URL=postgresql+asyncpg://user:pass@localhost/spotlight

# JWT
SECRET_KEY=your-secret-key
ACCESS_TOKEN_EXPIRE_MINUTES=30

# Elasticsearch
ELASTICSEARCH_URL=http://localhost:9200

# Stripe
STRIPE_SECRET_KEY=sk_test_...
STRIPE_WEBHOOK_SECRET=whsec_...
STRIPE_PRICE_ID=price_...
FRONTEND_URL=http://localhost:3000

# Social Media APIs
TWITTER_BEARER_TOKEN=...
APIFY_API_TOKEN=...
INSTAGRAM_APP_ID=...
INSTAGRAM_APP_SECRET=...

API Endpoints

Public

• GET /api/topics - List all topics
• GET /api/topics/{slug} - Get topic by slug
• GET /api/search?q=... - Search topics and posts
• GET /api/score?post_id=... - Get reliability score

Auth

• POST /api/auth/register - Register new user
• POST /api/auth/login - Login (returns JWT)
• GET /api/auth/me - Get current user info

Subscriptions

• POST /api/subscribe/create-checkout - Create Stripe checkout session
• POST /api/subscribe/webhook - Stripe webhook handler

Admin

• POST /api/topics - Create topic
• PATCH /api/topics/{id} - Update topic
• DELETE /api/topics/{id} - Delete topic
• POST /api/curation - Create curation
• POST /api/scrape?topic_slug=... - Trigger scraping job

Data Models

Topic

• id, title, slug, summary
• region, tags[], key_facts[]
• primary_article_id, status (draft/published)
• last_updated

Article

• id, title, source, url
• summary, publication_date
• full_text (optional), paywalled

SocialPost

• id, platform, post_id, url
• author_handle, timestamp
• embed_html, media_urls[]
• reliability_score, tags[]

Reliability Scoring

Each social post receives a score (0-100) based on:

1. Provenance Match (0-30): Tags matching topic/article
2. Source Credibility (0-25): Author verification, reputation
3. Content Consistency (0-20): Text quality, length
4. Verification Assets (0-25): Media attachments, geolocation

Deployment

Frontend (Vercel)

cd web
vercel deploy

Backend (Fly.io / Render)

cd api
fly deploy  # or render deploy

Database

• Use managed PostgreSQL (Supabase, Neon, or AWS RDS)
• Use Elastic Cloud for Elasticsearch

Development

Running Tests

# Backend
cd api
pytest

# Frontend
cd web
npm test

Code Quality

# Backend
black app/
isort app/
mypy app/

# Frontend
cd web
npm run lint

License

MIT

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request

Support

For issues and questions, please open a GitHub issue.