🌟 Starchive - Personal Data Liberation Platform

Export all your social media content with one click.

Starchive empowers you to reclaim your digital memories from major social platforms using official APIs, GDPR data rights, and advanced export techniques.

✨ Features

One-Click Export

Export from all connected platforms simultaneously
Real-time progress tracking
Background processing with job queue

Platform Support

YouTube - Videos, playlists, comments, metadata
Facebook - Posts, photos, videos, messages
Instagram - Posts, stories, reels, comments
TikTok - Videos, profile data, liked content
Twitter/X - Tweets, media, likes, bookmarks
Snapchat - Memories, stories

Export Options

Direct Download - Organized ZIP files
Cloud Storage - Google Drive, Dropbox, OneDrive
Custom Endpoint - Send to your API
Multiple Formats - JSON, CSV, HTML

Security & Privacy

OAuth2 authentication
Encrypted token storage
GDPR/CCPA compliant
Self-hosting option

User Experience

In-App Help System - Comprehensive help documentation with interactive FAQ
Real-time Progress - Live export status tracking
Theme Support - Dark/light mode with full customization
Responsive Design - Works on desktop, tablet, and mobile

🚀 Quick Start

Deploy to Vercel (Recommended)

Fork this repository to your GitHub account
Connect to Vercel:
- Import your forked repository to Vercel
- Configure environment variables (see below)
- Deploy automatically

Configure Environment Variables in Vercel dashboard:

# Required for production
NODE_ENV=production
FACEBOOK_APP_ID=your_facebook_app_id
FACEBOOK_APP_SECRET=your_facebook_app_secret
GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret
SESSION_SECRET=your_session_secret_key
FRONTEND_URL=https://your-app-name.vercel.app
VITE_API_BASE_URL=https://your-app-name.vercel.app/api

Local Development

# Clone the repository
git clone https://github.com/digitalrelab/star-export.git
cd star-export

# Install all dependencies
npm run install:all

# Start development environment
npm run dev

# Access the app
# Frontend: http://localhost:5173
# API: http://localhost:3001

Manual Setup

Prerequisites

Node.js 18+
PostgreSQL
Redis
yt-dlp (for video downloads)

Frontend Setup

cd star-export-app
npm install
npm run dev

Backend Setup

cd server
npm install
cp .env.example .env
# Configure environment variables
npm run prisma:generate
npm run prisma:migrate
npm run dev

🔧 Configuration

Required API Keys

📖 Detailed Setup Guide: See OAUTH_SETUP_GUIDE.md for complete step-by-step OAuth configuration instructions.

Google OAuth (for YouTube)
- Create project at Google Cloud Console
- Enable YouTube Data API v3
- Create OAuth 2.0 credentials
Facebook App (for Facebook/Instagram)
- Create app at Facebook Developers
- Add Facebook Login product
- Configure OAuth redirect URIs

Environment Variables

Backend (.env)

# Database
DATABASE_URL=postgresql://user:password@localhost:5432/starexport

# Redis
REDIS_URL=redis://localhost:6379

# Authentication
JWT_SECRET=your-secret-key
ENCRYPTION_KEY=32-character-key

# OAuth
GOOGLE_CLIENT_ID=your-google-client-id
GOOGLE_CLIENT_SECRET=your-google-client-secret
FACEBOOK_APP_ID=your-facebook-app-id
FACEBOOK_APP_SECRET=your-facebook-app-secret

# Server
PORT=5000
NODE_ENV=development

Frontend (.env.local for development)

VITE_API_BASE_URL=http://localhost:3001

🚀 Vercel Deployment Guide

Automatic Deployment

Fork this repository
Import to Vercel
Configure environment variables
Deploy automatically on every push

Environment Variables for Vercel

Set these in your Vercel dashboard under Settings > Environment Variables:

NODE_ENV=production
FACEBOOK_APP_ID=your_facebook_app_id
FACEBOOK_APP_SECRET=your_facebook_app_secret  
GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret
SESSION_SECRET=your_session_secret_key
FRONTEND_URL=https://your-app-name.vercel.app
VITE_API_BASE_URL=https://your-app-name.vercel.app/api
FACEBOOK_CALLBACK_URL=https://your-app-name.vercel.app/api/auth/facebook/callback
GOOGLE_CALLBACK_URL=https://your-app-name.vercel.app/api/auth/google/callback

OAuth Configuration

After deployment, update your OAuth apps:

Facebook App Settings:

Valid OAuth Redirect URIs: https://your-app-name.vercel.app/api/auth/facebook/callback

Google Cloud Console:

Authorized redirect URIs: https://your-app-name.vercel.app/api/auth/google/callback

📚 Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│   React App     │────▶│   Express API   │────▶│   PostgreSQL    │
│   (Vite)        │     │   (TypeScript)  │     │   Database      │
└─────────────────┘     └─────────────────┘     └─────────────────┘
                               │                         
                               ▼                         
                        ┌─────────────────┐     ┌─────────────────┐
                        │   Bull Queue    │────▶│     Redis       │
                        │   (Jobs)        │     │     Cache       │
                        └─────────────────┘     └─────────────────┘
                               │
                               ▼
                        ┌─────────────────┐
                        │    Export       │
                        │   Services      │
                        │  - yt-dlp       │
                        │  - APIs         │
                        │  - Playwright   │
                        └─────────────────┘

🛠️ Development

Project Structure

star-export/
├── star-export-app/       # React frontend
│   ├── src/
│   │   ├── components/    # UI components
│   │   ├── hooks/         # React hooks
│   │   ├── pages/         # Page components
│   │   ├── services/      # API services
│   │   └── store/         # Zustand state
│   └── ...
├── server/                # Node.js backend
│   ├── src/
│   │   ├── routes/        # API routes
│   │   ├── services/      # Business logic
│   │   ├── middleware/    # Express middleware
│   │   └── utils/         # Utilities
│   └── ...
├── docker-compose.yml     # Docker setup
└── Makefile              # Common commands

Key Technologies

Frontend: React, TypeScript, Vite, TailwindCSS, Zustand
Backend: Node.js, Express, TypeScript, Prisma, Bull
Database: PostgreSQL, Redis
Export: yt-dlp, Playwright, Platform APIs
Infrastructure: Docker, Nginx

Available Commands

# Development
make dev              # Start development environment
make logs             # View application logs
make shell-backend    # Access backend shell
make test             # Run test suite

# Database
make db-migrate-dev   # Run migrations
make db-studio        # Open Prisma Studio
make backup-db        # Backup database

# Production
make prod             # Start production environment
make build            # Build for production
make deploy           # Deploy to production

🔒 Security

All tokens encrypted with AES-256-GCM
JWT authentication with refresh tokens
Rate limiting on all endpoints
Input validation and sanitization
CORS and security headers configured
Regular dependency updates

📖 API Documentation

Authentication

POST /api/auth/register
POST /api/auth/login
GET  /api/auth/google
POST /api/auth/refresh

Export Management

POST /api/export/start
GET  /api/export/:id
GET  /api/export
POST /api/export/:id/cancel
GET  /api/export/:id/download

Platform Integration

GET  /api/platforms
POST /api/platforms/:platform/connect
POST /api/platforms/:platform/disconnect
GET  /api/platforms/:platform/status

🤝 Contributing

Fork the repository
Create your feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

yt-dlp for video downloads
Playwright for browser automation
Platform APIs for official data access

⚠️ Disclaimer

This tool is designed for exporting your own personal data. Always respect platform terms of service and only export content you have rights to access. The developers are not responsible for misuse of this tool.

📞 Support

Issues: GitHub Issues
Discussions: GitHub Discussions
Security: security@starexport.com

Made with ❤️ for digital sovereignty

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
server		server
star-export-app		star-export-app
.env.example		.env.example
.gitignore		.gitignore
API_DOCUMENTATION.md		API_DOCUMENTATION.md
ARCHITECTURE.md		ARCHITECTURE.md
CHANGELOG.md		CHANGELOG.md
DEPLOYMENT.md		DEPLOYMENT.md
DEVELOPMENT_GUIDE.md		DEVELOPMENT_GUIDE.md
DEVELOPMENT_JOURNEY.md		DEVELOPMENT_JOURNEY.md
FACEBOOK_INSTAGRAM_INTEGRATION.md		FACEBOOK_INSTAGRAM_INTEGRATION.md
HELP.md		HELP.md
HELP_PAGE_README.md		HELP_PAGE_README.md
INTEGRATION_PLAN.md		INTEGRATION_PLAN.md
MEDIA_EXPORT_GUIDE.md		MEDIA_EXPORT_GUIDE.md
OAUTH_SETUP_GUIDE.md		OAUTH_SETUP_GUIDE.md
PRODUCT_SPEC.md		PRODUCT_SPEC.md
README.md		README.md
TECHNICAL_IMPLEMENTATION.md		TECHNICAL_IMPLEMENTATION.md
TEST_MEDIA_EXPORT.md		TEST_MEDIA_EXPORT.md
THE_DEVELOPMENT_JOURNEY.md		THE_DEVELOPMENT_JOURNEY.md
dev-start.sh		dev-start.sh
package.json		package.json
vercel.json		vercel.json

digitalrelab/star-export

Folders and files

Latest commit

History

Repository files navigation