Add a tool to write GitHub metrics to CSV

github-reporting/README.md

@@ -0,0 +1,180 @@
+# GitHub Reporting
+
+This tool reads GitHub metrics stored in MongoDB Atlas (written by the [github-metrics](../github-metrics/README.md) tool) and exports them to
+CSV files for reporting to external stakeholders.
+
+## Overview
+
+The tool exports metrics to three separate CSV files for easy import into Google Sheets:
+
+- **summary.csv** - Core metrics per date/repo (clones, views, stars, forks, watchers)
+- **referrals.csv** - One row per referrer per date/repo
+- **top-paths.csv** - One row per path per date/repo
+
+The Date, Owner, and Repository columns in each file allow you to join/link data across sheets for analysis.
+
+## Prerequisites
+
+**Atlas**:
+
+- An Atlas Database User with read permissions for the **Developer Docs** -> **Project Metrics** project.
+- A valid connection string for the cluster above.
+
+Contact a member of the Developer Docs team to be added to this project and get the connection string.
+
+**System**:
+
+- Node.js/npm installed
+
+## Setup
+
+1. **Create a `.env` file**
+
+   Create a `.env` file that contains the following:
+
+   ```
+   ATLAS_CONNECTION_STRING="yourConnectionString"
+   ```
+
+   Replace the placeholder value with your connection string.
+
+   > Note: The `.env` file is in the `.gitignore`, so no worries about accidentally committing credentials.
+
+2. **Install the dependencies**
+
+   From the root of the directory, run:
+
+   ```
+   npm install
+   ```
+
+## Usage
+
+The tool supports two invocation methods: direct command-line arguments or a configuration file.
+
+### Method 1: Direct Command-Line Arguments
+
+Use the `export` command with options:
+
+```bash
+node --env-file=.env index.js export [options]
+```
+
+**Options:**
+
+| Option | Description                                                                                      |
+|--------|--------------------------------------------------------------------------------------------------|
+| `-s, --start-date <date>` | Start date for the report (ISO format, e.g., 2024-01-01)                                         |
+| `-e, --end-date <date>` | End date for the report (ISO format, e.g., 2024-12-31)                                           |
+| `-p, --projects <projects...>` | Space-separated list of owner/repo projects (e.g., mongodb/docs mongodb/sample-app-nodejs-mflix) |
+| `-o, --output <path>` | Output directory for CSV files                                                                   |
+
+**Examples:**
+
+```bash
+# Export all metrics from all projects
+node --env-file=.env index.js export -o my-report
+
+# Export metrics for a specific date range
+node --env-file=.env index.js export -s 2024-01-01 -e 2024-12-31 -o q4-report
+
+# Export metrics for specific projects
+node --env-file=.env index.js export -p mongodb/docs mongodb/docs-notebooks -o docs-report
+
+# Combine all options
+node --env-file=.env index.js export -s 2024-01-01 -e 2024-03-31 -p mongodb/docs -o q1-docs-report
+```
+
+### Method 2: Configuration File
+
+Use the `export-config` command with a JSON configuration file:
+
+```bash
+node --env-file=.env index.js export-config <config-file> [options]
+```
+
+**Options:**
+
+| Option | Description |
+|--------|-------------|
+| `-o, --output <path>` | Output directory for CSV files (overrides config file) |
+
+**Example configuration file (`config.json`):**
+
+```json
+{
+  "startDate": "2025-01-01",
+  "endDate": "2025-12-31",
+  "projects": [
+    { "owner": "mongodb", "repo": "docs" },
+    { "owner": "mongodb", "repo": "docs-notebooks" }
+  ],
+  "output": "annual-report"
+}
+```
+
+**Run with config file:**
+
+```bash
+node --env-file=.env index.js export-config config.json
+```
+
+**Override output directory:**
+
+```bash
+node --env-file=.env index.js export-config config.json -o different-output
+```
+
+## Output
+
+The tool creates a directory containing three CSV files:
+
+```
+my-report/
+├── summary.csv
+├── referrals.csv
+└── top-paths.csv
+```
+
+### summary.csv
+
+| Column | Description                                     |
+|--------|-------------------------------------------------|
+| Date | ISO timestamp of when metrics were collected    |
+| Owner | GitHub organization/owner                       |
+| Repository | Repository name                                 |
+| Clones | Number of clones in the last 14 days            |
+| Page Views | Total page views in the last 14 days            |
+| Unique Views | Unique visitors in the last 14 days             |
+| Stars | Star count (cumulative total, current count)    |
+| Forks | Fork count (cumulative total, current count)    |
+| Watchers | Watcher count (cumulative total, current count) |
+
+### referrals.csv
+
+| Column | Description |
+|--------|-------------|
+| Date | ISO timestamp of when metrics were collected |
+| Owner | GitHub organization/owner |
+| Repository | Repository name |
+| Referrer | Traffic source (e.g., google.com, github.com) |
+| Count | Total visits from this referrer |
+| Uniques | Unique visitors from this referrer |
+
+### top-paths.csv
+
+| Column | Description |
+|--------|-------------|
+| Date | ISO timestamp of when metrics were collected |
+| Owner | GitHub organization/owner |
+| Repository | Repository name |
+| Path | Path within the repository |
+| Count | Total visits to this path |
+| Uniques | Unique visitors to this path |
+
+## Importing to Google Sheets
+
+1. Create a new Google Sheet
+2. Go to **File** → **Import**
+3. Upload each CSV file as a separate sheet
+4. Use the Date, Owner, and Repository columns to create relationships between sheets using VLOOKUP or pivot tables

github-reporting/config.json.example

@@ -0,0 +1,10 @@
+{
+  "startDate": "2026-01-01",
+  "endDate": "2026-01-13",
+  "projects": [
+    { "owner": "mongodb", "repo": "sample-app-java-mflix" },
+    { "owner": "mongodb", "repo": "sample-app-nodejs-mflix" },
+    { "owner": "mongodb", "repo": "sample-app-python-mflix" }
+  ],
+  "output": "output"
+}

github-reporting/index.js

@@ -0,0 +1,127 @@
+#!/usr/bin/env node
+
+import { Command } from 'commander';
+import { readFileSync } from 'fs';
+import { readMetricsFromAtlas } from './read-from-db.js';
+import { writeMetricsToCsv, generateOutputDir } from './write-csv.js';
+
+const program = new Command();
+
+program
+    .name('github-reporting')
+    .description('Read GitHub metrics from MongoDB Atlas and export to CSV')
+    .version('1.0.0');
+
+// Direct invocation with command-line arguments
+program
+    .command('export')
+    .description('Export metrics to CSV using command-line arguments')
+    .option('-s, --start-date <date>', 'Start date for the report (ISO format, e.g., 2024-01-01)')
+    .option('-e, --end-date <date>', 'End date for the report (ISO format, e.g., 2024-12-31)')
+    .option('-p, --projects <projects...>', 'List of owner/repo projects (e.g., mongodb/docs realm/realm-js)')
+    .option('-o, --output <path>', 'Output directory for CSV files')
+    .action(async (options) => {
+        try {
+            const dateRange = buildDateRange(options.startDate, options.endDate);
+            const projects = parseProjects(options.projects);
+            const outputDir = options.output || generateOutputDir(dateRange);
+
+            console.log('Fetching metrics from MongoDB Atlas...');
+            if (dateRange.startDate || dateRange.endDate) {
+                console.log(`Date range: ${dateRange.startDate || 'beginning'} to ${dateRange.endDate || 'now'}`);
+            }
+            if (projects.length > 0) {
+                console.log(`Projects: ${projects.map(p => `${p.owner}/${p.repo}`).join(', ')}`);
+            } else {
+                console.log('Projects: all');
+            }
+
+            const metrics = await readMetricsFromAtlas(dateRange, projects);
+            await writeMetricsToCsv(metrics, outputDir);
+        } catch (error) {
+            console.error('Error:', error.message);
+            process.exit(1);
+        }
+    });
+
+// Config file invocation
+program
+    .command('export-config')
+    .description('Export metrics to CSV using a configuration file')
+    .argument('<config-file>', 'Path to JSON configuration file')
+    .option('-o, --output <path>', 'Output directory for CSV files (overrides config file)')
+    .action(async (configFile, options) => {
+        try {
+            const config = loadConfig(configFile);
+            const dateRange = buildDateRange(config.startDate, config.endDate);
+            const projects = config.projects || [];
+            const outputDir = options.output || config.output || generateOutputDir(dateRange);
+
+            console.log('Fetching metrics from MongoDB Atlas...');
+            if (dateRange.startDate || dateRange.endDate) {
+                console.log(`Date range: ${dateRange.startDate || 'beginning'} to ${dateRange.endDate || 'now'}`);
+            }
+            if (projects.length > 0) {
+                console.log(`Projects: ${projects.map(p => `${p.owner}/${p.repo}`).join(', ')}`);
+            } else {
+                console.log('Projects: all');
+            }
+
+            const metrics = await readMetricsFromAtlas(dateRange, projects);
+            await writeMetricsToCsv(metrics, outputDir);
+        } catch (error) {
+            console.error('Error:', error.message);
+            process.exit(1);
+        }
+    });
+
+/**
+ * Build a date range object from start and end date strings.
+ */
+function buildDateRange(startDate, endDate) {
+    const dateRange = {};
+    if (startDate) {
+        dateRange.startDate = startDate;
+    }
+    if (endDate) {
+        dateRange.endDate = endDate;
+    }
+    return dateRange;
+}
+
+/**
+ * Parse project strings (owner/repo format) into project objects.
+ */
+function parseProjects(projectStrings) {
+    if (!projectStrings || projectStrings.length === 0) {
+        return [];
+    }
+
+    return projectStrings.map(projectStr => {
+        const parts = projectStr.split('/');
+        if (parts.length !== 2) {
+            throw new Error(`Invalid project format: "${projectStr}". Expected format: owner/repo`);
+        }
+        return { owner: parts[0], repo: parts[1] };
+    });
+}
+
+/**
+ * Load and parse a JSON configuration file.
+ */
+function loadConfig(configPath) {
+    try {
+        const content = readFileSync(configPath, 'utf-8');
+        return JSON.parse(content);
+    } catch (error) {
+        if (error.code === 'ENOENT') {
+            throw new Error(`Configuration file not found: ${configPath}`);
+        }
+        if (error instanceof SyntaxError) {
+            throw new Error(`Invalid JSON in configuration file: ${error.message}`);
+        }
+        throw error;
+    }
+}
+
+program.parse();