ONS-Innovation · smstone0 · Dec 3, 2025 · Dec 10, 2025 · Dec 10, 2025 · Dec 10, 2025
diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # GitHub Copilot Usage Lambda
 
-This repository contains the AWS Lambda Function for updating the GitHub Copilot dashboard's historic information, stored within an S3 bucket.
+This repository contains the AWS Lambda Function for updating the GitHub Copilot dashboard's organisation-wide historic data, Copilot teams, and teams history.
 
 The Copilot dashboard can be found on the Copilot tab within the Digital Landscape.
 
@@ -16,7 +16,7 @@ The Copilot dashboard can be found on the Copilot tab within the Digital Landsca
   - [Makefile](#makefile)
   - [AWS Lambda Scripts](#aws-lambda-scripts)
     - [Setup - Running in a container](#setup---running-in-a-container)
-    - [Setup - running outside of a Container (Development only)](#setup---running-outside-of-a-container-development-only)
+    - [Setup - running outside of a Container (Development only)](#running-outside-of-a-container-development-only)
     - [Storing the container on AWS Elastic Container Registry (ECR)](#storing-the-container-on-aws-elastic-container-registry-ecr)
     - [Deployment to AWS](#deployment-to-aws)
       - [Deployment Prerequisites](#deployment-prerequisites)
@@ -138,7 +138,23 @@ Further information can be found in [this project's documentation](/docs/index.m
    docker stop 3f7d64676b1a
    ```
 
-### Setup - running outside of a Container (Development only)
+### Setup
+
+Export the required environment variables:
+
+  ```bash
+  export AWS_ACCESS_KEY_ID=<aws_access_key_id>
+  export AWS_SECRET_ACCESS_KEY=<aws_secret_access_key>
+  export AWS_DEFAULT_REGION=eu-west-2
+  export AWS_SECRET_NAME=<aws_secret_name>
+  export GITHUB_ORG=ONSDigital
+  export GITHUB_APP_CLIENT_ID=<github_app_client_id>
+  export AWS_ACCOUNT_NAME=<sdp-dev/sdp-prod>
+  ```
+
+The lambda can be run outside of a container for development purposes, or inside a container image to push to AWS ECR.
+
+#### Running outside of a Container (Development only)
 
 To run the Lambda function outside of a container, we need to execute the `handler()` function.
 
@@ -153,25 +169,13 @@ To run the Lambda function outside of a container, we need to execute the `handl
 
    **Please Note:** If uncommenting the above in `main.py`, make sure you re-comment the code _before_ pushing back to GitHub.
 
-2. Export the required environment variables:
-
-   ```bash
-   export AWS_ACCESS_KEY_ID=<aws_access_key_id>
-   export AWS_SECRET_ACCESS_KEY=<aws_secret_access_key>
-   export AWS_DEFAULT_REGION=eu-west-2
-   export AWS_SECRET_NAME=<aws_secret_name>
-   export GITHUB_ORG=ONSDigital
-   export GITHUB_APP_CLIENT_ID=<github_app_client_id>
-   export AWS_ACCOUNT_NAME=<sdp-dev/sdp-prod>
-   ```
-
-3. Run the script.
+2. Run the script.
 
    ```bash
    python3 src/main.py
    ```
 
-### Storing the container on AWS Elastic Container Registry (ECR)
+#### Storing the container on AWS Elastic Container Registry (ECR)
 
 When you make changes to the Lambda Script, a new container image must be pushed to ECR.
 
@@ -294,13 +298,7 @@ If the application has been modified, the following can be performed to update t
 
   The reconfigure options ensures that the backend state is reconfigured to point to the appropriate S3 bucket.
 
-  **_Please Note:_** This step requires an **AWS_ACCESS_KEY_ID** and **AWS_SECRET_ACCESS_KEY** to be loaded into the environment if not already in place.
-  This can be done using:
-
-  ```bash
-  export AWS_ACCESS_KEY_ID="<aws_access_key_id>"
-  export AWS_SECRET_ACCESS_KEY="<aws_secret_access_key>"
-  ```
+  **_Please Note:_** This step requires an **AWS_ACCESS_KEY_ID** and **AWS_SECRET_ACCESS_KEY** to be loaded into the environment if not already in place. Please refer to [setup](#setup).
 
 - Refresh the local state to ensure it is in sync with the backend
 

diff --git a/docs/diagrams/architecture.drawio b/docs/diagrams/architecture.drawio
@@ -1,53 +1,52 @@
-<mxfile host="app.diagrams.net" modified="2024-08-06T13:01:18.866Z" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36" etag="SMw1gVABOdbHb4xL4Wt8" version="24.6.5" type="device">
-  <diagram name="Page-1" id="bDBSMM_U5Odbe-mlCb5Z">
-    <mxGraphModel dx="754" dy="834" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
+<mxfile host="app.diagrams.net" agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36" version="29.2.9">
+  <diagram name="Page-1" id="3vBX3EtQdH57ldrSx5Wk">
+    <mxGraphModel dx="1395" dy="894" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
       <root>
         <mxCell id="0" />
         <mxCell id="1" parent="0" />
-        <mxCell id="ftDg6omauAznXFmt9yRm-1" value="CoPilot Usage Dashboard" style="rounded=0;whiteSpace=wrap;html=1;align=left;verticalAlign=top;" vertex="1" parent="1">
-          <mxGeometry x="40" y="40" width="710" height="560" as="geometry" />
-        </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-22" value="Get Historic&lt;div&gt;Data&lt;/div&gt;" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="1" source="ftDg6omauAznXFmt9yRm-3" target="ftDg6omauAznXFmt9yRm-11">
-          <mxGeometry relative="1" as="geometry" />
-        </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-33" value="Display Data to&lt;div&gt;User&lt;/div&gt;" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="1" source="ftDg6omauAznXFmt9yRm-3" target="ftDg6omauAznXFmt9yRm-30">
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--1" parent="1" style="rounded=0;whiteSpace=wrap;html=1;align=left;verticalAlign=top;" value="Copilot Usage Lambda &amp;amp;&amp;nbsp;&lt;div&gt;Digital&amp;nbsp;&lt;span style=&quot;background-color: transparent; color: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));&quot;&gt;Landscape Architecture&lt;/span&gt;&lt;/div&gt;" vertex="1">
+          <mxGeometry height="560" width="630" x="160" y="40" as="geometry" />
+        </mxCell>
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--2" edge="1" parent="1" source="CpdcWXR5zgmDa5A5yRl--4" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;startArrow=classic;startFill=1;" target="CpdcWXR5zgmDa5A5yRl--7" value="Get Historic &amp;amp; Team&lt;div&gt;Data&lt;/div&gt;">
+          <mxGeometry relative="1" as="geometry">
+            <mxPoint as="offset" />
+            <Array as="points">
+              <mxPoint x="540" y="240" />
+              <mxPoint x="540" y="240" />
+            </Array>
+          </mxGeometry>
+        </mxCell>
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--3" edge="1" parent="1" source="CpdcWXR5zgmDa5A5yRl--4" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" target="CpdcWXR5zgmDa5A5yRl--15" value="Display Data to&lt;div&gt;User&lt;/div&gt;">
           <mxGeometry relative="1" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-3" value="Dashboard" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=top;spacingTop=25;" vertex="1" parent="1">
-          <mxGeometry x="280" y="200" width="160" height="160" as="geometry" />
-        </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-4" value="GitHub API Toolkit&lt;div&gt;(Imported)&lt;/div&gt;" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
-          <mxGeometry x="300" y="290" width="120" height="60" as="geometry" />
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--4" parent="1" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=top;spacingTop=25;fillColor=light-dark(#FFFFFF,#333F5C);" value="Digital Landscape: Copilot Page" vertex="1">
+          <mxGeometry height="90" width="120" x="300" y="200" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-5" value="GitHub API" style="ellipse;shape=cloud;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" vertex="1" parent="1">
-          <mxGeometry x="300" y="480" width="120" height="80" as="geometry" />
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--6" parent="1" style="ellipse;shape=cloud;whiteSpace=wrap;html=1;fillColor=#d5e8d4;strokeColor=#82b366;" value="GitHub API" vertex="1">
+          <mxGeometry height="80" width="120" x="300" y="480" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-11" value="S3&lt;div&gt;Bucket&lt;/div&gt;" style="ellipse;shape=cloud;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" vertex="1" parent="1">
-          <mxGeometry x="560" y="240" width="120" height="80" as="geometry" />
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--7" parent="1" style="ellipse;shape=cloud;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" value="S3&amp;nbsp;&lt;span style=&quot;background-color: transparent; color: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));&quot;&gt;Buckets&lt;/span&gt;&lt;div&gt;&lt;ul&gt;&lt;li&gt;Historic Usage Data&lt;/li&gt;&lt;li&gt;Copilot Teams Data&lt;/li&gt;&lt;/ul&gt;&lt;/div&gt;" vertex="1">
+          <mxGeometry height="120" width="190" x="560" y="180" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-24" value="Push Update&lt;div&gt;to S3&lt;/div&gt;" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" edge="1" parent="1" source="ftDg6omauAznXFmt9yRm-19" target="ftDg6omauAznXFmt9yRm-11">
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--8" edge="1" parent="1" source="CpdcWXR5zgmDa5A5yRl--9" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" target="CpdcWXR5zgmDa5A5yRl--7" value="Push updated data&lt;div&gt;to S3&lt;/div&gt;">
           <mxGeometry relative="1" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-19" value="Lambda Function" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=top;spacingTop=25;" vertex="1" parent="1">
-          <mxGeometry x="540" y="400" width="160" height="160" as="geometry" />
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--9" parent="1" style="rounded=1;whiteSpace=wrap;html=1;verticalAlign=top;spacingTop=25;" value="Lambda Function" vertex="1">
+          <mxGeometry height="160" width="160" x="575" y="400" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-20" value="GitHub API Toolkit&lt;div&gt;(Imported)&lt;/div&gt;" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent="1">
-          <mxGeometry x="560" y="490" width="120" height="60" as="geometry" />
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--10" parent="1" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f8cecc;strokeColor=#b85450;" value="GitHub API Toolkit&lt;div&gt;(Imported)&lt;/div&gt;" vertex="1">
+          <mxGeometry height="60" width="120" x="595" y="490" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-25" value="" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fillColor=#dae8fc;strokeColor=#6c8ebf;" vertex="1" parent="1">
-          <mxGeometry x="80" y="240" width="80" height="80" as="geometry" />
-        </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-26" value="Get API Response&lt;div&gt;to Update&lt;/div&gt;&lt;div&gt;Historic Data&lt;/div&gt;" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=0.875;entryY=0.5;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" source="ftDg6omauAznXFmt9yRm-20" target="ftDg6omauAznXFmt9yRm-5">
-          <mxGeometry relative="1" as="geometry" />
-        </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-28" value="Get Live Data" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0.075;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" source="ftDg6omauAznXFmt9yRm-4" target="ftDg6omauAznXFmt9yRm-5">
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--12" edge="1" parent="1" source="CpdcWXR5zgmDa5A5yRl--10" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=0.875;entryY=0.5;entryDx=0;entryDy=0;entryPerimeter=0;startArrow=classic;startFill=1;" target="CpdcWXR5zgmDa5A5yRl--6" value="Get API Responses&lt;div&gt;to Update&lt;/div&gt;&lt;div&gt;Historic Data &amp;amp;&lt;/div&gt;&lt;div&gt;Copilot teams&lt;/div&gt;">
           <mxGeometry relative="1" as="geometry" />
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-29" value="Get Example&lt;div&gt;Dataset&lt;/div&gt;" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" source="ftDg6omauAznXFmt9yRm-3" target="ftDg6omauAznXFmt9yRm-25">
-          <mxGeometry relative="1" as="geometry" />
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--13" edge="1" parent="1" source="CpdcWXR5zgmDa5A5yRl--4" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0.075;entryDx=0;entryDy=0;entryPerimeter=0;startArrow=classic;startFill=1;" target="CpdcWXR5zgmDa5A5yRl--6" value="Get Live Data">
+          <mxGeometry relative="1" as="geometry">
+            <mxPoint x="360" y="350" as="sourcePoint" />
+          </mxGeometry>
         </mxCell>
-        <mxCell id="ftDg6omauAznXFmt9yRm-30" value="User" style="shape=umlActor;verticalLabelPosition=top;verticalAlign=bottom;html=1;outlineConnect=0;labelPosition=center;align=center;" vertex="1" parent="1">
-          <mxGeometry x="345" y="80" width="30" height="60" as="geometry" />
+        <mxCell id="CpdcWXR5zgmDa5A5yRl--15" parent="1" style="shape=umlActor;verticalLabelPosition=top;verticalAlign=bottom;html=1;outlineConnect=0;labelPosition=center;align=center;" value="User" vertex="1">
+          <mxGeometry height="60" width="30" x="345" y="80" as="geometry" />
         </mxCell>
       </root>
     </mxGraphModel>

diff --git a/docs/diagrams/architecture.png b/docs/diagrams/architecture.png
diff --git a/docs/index.md b/docs/index.md
@@ -1,42 +1,17 @@
-# GitHub Copilot Usage Dashboard
+# GitHub Copilot Usage Lambda
 
 ## Overview
 
-This project contains an AWS Lambda Function which updates the GitHub Copilot dashboard's historic information, stored within an S3 bucket.
+This project contains a Python script to automatically update the GitHub Copilot dashboard's historic information for the Copilot dashboard within the Digital Landscape. This allows trend analysis into how Software Engineers are using Copilot within ONS over time.
 
 The Copilot dashboard can be found on the Copilot tab within the Digital Landscape.
 
 [View the Digital Landscape's repository](https://github.com/ONS-Innovation/keh-digital-landscape).
 
-## Techstack Overview
-
-## Architecture Overview
-
-![Architecture Diagram](./diagrams/architecture.png)
-
-This project uses 2 major components:
-
-- The Lambda Function
-- The GitHub API Toolkit (**stored in another repository** - [Repository Link](https://github.com/ONS-Innovation/github-api-package))
-
-### The Lambda Function
-
-This component updates the dashboard's historic information, stored within an S3 bucket. The lambda imports the GitHub API Toolkit to get the API response containing the usage information. The script then adds any new data to the existing historic data within the S3 bucket.
-
-### The GitHub API Toolkit
-
-This component is an imported library which is shared across multiple GitHub tools. The toolkit allows applications to make authenticated requests to the GitHub API. It is imported and used by both the dashboard and lambda function.
-
-## High Level Data Overview
-
-### Endpoint
-
-[View docs for the Copilot usage data endpoint](https://docs.github.com/en/rest/copilot/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-organization-members).
-
-### Historic Data
-
-This section gathers data from AWS S3. The Copilot usage endpoints have a limitation where they only return the last 28 days worth of information. To get around this, the project has an AWS Lambda function which runs weekly and stores data within an S3 bucket.
-
 ## Getting Started
 
 To setup and use the project, please refer to the [README](https://github.com/ONS-Innovation/github-copilot-usage-lambda/blob/main/README.md).
+
+## Technical Documentation
+
+For information about the technical aspects of the Copilot Usage Lambda, please refer to the [Technical Documentation](./technical_documentation/overview.md).
diff --git a/docs/configuration.md → .../technical_documentation/configuration.md b/docs/configuration.md → .../technical_documentation/configuration.md
@@ -15,7 +15,7 @@ The `config.json` file contains the following:
 }
 ```
 
-### `features` Section
+### `features`
 
 This section contains feature flags that control which of the tool's features are enabled or disabled.
 
@@ -29,8 +29,6 @@ When deploying to AWS, this should be set to `false` to avoid files being writte
 
 If set to `true`, the tool will skip writing to the appropriate AWS S3 bucket and instead write data for copilot teams, historic usage, and teams history to `local_data`.
 
-**When deploying to AWS, this must be set to `false` to ensure the tool writes to AWS.**
-
 When debugging locally, you can set this to `true` to use the local configuration file. This is useful if you need to see the logs locally, without affecting the cloud deployment.
 
 ### Example During Local Testing

diff --git a/docs/technical_documentation/overview.md b/docs/technical_documentation/overview.md
@@ -0,0 +1,44 @@
+# Overview
+
+AWS Lambda Function which updates the GitHub Copilot dashboard's:
+
+- Organisation-wide historic data
+- Copilot teams
+- Teams history
+
+## Tech Stack Overview
+
+This project uses:
+
+- Python
+- AWS Lambda
+- AWS S3
+
+## Architecture Overview
+
+![Architecture Diagram](../diagrams/architecture.png)
+
+This project uses 2 major components:
+
+- The Lambda Function
+- The GitHub API Toolkit (**stored in another repository** - [Repository Link](https://github.com/ONS-Innovation/github-api-package))
+
+### The Lambda Function
+
+This component updates the Digital Landscape's Copilot dashboard data, stored within S3 buckets. The lambda imports the GitHub API Toolkit to get the API response containing the data, then adds any new data to the relevant S3 bucket.
+
+### The GitHub API Toolkit
+
+This component is an imported library which is shared across multiple GitHub tools. The toolkit allows applications to make authenticated requests to the GitHub API. It is imported and used by both the dashboard and lambda function.
+
+### Endpoint
+
+[View docs for the Copilot usage data endpoint](https://docs.github.com/en/rest/copilot/copilot-usage?apiVersion=2022-11-28#get-a-summary-of-copilot-usage-for-organization-members).
+
+### Historic Usage Data
+
+This section gathers data from AWS S3. The Copilot usage endpoints have a limitation where they only return the last 100 days worth of information. To get around this, the project has an AWS Lambda function which runs weekly and stores data within an S3 bucket.
+
+### Copilot Teams Data
+
+This section gathers a list of teams within the organisation with Copilot data and updates the S3 bucket accordingly. This allows all relevant teams to be displayed within the dashboard.