Function(s) and scripts for PurlDB to subscribe to Package updates over AP

[keshav-space]

[keshav-space]

Completed in: #49, #50 and aboutcode-org/purldb#557

To test Package subscription over ActivityPub in PurlDB follow the below steps

Inital PurlDB setup

1. Clone PurlDB git clone https://github.com/aboutcode-org/purldb.git && cd purldb
2. Checkout federatedcode integration branch git checkout federatedcode-integration
3. Run make dev && make envfile && make postgres
4. Add federartedcode host in .env file FEDERATEDCODE_HOST_URL="http://127.0.0.1:8000"
5. Run make run
6. Go to http://127.0.0.1:8001/api/package_activity/ (Package update received from FederatedCode AP will be available here)

Initial FederatedCode setup

7. Install FederatedCode (https://github.com/aboutcode-org/federatedcode?tab=readme-ov-file#quick-installation)
8. Create superuser python manage.py createsuperuser
9. Login to federatedcode admin panel http://127.0.0.1:8000/admin/
10. Go to http://127.0.0.1:8000/admin/fedcode/service/add/ and create service slect newly created superuser as user, and leave remote-actor empty.
11. Go to http://127.0.0.1:8000/admin/fedcode/remoteactor/add/ and add purldb remote actor. Populate url with http://127.0.0.1:8001/api/api/users/@purldb and put username as purldb
12. Go to http://127.0.0.1:8000/admin/fedcode/person/add/ to add purldb. For now put any thing in summary and public key, leave user empty and in Remote actor select the one created in step 11.
13. Clone this repo https://github.com/aboutcode-org/aboutcode-packages-npm-385
14. Go to go to http://127.0.0.1:8000/create-repo and add the url of your clone created in step 13 https://github.com/<YOUR-GITHUB-USERNAME>/aboutcode-packages-npm-385
15. Run python3 manage.py sync sync_scancode_scans to sync scan

Demo

Once the inital setup is done follow these steps

1. From Purldb subscribe to package update using this command python manage_purldb.py subscribe_package "pkg:npm/atlasboard"
2. Go to your fork (created in step 14) https://github.com/<YOUR-GITHUB-USERNAME>/aboutcode-packages-npm-385/blob/main/npm/atlasboard/1.1.10/scancodeio.json and make any change in scan and commit the change.
3. In FederatedCode run python3 manage.py sync sync_scancode_scans and python3 manage.py federate
4. Visit http://127.0.0.1:8001/api/package_activity/ and you will find the update for pkg:npm/atlasboard in package_activity.

Important

Replace YOUR-GITHUB-USERNAME with your GitHub user/org name.

[keshav-space]

@pombredanne Here is the instruction to test the package update in purldb using the docker build.

Note

This is very specific to running both purdb and federatedcode on same machine other wise we already have docs here https://federatedcode.readthedocs.io/en/latest/tutorial_getting_started.html and https://federatedcode.readthedocs.io/en/latest/tutorial_remote_subscribe_package_update.html

Important

This is designed to be used on linux 64-bits amd with python 3.11 or 3.12

Prerequisites

1. Obtain the IPv4 address of your machine (you can use ip addr show).

Both federatedcode and purldb need to communicate with each other. They cannot reach each other using the host's loopback address because the host's loopback is not accessible from inside the Docker containers.

Initial PurlDB setup

1. Clone PurlDB git clone https://github.com/aboutcode-org/purldb.git and cd into it.
2. Now edit the docker-compose.yml and add "Host(Your-IPv4-address)" in nginx route rule.
   nginx label should look like.

services:
  nginx:
    image: nginx
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.web.rule=
        Host(`127.0.0.1`)
        || Host(`localhost`) || Host(`Your-IPv4-address`)"
      - "traefik.http.routers.web.entrypoints=web"

3. Edit the docker_purldb.env file and add these lines.

FEDERATEDCODE_HOST_URL="http://Your-IPv4-address:81/"
ALLOWED_HOSTS="Your-IPv4-address"

4. Create purldb network docker network create purldb
5. Build image docker compose build
6. docker compose up
7. Go to http://Your-IPv4-address/api/package_activity/ (Package update received from FederatedCode AP will be available here)

Initial FederatedCode setup

8. Clone FederatedCode https://github.com/aboutcode-org/federatedcode and cd into it.
9. Edit docker-compose.yml and change port exposed by nginx to 81 and 442 since these ports are already in use by purldb.

   nginx:
     image: nginx
     ports:
      - 81:80
      - 442:443

10. Now edit the docker.env file and add these lines

ALLOWED_HOSTS="Your-IPv4-address"
CSRF_TRUSTED_ORIGINS="http://Your-IPv4-address:81"

11. Build image docker compose build
12. docker compose up
13. Create superuser docker compose -f docker-compose.yml run --rm web python manage.py createsuperuser
14. Login to federatedcode admin panel http://Your-IPv4-address:81/admin/
15. Go to http://Your-IPv4-address:81/admin/fedcode/service/add/ and create service select newly created superuser as user, and leave remote-actor empty.
16. Go to http://Your-IPv4-address:81/admin/fedcode/remoteactor/add/ and add purldb remote actor. Populate url with http://Your-IPv4-address/api/users/@purldb/ and put username as purldb
17. Go to http://Your-IPv4-address:81/admin/fedcode/person/add/ to add purldb. For now put any thing in summary and public key, leave user empty and in Remote actor select the one created in step 16.
18. Fork this repo https://github.com/aboutcode-org/aboutcode-packages-npm-385
19. Go to go to http://Your-IPv4-address:81/create-repo and add the url of your clone created in step 18 https://github.com//aboutcode-packages-npm-385
20. Run docker compose -f docker-compose.yml run --rm web python3 manage.py sync sync_scancode_scans to sync scan

Demo

Once the initial setup is done follow these steps

1. cd into purldb clone and subscribe to package update using this command docker compose -f docker-compose.yml run --rm web python manage_purldb.py subscribe_package "pkg:npm/atlasboard" . On successful subscribe you will get this response {"status": "success", "message": "Successfully subscribed to package pkg:npm/atlasboard"}
2. Go to your fork (created in step 18) https://github.com/<YOUR-GITHUB-USERNAME>/aboutcode-packages-npm-385/blob/main/npm/atlasboard/1.1.10/scancodeio.json and make any change in scan and commit the change.
3. In FederatedCode run docker compose -f docker-compose.yml run --rm web python3 manage.py sync sync_scancode_scans and docker compose -f docker-compose.yml run --rm web python3 manage.py federate
4. Visit PurlDB http://Your-IPv4-address/api/package_activity/ and you will find the update for pkg:npm/atlasboard in package_activity.

Important

Replace YOUR-GITHUB-USERNAME with your GitHub user/org name.