Codex API

Pre-requisites

• Install Docker Desktop
• Install Homebrew
• Install nvm and Node.js >= v16
• Install ejson (more on ejson workflow in the docs)

$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
$ nvm list-remote 16
$ nvm install v16.X.X
$ brew tap shopify/shopify && brew install ejson

Getting Started

• Copy the "EJSON Key (Development)" from 1Password into /opt/ejson/keys/<key-id>
• Copy the NPM_TASKFORCESH_TOKEN from 1Password into your shell environment:
  • ex. echo 'export NPM_TASKFORCESH_TOKEN=<token value>' >> ~/.zshenv
• Run docker-compose -f docker-compose.development.yml up -d db redis to start PostgreSQL and Redis
  • You can skip the -f option with a simple soft-link sudo ln -s docker-compose.development.yml docker-compose.yml
• Ensure database is up (docker-compose logs db --follow to check) and migrate/seed the database:
  • npm run decrypt:dev && npm run build && npm run db:setup

Developing

Start the server in dev mode, npm run dev (this includes a file watch for your changes). Happy hacking!

Configuration

Environment-specific configuration is now packaged as part of the application and encrypted using ejson. You can still override / configure the application using a .env file or explicitly setting ENVAR1=somevalue as part of the start command. For development, you need only concern yourself with src/config/config.development.ejson.

Adding new Configuration

To add new configuration variables, create the appropriate keys in the config.<environment>.ejson files, then encrypt them ejson encrypt src/config/config.<environment>.ejson. If the key does not sensitive or secret information, it should be prefixed with an underscore (ex. "_MY_CONFIG_VARIABLE": "server1.com").

SSL / HTTPS

Check out the [README-https.md] for more detail. To set up HTTPS on your local server is easy:

1. Ensure you have dev.usecodex.com as localhost entry in your /etc/hosts file:
   • echo "127.0.0.1 dev.usecodex.com" | sudo tee -a /etc/hosts
2. Install NGINX (uses Homebrew): npm run nginx:install
3. Configure NGINX: npm run nginx:configure
4. You're done! curl https://dev.usecodex.com

The SSL certificate expires every 90 days, but is renewed continually on a Digital Ocean droplet. Get the certificates from:

• /etc/letsencrypt/live/dev.usecodex.com/fullchain.pem
• /etc/letsencrypt/live/dev.usecodex.com/privkey.pem

Docker

You can build a runnable docker container with npm run build:docker. To run the container in the local Docker Compose network:

# Grab dev key pair from 1Password
export EJSON_PUBLIC_KEY=...
export EJSON_PRIVATE_KEY=...
npm run start:docker

Recommended: to debug the built image, you can use dive to inspect the filesystem, layer by built layer.

Signing up

By default when you sign up in development, it will not automatically register you (you will not receive the email, but it will be printed on the console for you to copy paste and visit). If you want to skip this step, set AUTO_VERIFY_USERS=true in your working .env.

Test Suite

Tests live in tests/, run them with npm t or npm test. We use:

• Mocha as test runner
• Chai for expectations
• Chai HTTP for HTTP tests
• ts-mockito for mocking, verification, etc.

Integration Tests

In order to run integration tests, you'll need a Github Personal Access Token GITHUB_PAT, you can create one here.

Database Administration

The PostgreSQL data is stored (volume mounted into the Docker container) in your project directory under .pgdata; it is a git ignored. This gives you the flexibility to manage different copies of your database with different schema versions or data sets. If the .pgdata directory is missing, starting the db Docker container will recreate and reinitialize the entire database. You can keep a copy of your current database by renaming the directory:

$ docker-compose stop db
$ mv .pgdata .pgdata.original-branch
$ git checkout -b a-new-branch
$ docker-compose up db -d

Similarly, you can restore that copy by swapping the directories around:

$ docker-compose stop db
$ mv .pgdata .pgdata.a-new-branch
$ git checkout original-branch
$ mv .pgdata.original-branch .pgdata
$ docker-compose up db -d

pgAdmin

pgAdmin is useful for interacting with the database, start it with docker-compose up -d pgAdmin:

• Visit http://localhost:8080:
  • Login: admin@usecodex.com
  • Password: password
• Create a server in pgAdmin:
  • Host: db (this is the PostgreSQL hostname within the docker-compose network)
  • Port: 5432
  • User: codex
  • Password: password

Database ORM & Migrations

We use TypeORM.

• Database setup: npm run db:setup
• Generate migration: npm run db:migration:generate --migration_name=<your migration name>
  • PostgreSQL server must be running (docker-compose up -d db)
• Reset database: npm run db:reset THIS WILL NUKE YOUR CURRENT DATASET

CI/CD

Continuous Integration is achieved with GitHub Actions. Every commit pushed to GitHub runs the test suite automatically, and is required to pass to merge into development, staging, production, and main branches. Have a look at npm_test.yml for more detail.

Successful merges to development, staging, production, and main all produce a Docker image published to the Github Container Registry. Check out publish_image.yml to see how. The available images are tagged as such:

• ghcr.io/usecodex/codex-api:<git-sha> the short-form git SHA of the merged commit
• ghcr.io/usecodex/codex-api:<branch-name> the latest merge into the respective persistent branch (ex. development)

Slack

Create a Slack app

Create an app on the Slack web app. - https://api.slack.com/apps?new_app=1 👉 Create a new Slack app using the manifest file in src/modules/mire/slack/config/manifest.json

Server settings

Add the following environment variables to your config.local.json file. Their values can be found in the Slack App you created. The SLACK_STATE_SECRET is whatever you want to define it as.

{
	"SLACK_CLIENT_ID": "",
  "SLACK_CLIENT_SECRET": "",
  "SLACK_SIGNING_SECRET": "",
  "SLACK_STATE_SECRET": "shh...",
  "SLACK_APP_TOKEN": ""
}

💡 Make sure mire-slack-app feature is enabled in your database

Install into a workspace by following this URL: https://<ngrok_url>/slack/events/slack/install (make sure this is your ngrok & not your local or it won't work)

Troubleshooting

I can't <build the app | run the app | run the tests>!

npm run decrypt:dev

codex-api@1.x.x decrypt:dev ejson decrypt -o src/config/config.development.json src/config/config.development.ejson

Decryption failed: couldn't read key file (open /opt/ejson/keys/: no such file or directory)

If this error occurs, it's because you don't have the appropriate ejson key. Grab it from 1Password and put it in your /opt/ejson/keys/ directory. The file name is the public key ("username" in 1Password), while the content of the file is the private key ("password" in 1Password).

Error: ENOENT: no such file or directory, open '../id_rsa_priv.pem'

Can't find the .pem file? * Run npm run build