Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,9 @@ We employ several techniques to output a lot of value in a unit of time:
While some software houses lose \~30% of their performance (due to miscommunications, etc.), we stay way below 1% (the exact number is hard to measure when the amount of waste is so low).
As mentioned above, we tend to pause work on a ticket when we are not sure, choosing a small delay on a particular task over potentially having to discard work due to a bad assumption. Our clients prefer it this way.
10. Use modern IDEs. Pycharm, Cursor, Windsurf or VisualStudio with a LLM plugin. Some people are trying to use VIM with plugins for LLMs, but nowadays it's mostly Cursor/Windsurf/Pycharm.
11. Use LLMs (ChatGPT, GitHub Copilot, Claude etc) to speed up the work on the code, though watch every single byte of the diff like it's been written by a party you shouldn't trust.
11. Use LLMs to speed up coding, but we still own QA: read every byte and aim for our normal quality. For small,
contained “one‑shot” changes we may relax QA—and even skip review—if the change is low‑risk and self‑explanatory (or
briefly noted in the repository for future maintainers). See the [LLM‑assisted coding agreement](agreements.md#llm-assisted-coding).

# Code Review

Expand Down
50 changes: 50 additions & 0 deletions agreements.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,56 @@ That's one of the reasons we have someone in that role for every project.
This also allows for other solutions such as, for example, splitting the cost of a fix between two clients, discounts etc - something you couldn't do on your own.
Fortunately, with a dedicated client contact person, you don't have to!

## LLM-Assisted Coding

Motivation

- LLMs let us move faster (including during reviews), but cumulative low-quality changes can erode long-term
maintainability — one of our trademarks.
- Responsibility remains human: both the author and the reviewer own the quality of what ships, regardless of tooling.
- We may intentionally accept slightly lower quality for one‑shot, low‑risk changes to gain speed, but we do so when it is
clearly safe.
- There’s a trade‑off: more features with less quality vs fewer features with higher quality. Over time, lower quality
makes teams ship fewer features due to maintenance drag; we optimize for sustained maintainability while using
one‑shots to keep speed where risk is minimal.
- Practical observation: there’s a limit to how much an LLM can safely rewrite in a day without agreed QA.

**Never include secrets or client‑sensitive data in prompts or any saved notes.**

Scope and rules

- Default path: normal QA and review.
- One‑shot changes (small, well‑separated changes with limited implications on the rest of the codebase, independently
judgeable, and reasonably redoable within one day) may be merged with relaxed quality and may skip review if all of
the following hold:
- The change is low‑risk, contained, and has minimal blast radius.
- The author performs a basic functional check.
- The change is either self‑explanatory on its own, or (if not obvious) accompanied by a short note capturing the
ask/purpose/constraints so the next person can pick it up without guesswork. The note must live in the repository
(in the code or an accompanying `.md`). A PR/commit message alone is not sufficient.
- The one‑shot relaxation is mentioned explicitly in the PR/commit (e.g. include `llm-one-shot`).
- The change is kept standalone (not tucked into a larger PR).
- Larger/core contributions done with LLM assistance require a case‑by‑case, agreed QA/release plan. If we must merge a
larger LLM change before full QA, do it consciously with that plan in place. Do not merge low‑quality core code
without such agreement.
- Use case‑by‑case judgement for one‑shots; if uncertain, take the normal QA/review path.

Author responsibilities

- Read every byte of generated code, functionally test it, and prepare the change for regular review (structure the diff,
write tests/docs where applicable).
- When using the one‑shot relaxation, make the change pick‑up‑able: if it is not trivial/self‑explanatory, add a short
note capturing the ask/purpose/constraints in the repository (in the code or an accompanying `.md`). If it is obvious,
no note is needed. Do not rely on PR/commit messages as the only place the intent is recorded.

Review expectations

- Normal review applies; the reviewer may use or not use LLM tooling.
- If a one‑shot change is being reviewed anyway, confirm it is low‑risk/contained and either self‑explanatory or
accompanied by a brief note in the repository as above (not only in the PR/commit message).
- Confirm maintainability isn’t degraded (structure, naming, tests, docs), and that no secrets are stored in any notes or
prompts.

## Fast track decisions via Slack instead of standard Sociocracy approach

At Reef Technologies, we mostly make decisions in our weekly Sociocracy meetings.
Expand Down
14 changes: 14 additions & 0 deletions docs/Code_Review.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ As with every policy, if something is bad and you care about it enough, feel fre
and the governance process, operating in front of you and with you, will decide whether to change (most likely yes).
</details>

Responsibility remains with humans: authors and reviewers are accountable for code quality whether or not LLMs were
used. See the [LLM‑Assisted Coding agreement](../agreements.md#llm-assisted-coding).
Tiny, low‑risk one‑shot LLM changes may skip review if they meet the agreement’s conditions (well‑separated with limited
implications and low blast radius, basic functional check, self‑explanatory or briefly noted in the repository).
Otherwise, request review as usual.

## When/how to request a code review

1. If you are a developer, always perform basic functional testing (manually!) of your code and self-review your PR
Expand All @@ -31,6 +37,11 @@ and the governance process, operating in front of you and with you, will decide
commits instead, like “Fix XYZ”, “Add tests”. <details><summary>Note.</summary>This is a limitation of GitHub and a
primary motivator to consider moving development off to gitlab / gerrit, though as of writing this document we have
not decided to switch.</details>
6. Follow the [LLM‑Assisted Coding agreement](../agreements.md#llm-assisted-coding). If you’re using the LLM one‑shot
relaxation, keep the change standalone (not tucked into larger PRs), make it self‑explanatory or add a brief note,
ensure the note lives in the repository (in the code or an accompanying `.md`, not only in the PR/commit message),
read/functionally verify all generated code before requesting review or merging, and mention `llm-one-shot` in the
PR/commit.

## How to review

Expand Down Expand Up @@ -101,6 +112,9 @@ Note: not all of these apply to all PRs, of course.
12. Performance / memory considerations. Will it OOM in a corner case after the change?
13. Self-healing. If a (network, HDD) device or a part of the system breaks temporarily, will the rest of the system
recover automatically or will it require manual intervention?
14. If a one‑shot LLM change is being reviewed: is it low‑risk/contained and either self‑explanatory or briefly noted in
the repository?
15. Maintainability not degraded (structure, naming, tests, docs); no secrets in notes/prompts.

## How to resolve the reviewer’s comments

Expand Down