Skip to content

Added new internal attributes $createdBy and$ updatedBy #680

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ArnabChatterjee20k wants to merge 18 commits into
base: main
Choose a base branch
from
Open

Added new internal attributes $createdBy and$ updatedBy #680

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
f77eb9a
added new internal attributes to the adapters
ArnabChatterjee20k Sep 1, 2025
484d045
added get user method to the Authorization
ArnabChatterjee20k Sep 1, 2025
1bf9fb8
added helper methods and validators
ArnabChatterjee20k Sep 1, 2025
28a979e
updated existing methods and tests
ArnabChatterjee20k Sep 1, 2025
8a85666
updated attribute limit tests
ArnabChatterjee20k Sep 2, 2025
a6789e3
added null checker for the createdBy
ArnabChatterjee20k Sep 2, 2025
2a4805e
updated unsetting of createdby and updatedby in the find
ArnabChatterjee20k Sep 2, 2025
67791d1
updated sqlite adapter
ArnabChatterjee20k Sep 2, 2025
6e40e79
* add indexes for createdBy and updatedBy fields in MariaDB and Postg…
ArnabChatterjee20k Sep 15, 2025
5e17969
updated failing test due to collection name
ArnabChatterjee20k Sep 15, 2025
1140b58
updated collection names
ArnabChatterjee20k Sep 15, 2025
c20f225
linting
ArnabChatterjee20k Sep 15, 2025
e706390
updated updatedby in updatedocuments
ArnabChatterjee20k Sep 15, 2025
2cd4666
fixed updatedBy for increment and decrement attribute
ArnabChatterjee20k Sep 16, 2025
731fea0
Merge remote-tracking branch 'upstream/main' into dat-677
ArnabChatterjee20k Sep 16, 2025
f4a449b
updated sql with the internal keys and sqlite with the new internal a…
ArnabChatterjee20k Sep 16, 2025
e95802f
code quality checks
ArnabChatterjee20k Sep 16, 2025
bd8a8ac
updated postgres adapter and tests
ArnabChatterjee20k Sep 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions src/Database/Adapter.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1233,6 +1233,7 @@ protected function escapeWildcards(string $value): string
* @param string $attribute
* @param int|float $value
* @param string $updatedAt
* @param string|null $updatedBy
* @param int|float|null $min
* @param int|float|null $max
* @return bool
Expand All @@ -1244,6 +1245,7 @@ abstract public function increaseDocumentAttribute(
string $attribute,
int|float $value,
string $updatedAt,
?string $updatedBy,
Copy link
Member

@abnegate abnegate Sep 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this added for any SQL adapters 🤔

Copy link
Contributor Author

@ArnabChatterjee20k ArnabChatterjee20k Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its present -> https://github.com/utopia-php/database/pull/680/files#diff-14149d1580fa54670629e57c6d684530e50ab17fb75d888418906c30b1595225R1261

int|float|null $min = null,
int|float|null $max = null
): bool;
Expand Down
17 changes: 15 additions & 2 deletions src/Database/Adapter/MariaDB.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,8 @@ public function createCollection(string $name, array $attributes = [], array $in
_uid VARCHAR(255) NOT NULL,
_createdAt DATETIME(3) DEFAULT NULL,
_updatedAt DATETIME(3) DEFAULT NULL,
_createdBy VARCHAR(255) DEFAULT NULL,
_updatedBy VARCHAR(255) DEFAULT NULL,
_permissions MEDIUMTEXT DEFAULT NULL,
PRIMARY KEY (_id),
" . \implode(' ', $attributeStrings) . "
Expand All @@ -171,13 +173,17 @@ public function createCollection(string $name, array $attributes = [], array $in
UNIQUE KEY _uid (_uid, _tenant),
KEY _created_at (_tenant, _createdAt),
KEY _updated_at (_tenant, _updatedAt),
KEY _created_by (_tenant, _createdBy),
KEY _updated_by (_tenant, _updatedBy),
KEY _tenant_id (_tenant, _id)
";
} else {
$collection .= "
UNIQUE KEY _uid (_uid),
KEY _created_at (_createdAt),
KEY _updated_at (_updatedAt)
KEY _updated_at (_updatedAt),
KEY _created_by (_createdBy),
KEY _updated_by (_updatedBy)
";
}

Expand Down Expand Up @@ -827,6 +833,8 @@ public function createDocument(Document $collection, Document $document): Docume
$attributes = $document->getAttributes();
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_createdBy'] = $document->getCreatedBy();
$attributes['_updatedBy'] = $document->getUpdatedBy();
$attributes['_permissions'] = \json_encode($document->getPermissions());

if ($this->sharedTables) {
Expand Down Expand Up @@ -953,6 +961,7 @@ public function updateDocument(Document $collection, string $id, Document $docum
$attributes = $document->getAttributes();
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_updatedBy'] = $document->getUpdatedBy();
$attributes['_permissions'] = json_encode($document->getPermissions());

$name = $this->filter($collection);
Expand Down Expand Up @@ -1241,6 +1250,7 @@ public function getUpsertStatement(
* @param string $attribute
* @param int|float $value
* @param string $updatedAt
* @param string|null $updatedBy
* @param int|float|null $min
* @param int|float|null $max
* @return bool
Expand All @@ -1252,6 +1262,7 @@ public function increaseDocumentAttribute(
string $attribute,
int|float $value,
string $updatedAt,
?string $updatedBy,
int|float|null $min = null,
int|float|null $max = null
): bool {
Expand All @@ -1265,7 +1276,8 @@ public function increaseDocumentAttribute(
UPDATE {$this->getSQLTable($name)}
SET
`{$attribute}` = `{$attribute}` + :val,
`_updatedAt` = :updatedAt
`_updatedAt` = :updatedAt,
`_updatedBy` = :updatedBy
WHERE _uid = :_uid
{$this->getTenantQuery($collection)}
";
Expand All @@ -1278,6 +1290,7 @@ public function increaseDocumentAttribute(
$stmt->bindValue(':_uid', $id);
$stmt->bindValue(':val', $value);
$stmt->bindValue(':updatedAt', $updatedAt);
$stmt->bindValue(':updatedBy', $updatedBy);

if ($this->sharedTables) {
$stmt->bindValue(':_tenant', $this->tenant);
Expand Down
2 changes: 1 addition & 1 deletion src/Database/Adapter/Pool.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -475,7 +475,7 @@ protected function getAttributeProjection(array $selections, string $prefix): st
return $this->delegate(__FUNCTION__, \func_get_args());
}

public function increaseDocumentAttribute(string $collection, string $id, string $attribute, float|int $value, string $updatedAt, float|int|null $min = null, float|int|null $max = null): bool
public function increaseDocumentAttribute(string $collection, string $id, string $attribute, float|int $value, string $updatedAt, ?string $updatedBy, float|int|null $min = null, float|int|null $max = null): bool
{
return $this->delegate(__FUNCTION__, \func_get_args());
}
Expand Down
17 changes: 15 additions & 2 deletions src/Database/Adapter/Postgres.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,8 @@ public function createCollection(string $name, array $attributes = [], array $in
" . $sqlTenant . "
\"_createdAt\" TIMESTAMP(3) DEFAULT NULL,
\"_updatedAt\" TIMESTAMP(3) DEFAULT NULL,
\"_createdBy\" VARCHAR(255) DEFAULT NULL,
\"_updatedBy\" VARCHAR(255) DEFAULT NULL,
" . \implode(' ', $attributeStrings) . "
_permissions TEXT DEFAULT NULL
);
Expand All @@ -247,13 +249,17 @@ public function createCollection(string $name, array $attributes = [], array $in
CREATE UNIQUE INDEX \"{$namespace}_{$this->tenant}_{$id}_uid\" ON {$this->getSQLTable($id)} (\"_uid\", \"_tenant\");
CREATE INDEX \"{$namespace}_{$this->tenant}_{$id}_created\" ON {$this->getSQLTable($id)} (_tenant, \"_createdAt\");
CREATE INDEX \"{$namespace}_{$this->tenant}_{$id}_updated\" ON {$this->getSQLTable($id)} (_tenant, \"_updatedAt\");
CREATE INDEX \"{$namespace}_{$this->tenant}_{$id}_created_by\" ON {$this->getSQLTable($id)} (_tenant, \"_createdBy\");
CREATE INDEX \"{$namespace}_{$this->tenant}_{$id}_updated_by\" ON {$this->getSQLTable($id)} (_tenant, \"_updatedBy\");
CREATE INDEX \"{$namespace}_{$this->tenant}_{$id}_tenant_id\" ON {$this->getSQLTable($id)} (_tenant, _id);
";
} else {
$collection .= "
CREATE UNIQUE INDEX \"{$namespace}_{$id}_uid\" ON {$this->getSQLTable($id)} (\"_uid\");
CREATE INDEX \"{$namespace}_{$id}_created\" ON {$this->getSQLTable($id)} (\"_createdAt\");
CREATE INDEX \"{$namespace}_{$id}_updated\" ON {$this->getSQLTable($id)} (\"_updatedAt\");
CREATE INDEX \"{$namespace}_{$id}_created_by\" ON {$this->getSQLTable($id)} (\"_createdBy\");
CREATE INDEX \"{$namespace}_{$id}_updated_by\" ON {$this->getSQLTable($id)} (\"_updatedBy\");
";
}

Expand Down Expand Up @@ -961,6 +967,8 @@ public function createDocument(Document $collection, Document $document): Docume
$attributes = $document->getAttributes();
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_createdBy'] = $document->getCreatedBy();
$attributes['_updatedBy'] = $document->getUpdatedBy();
$attributes['_permissions'] = \json_encode($document->getPermissions());

if ($this->sharedTables) {
Expand Down Expand Up @@ -1075,6 +1083,8 @@ public function updateDocument(Document $collection, string $id, Document $docum
$attributes = $document->getAttributes();
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_createdBy'] = $document->getCreatedBy();
$attributes['_updatedBy'] = $document->getUpdatedBy();
$attributes['_permissions'] = json_encode($document->getPermissions());

$name = $this->filter($collection);
Expand Down Expand Up @@ -1343,12 +1353,13 @@ protected function getUpsertStatement(
* @param string $attribute
* @param int|float $value
* @param string $updatedAt
* @param string|null $updatedBy
* @param int|float|null $min
* @param int|float|null $max
* @return bool
* @throws DatabaseException
*/
public function increaseDocumentAttribute(string $collection, string $id, string $attribute, int|float $value, string $updatedAt, int|float|null $min = null, int|float|null $max = null): bool
public function increaseDocumentAttribute(string $collection, string $id, string $attribute, int|float $value, string $updatedAt, ?string $updatedBy, int|float|null $min = null, int|float|null $max = null): bool
{
$name = $this->filter($collection);
$attribute = $this->filter($attribute);
Expand All @@ -1360,7 +1371,8 @@ public function increaseDocumentAttribute(string $collection, string $id, string
UPDATE {$this->getSQLTable($name)}
SET
\"{$attribute}\" = \"{$attribute}\" + :val,
\"_updatedAt\" = :updatedAt
\"_updatedAt\" = :updatedAt,
\"_updatedBy\" = :updatedBy
WHERE _uid = :_uid
{$this->getTenantQuery($collection)}
";
Expand All @@ -1373,6 +1385,7 @@ public function increaseDocumentAttribute(string $collection, string $id, string
$stmt->bindValue(':_uid', $id);
$stmt->bindValue(':val', $value);
$stmt->bindValue(':updatedAt', $updatedAt);
$stmt->bindValue(':updatedBy', $updatedBy);

if ($this->sharedTables) {
$stmt->bindValue(':_tenant', $this->tenant);
Expand Down
36 changes: 35 additions & 1 deletion src/Database/Adapter/SQL.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -408,6 +408,14 @@ public function getDocument(Document $collection, string $id, array $queries = [
$document['$updatedAt'] = $document['_updatedAt'];
unset($document['_updatedAt']);
}
if (\array_key_exists('_createdBy', $document)) {
$document['$createdBy'] = $document['_createdBy'];
unset($document['_createdBy']);
}
if (\array_key_exists('_updatedBy', $document)) {
$document['$updatedBy'] = $document['_updatedBy'];
unset($document['_updatedBy']);
}
if (\array_key_exists('_permissions', $document)) {
$document['$permissions'] = json_decode($document['_permissions'] ?? '[]', true);
unset($document['_permissions']);
Expand Down Expand Up @@ -468,6 +476,14 @@ public function updateDocuments(Document $collection, Document $updates, array $
$attributes['_createdAt'] = $updates->getCreatedAt();
}

if (!empty($updates->getCreatedBy())) {
$attributes['_createdBy'] = $updates->getCreatedBy();
}

if (!empty($updates->getUpdatedBy())) {
$attributes['_updatedBy'] = $updates->getUpdatedBy();
}

if ($updates->offsetExists('$permissions')) {
$attributes['_permissions'] = json_encode($updates->getPermissions());
}
Expand Down Expand Up @@ -1053,10 +1069,12 @@ public function getAttributeWidth(Document $collection): int
* `_tenant` int => 4 bytes
* `_createdAt` datetime(3) => 7 bytes
* `_updatedAt` datetime(3) => 7 bytes
* `_createdBy` varchar(255) => 1021 (4 * 255 + 1) bytes
* `_updatedBy` varchar(255) => 1021 (4 * 255 + 1) bytes
* `_permissions` mediumtext => 20
*/

$total = 1067;
$total = 3109;

$attributes = $collection->getAttributes()['attributes'];

Expand Down Expand Up @@ -1892,6 +1910,8 @@ protected function getAttributeProjection(array $selections, string $prefix): st
'$permissions',
'$createdAt',
'$updatedAt',
'$createdBy',
'$updatedBy'
];

$selections = \array_diff($selections, [...$internalKeys, '$collection']);
Expand Down Expand Up @@ -1919,6 +1939,8 @@ protected function getInternalKeyForAttribute(string $attribute): string
'$tenant' => '_tenant',
'$createdAt' => '_createdAt',
'$updatedAt' => '_updatedAt',
'$createdBy' => '_createdBy',
'$updatedBy' => '_updatedBy',
'$permissions' => '_permissions',
default => $attribute
};
Expand Down Expand Up @@ -2013,6 +2035,8 @@ public function createDocuments(Document $collection, array $documents): array
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_permissions'] = \json_encode($document->getPermissions());
$attributes['_createdBy'] = $document->getCreatedBy();
$attributes['_updatedBy'] = $document->getUpdatedBy();

if (!empty($document->getSequence())) {
$attributes['_id'] = $document->getSequence();
Expand Down Expand Up @@ -2127,6 +2151,8 @@ public function upsertDocuments(
$attributes['_uid'] = $document->getId();
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_createdBy'] = $document->getCreatedBy();
$attributes['_updatedBy'] = $document->getUpdatedBy();
$attributes['_permissions'] = \json_encode($document->getPermissions());
Comment on lines +2154 to 2156
Copy link
Contributor

@coderabbitai coderabbitai bot Sep 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Upsert path sets auditing fields: verify adapter update clause

Attributes include _updatedBy, but MariaDB’s increment upsert path doesn’t update it yet. Align adapter upsert (increment) to set _updatedBy too (see related comment in MariaDB::getUpsertStatement).

🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "PWD: $(pwd)"
echo "---- locate SQL.php ----"
rg -n --hidden --no-ignore -S "src/Database/Adapter/SQL.php" || true
echo "---- show SQL.php 2130-2200 ----"
if [ -f src/Database/Adapter/SQL.php ]; then
  sed -n '2130,2200p' src/Database/Adapter/SQL.php || true
else
  echo "src/Database/Adapter/SQL.php not found"
fi
echo "---- search for getUpsertStatement definitions ----"
rg -n --hidden --no-ignore -S "function\s+getUpsertStatement" || true
echo "---- search for class MariaDB ----"
rg -n --hidden --no-ignore -S "class\s+MariaDB" || true
echo "---- show any MariaDB file contents around getUpsertStatement if found ----"
for f in $(rg -n --hidden --no-ignore -S "class\s+MariaDB" -g '!vendor' --files-with-matches || true); do
  echo "---- $f ----"
  sed -n '1,240p' "$f" || true
done
echo "---- search for _updatedBy/_createdBy usages ----"
rg -n --hidden --no-ignore -S "_updatedBy|_createdBy" || true
echo "---- search for ON DUPLICATE KEY UPDATE / upsert occurrences ----"
rg -n --hidden --no-ignore -S "ON DUPLICATE KEY UPDATE|ON CONFLICT|\\bupsert\\b" || true

Length of output: 9672

🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "PWD: $(pwd)"
echo "---- show MariaDB.php 1100-1320 ----"
if [ -f src/Database/Adapter/MariaDB.php ]; then
  sed -n '1100,1320p' src/Database/Adapter/MariaDB.php || true
else
  echo "src/Database/Adapter/MariaDB.php not found"
fi

echo "---- show SQL.php 1520-1620 (abstract getUpsertStatement) ----"
if [ -f src/Database/Adapter/SQL.php ]; then
  sed -n '1520,1620p' src/Database/Adapter/SQL.php || true
else
  echo "src/Database/Adapter/SQL.php not found"
fi

echo "---- show SQL.php 1980-2260 (createOrUpdateDocuments area) ----"
if [ -f src/Database/Adapter/SQL.php ]; then
  sed -n '1980,2260p' src/Database/Adapter/SQL.php || true
else
  echo "src/Database/Adapter/SQL.php not found"
fi

echo "---- search for getUpsertStatement call sites ----"
rg -n --hidden --no-ignore -S "getUpsertStatement\(" || true

echo "---- search for ON DUPLICATE KEY UPDATE in MariaDB.php ----"
rg -n --hidden --no-ignore -S "ON DUPLICATE KEY UPDATE|ON DUPLICATE KEY" src/Database/Adapter/MariaDB.php || true

echo "---- search for _updatedBy occurrences in MariaDB.php ----"
rg -n --hidden --no-ignore -S "_updatedBy" src/Database/Adapter/MariaDB.php || true

Length of output: 20899

Upsert increment path must update _updatedBy (MariaDB adapter)

createOrUpdateDocuments sets _updatedBy (src/Database/Adapter/SQL.php:2183-2185), but MariaDB::getUpsertStatement’s increment branch only updates the incremented attribute and _updatedAt — add _updatedBy to the increment ON DUPLICATE KEY UPDATE clause (e.g. include $getUpdateClause('_updatedBy') so the sharedTables IF logic is preserved).

🤖 Prompt for AI Agents 
In src/Database/Adapter/SQL.php around lines 2183-2185, createOrUpdateDocuments
sets _updatedBy but the MariaDB::getUpsertStatement increment branch only
updates the incremented attribute and _updatedAt; modify the increment ON
DUPLICATE KEY UPDATE clause to also update _updatedBy by adding
$getUpdateClause('_updatedBy') (ensuring you insert it in the same
IF(sharedTables) logic path so the sharedTables behavior and clause ordering
remain consistent).


if (!empty($document->getSequence())) {
Expand Down Expand Up @@ -2504,6 +2530,14 @@ public function find(Document $collection, array $queries = [], ?int $limit = 25
$results[$index]['$updatedAt'] = $document['_updatedAt'];
unset($results[$index]['_updatedAt']);
}
if (\array_key_exists('_createdBy', $document)) {
$results[$index]['$createdBy'] = $document['_createdBy'];
unset($results[$index]['_createdBy']);
}
if (\array_key_exists('_updatedBy', $document)) {
$results[$index]['$updatedBy'] = $document['_updatedBy'];
unset($results[$index]['_updatedBy']);
}
if (\array_key_exists('_permissions', $document)) {
$results[$index]['$permissions'] = \json_decode($document['_permissions'] ?? '[]', true);
unset($results[$index]['_permissions']);
Expand Down
8 changes: 8 additions & 0 deletions src/Database/Adapter/SQLite.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,8 @@ public function createCollection(string $name, array $attributes = [], array $in
{$tenantQuery}
`_createdAt` DATETIME(3) DEFAULT NULL,
`_updatedAt` DATETIME(3) DEFAULT NULL,
`_createdBy` VARCHAR(255) DEFAULT NULL,
`_updatedBy` VARCHAR(255) DEFAULT NULL,
`_permissions` MEDIUMTEXT DEFAULT NULL".(!empty($attributes) ? ',' : '')."
" . \substr(\implode(' ', $attributeStrings), 0, -2) . "
)
Expand Down Expand Up @@ -525,6 +527,8 @@ public function createDocument(Document $collection, Document $document): Docume
$attributes = $document->getAttributes();
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_createdBy'] = $document->getCreatedBy();
$attributes['_updatedBy'] = $document->getUpdatedBy();
$attributes['_permissions'] = json_encode($document->getPermissions());

if ($this->sharedTables) {
Expand Down Expand Up @@ -647,6 +651,8 @@ public function updateDocument(Document $collection, string $id, Document $docum
$attributes = $document->getAttributes();
$attributes['_createdAt'] = $document->getCreatedAt();
$attributes['_updatedAt'] = $document->getUpdatedAt();
$attributes['_createdBy'] = $document->getCreatedBy();
$attributes['_updatedBy'] = $document->getUpdatedBy();
$attributes['_permissions'] = json_encode($document->getPermissions());

if ($this->sharedTables) {
Expand Down Expand Up @@ -1032,6 +1038,8 @@ protected function getSQLIndex(string $collection, string $id, string $type, arr
'$id' => ID::custom('_uid'),
'$createdAt' => '_createdAt',
'$updatedAt' => '_updatedAt',
'$createdBy' => '_createdBy',
'$updatedBy' => '_updatedBy',
default => $attribute
}, $attributes);

Expand Down
Loading
Loading