Rclone #70

nebukadhezer · 2026-01-29T17:49:37Z

Changelog Description

Added Rclone as a provider. All is done via subprocess. so rclone needs to be present on PATH or the absolute path needs to be set.

Additional review information

need to double check the linter so leaving this as a draft until then

Testing notes:

Tested with Nextcloud as a rclone remote, via config file and just envvars, following the paradigm of some other providers to implement both. Tested on windows only.

…ks as to why files are processed mutiple times

Copilot

Pull request overview

This pull request adds Rclone as a new provider for the SiteSync addon, enabling file synchronization through Rclone remotes. The implementation follows the existing provider pattern and supports both configuration file-based and environment variable-based authentication.

Changes:

Added RClone provider settings model with multiplatform path support and both config file and environment variable configuration options
Implemented RCloneHandler client with full file operations (upload, download, delete, list, create folder) via subprocess calls to rclone
Registered the new provider in the factory with a batch limit of 20

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
server/settings/settings.py	Imports RClone settings model and registers "rclone" as a provider option
server/settings/providers/rclone.py	Defines RCloneSubmodel with configuration fields for rclone executable, config path, remote settings, and authentication
client/ayon_sitesync/providers/rclone.py	Implements RCloneHandler class with all required provider methods using rclone subprocess calls
client/ayon_sitesync/providers/lib.py	Registers RCloneHandler in the provider factory
client/ayon_sitesync/providers/resources/rclone.png	Adds Rclone provider icon

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-29T19:08:06Z

client/ayon_sitesync/providers/rclone.py

+        cmd = [self.rclone_path, "--config",
+               self._config_path]


When a config path is provided, the code constructs a command including "--config" with an empty string (line 208-209). This could cause issues if self._config_path is an empty string instead of being truly absent. The condition at line 221 checks 'if not self._config_path', which would be True for empty strings, but by then the command has already been built incorrectly. Consider restructuring the logic to handle config path construction only when it's non-empty, or ensure empty strings are treated consistently.

Copilot · 2026-01-29T19:08:07Z

server/settings/providers/rclone.py

+                      description="Egg webdav, Use this if you do not have a rclone.conf")
+
+    url: str = Field("", title="Remote Url",
+                     description="Use this if you do not have a rclone.conf")
+
+    vendor: str = Field("", title="Vendor",
+                        description="Egg nextcloud, check rclone docs for the right name, Use this if you do not have a rclone.conf")


The description contains a typo. "Egg" should be "e.g." (the Latin abbreviation for "exempli gratia" meaning "for example").

Suggested change

description="Egg webdav, Use this if you do not have a rclone.conf")

url: str = Field("", title="Remote Url",

description="Use this if you do not have a rclone.conf")

vendor: str = Field("", title="Vendor",

description="Egg nextcloud, check rclone docs for the right name, Use this if you do not have a rclone.conf")

description="e.g. webdav, Use this if you do not have a rclone.conf")

url: str = Field("", title="Remote Url",

description="Use this if you do not have a rclone.conf")

vendor: str = Field("", title="Vendor",

description="e.g. nextcloud, check rclone docs for the right name, Use this if you do not have a rclone.conf")

client/ayon_sitesync/providers/rclone.py

Copilot · 2026-01-29T19:08:07Z

client/ayon_sitesync/providers/rclone.py

+            env[env_url] = self.url
+            env_user = f"RCLONE_CONFIG_{self.remote_name.upper()}_USER"
+            env[env_user] = self.user
+            env["RCLONE_CONFIG"] = "NUL"


Using "NUL" as the RCLONE_CONFIG value is Windows-specific. On Unix-based systems (Linux/macOS), the null device is "/dev/null". This should use os.devnull which provides a cross-platform null device path, or use conditional logic based on the platform to ensure compatibility across Windows, Linux, and macOS environments.

Suggested change

env["RCLONE_CONFIG"] = "NUL"

env["RCLONE_CONFIG"] = os.devnull

Copilot · 2026-01-29T19:08:08Z

client/ayon_sitesync/providers/rclone.py

+        return subprocess.check_output(cmd).decode().strip()
+
+    @staticmethod
+    def _parse_rclone_json(output: dict):


The type hint for the 'output' parameter is incorrect. The parameter is typed as 'dict' but the function actually handles string and bytes types (as shown by the isinstance checks on lines 303-304). The type hint should be 'str | bytes' or 'Union[str, bytes]' to accurately reflect the actual parameter types.

Suggested change

def _parse_rclone_json(output: dict):

def _parse_rclone_json(output: str | bytes):

client/ayon_sitesync/providers/rclone.py

Copilot · 2026-01-29T19:08:10Z

server/settings/providers/rclone.py

@@ -0,0 +1,66 @@
+from pydantic import Field
+from ayon_server.settings import BaseSettingsModel, secrets_enum


The import secrets_enum is not used anywhere in this file. This should be removed to avoid unused imports.

Suggested change

from ayon_server.settings import BaseSettingsModel, secrets_enum

from ayon_server.settings import BaseSettingsModel

Copilot · 2026-01-29T19:08:10Z

server/settings/providers/rclone.py

+    type: str = Field("", title="Remote Type",
+                      description="Egg webdav, Use this if you do not have a rclone.conf")
+
+    url: str = Field("", title="Remote Url",
+                     description="Use this if you do not have a rclone.conf")
+
+    vendor: str = Field("", title="Vendor",
+                        description="Egg nextcloud, check rclone docs for the right name, Use this if you do not have a rclone.conf")
+
+    user: str = Field("", title="User",
+                      description="Use this if you do not have a rclone.conf")


The fields 'type', 'url', 'vendor', and 'user' are missing the 'scope' parameter. Looking at other fields in this model and similar settings in other providers (e.g., sftp.py lines 37-41), these fields should have scope defined. Consider adding scope=["studio", "project", "site"] to maintain consistency with other provider-specific configuration fields.

hmm I would make them consistent... but I am not sure which ones to take as a baseline... just studio project or all three

Copilot · 2026-01-29T19:08:11Z

client/ayon_sitesync/providers/rclone.py

+        self.rclone_path = self.presets.get("rclone_executable_path", {}).get(
+            platform.system().lower(), "rclone")
+
+        self._root = self.presets.get("root", "").strip("/")


The _root instance variable is set at line 19 but never used anywhere in the class. The get_roots_config method uses self.presets.get('root', '/') instead. Consider removing this unused variable or using it in get_roots_config if it was intended to be used.

Suggested change

self._root = self.presets.get("root", "").strip("/")

Copilot · 2026-01-29T19:08:11Z

client/ayon_sitesync/providers/rclone.py

+    def _parse_time(self, rclone_time_str):
+        """Parse rclone's ISO8601 time string to timestamp."""
+        dt = datetime.fromisoformat(rclone_time_str.replace("Z", "+00:00"))
+        return dt.timestamp()
+


The _parse_time method is defined but never used anywhere in the class. If this method is not needed for future functionality, it should be removed to avoid maintaining unused code. If it's intended for future use, consider adding a comment explaining its purpose.

Suggested change

def _parse_time(self, rclone_time_str):

"""Parse rclone's ISO8601 time string to timestamp."""

dt = datetime.fromisoformat(rclone_time_str.replace("Z", "+00:00"))

return dt.timestamp()

…hints, better handle parsed rclone json output, reduce exceptions from non existing folders

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

nebukadhezer · 2026-02-02T20:05:15Z

@kalisp I am not sure about the scope settings... especially with the PW not being obscured and then everyone can see it on the site settings so I set them to project and studio only... would convert the PR now away from draft.

nebukadhezer and others added 16 commits January 19, 2026 10:40

adding nextcloud provider with WebDAV support

075415c

start adding rclone provider

78727bb

wip rclone provider:

36b0cf9

wip getting started with testing get tree

2643915

wip getting started with testing get tree

82a500c

first test with addon rclone provider configuration

ee2edf9

make settings work

94ef48c

start testing locally with rclone provider support

7baa0f9

puah wip commit this somewhat works but leaves a lot of questions mar…

713a3ed

…ks as to why files are processed mutiple times

clean up for testing add webbased settings if conf file is not feasable

0ee7707

forgot parenthesis in url

9f35b89

fix remote name fallback

7366bae

remove secrets enum wont work

ce6156e

clean up logging

3b924af

clean up logging

323fbd7

clean up settings

2c5705e

BigRoy requested review from Copilot and kalisp January 29, 2026 19:02

BigRoy assigned kalisp Jan 29, 2026

BigRoy added type: enhancement Improvement of existing functionality or minor addition community Issues and PRs coming from the community members labels Jan 29, 2026

Copilot started reviewing on behalf of BigRoy January 29, 2026 19:02 View session

Copilot AI reviewed Jan 29, 2026

View reviewed changes

nebukadhezer and others added 6 commits February 2, 2026 20:48

apply fixes to rclone provider, improve config handling and add type …

eed2f0e

…hints, better handle parsed rclone json output, reduce exceptions from non existing folders

Update client/ayon_sitesync/providers/rclone.py

999a88a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

23eb2c3

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

5254ff3

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestion from @Copilot

cbbb8d7

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

set all scopes to studio, project,

2ab563e

nebukadhezer marked this pull request as ready for review February 3, 2026 07:07

nebukadhezer added 2 commits February 3, 2026 11:12

add creation flag to suppress windows popping up

369b071

oversaw an info log level for run command execution

35541dd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rclone #70

Rclone #70

nebukadhezer commented Jan 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

nebukadhezer Feb 2, 2026

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

Copilot AI Jan 29, 2026

Uh oh!

nebukadhezer commented Feb 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	env["RCLONE_CONFIG"] = "NUL"
	env["RCLONE_CONFIG"] = os.devnull

	def _parse_rclone_json(output: dict):
	def _parse_rclone_json(output: str \| bytes):

		@@ -0,0 +1,66 @@
		from pydantic import Field
		from ayon_server.settings import BaseSettingsModel, secrets_enum

Rclone #70

Are you sure you want to change the base?

Rclone #70

Conversation

nebukadhezer commented Jan 29, 2026

Changelog Description

Additional review information

Testing notes:

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

nebukadhezer Feb 2, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

Uh oh!

nebukadhezer commented Feb 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants