Split ayon-python-api API Reference documentation per concept

[rdelillo]

Is there an existing issue for this?

• I have searched the existing issues.

Please describe the feature you have in mind and explain what the current shortcomings are?

The ayon-python-api public API is currently very flat by design.
(Everything get imported into the global __init__.py and exposed via __all__).

This might not be ideal to work with and certainly makes the API Reference very hard to browse. This could refrain from bigger adoption.

How would you imagine the implementation of the feature?

• Replace the automated_api.py script by a dynamic function creation at __init__.py and uses decorator to handle connection.
• Expose public API via modules based on features/classes
• Set deprecation warning with redirect to ensure backward compatibility

Are there any labels you wish to add?

• I have added the relevant labels to the enhancement request.

Describe alternatives you've considered:

No response

Additional context:

No response

[iLLiCiTiT]

Do you have in mind idea how it should be split?

If it's related to documentation, wouldn't be better to create mapping for documentation rather than changing the code itself? I'm not sure if it makes sense to split it.

Because from "docs reader" point of view I 100% agree, from "coder who use it" point of view I don't agree that much.

[rdelillo]

Do you have in mind idea how it should be split?

If it's related to documentation, wouldn't be better to create mapping for documentation rather than changing the code itself? I'm not sure if it makes sense to split it.

Because from "docs reader" point of view I 100% agree, from "coder who use it" point of view I don't agree that much.

That's a good question, for now I can think of 2 options.
We could either group by related concept

* representation
    * get_representations
    * get_representation_by_id
    * get_representation_by_name
    * get_representations_hierarchy
    * get_representation_hierarchy
    * [...]

* tasks
    * create_task
    * update_task
    * delete_task
    * [...]

* product
    * get_products
    * get_product_by_id
    * get_product_by_name
    * get_product_types
    * get_project_product_types
    * get_product_type_names
    * [...]

Or by action scope (read, write, update) similar to Pixar Tractor API

* get
    * get_representations
    * get_representation_by_id
    * get_representation_by_name
    * get_representations_hierarchy
    * get_products
    * get_product_by_id
    * get_product_by_name
    * version_is_latest
    * [...]

* create
    * create_product
    * create_representation
    * create_version
    * create_task
    * [...]

* update
    * update_product
    * update_representation
    * update_task
    * update_version
    * [...]
  
 * delete
     * delete_product
     * delete_representation
     * delete_task
     * delete_version
     * [...]

I initially thought it could me more convenient to split that in multiple python modules, but I do not have any other reasons that personal preference here. (I'd have investigated having the grouping done through the automated_api logic with some kind of decorator for connection handling).
But maybe that's too complex/over-engineer...

But as you pointed out, this can totally be restricted to be a documentation thing only. In which case, I feel we would have to create intermediary .rst file for the grouping to specify each member per group. Only caveat I can foresee if that we wouldn't "simply" rely on the sphinx auto-doc mechanism anymore but have to maintain those lists on each function add/remove.

Please let me know if I should update this ticket to be a documentation-thing only.

[iLLiCiTiT]

I'm not a god, nor demi-god, so I can be convinced if I'm the only one who thinks it should not be split code-wise.
EDITED NOTE: We can move the code to related files, but at the end I would still like to be able to do just from ayon_api import get_project, get_folders, I usually want to import functions for multiple entities, and I'm lazy, sorry...

I fully agree it should be done for documentation, by "concept".
BTW Even if we would split it, we would still need to make some categorization and order of the functions. For example simplest functions like get, post, put, patch, delete should live next to each other in documentation, and there are much more functions that are semi-related and are not alphabetically correct.

Another question will be how to handle ServerAPI (which is the "big boss" implementing the code for those public functions).

[rdelillo]

Well maybe we can limit the scope of this ticket to split the documentation per concept then (and sub-organise/order them underneath) ? That's something already good enough for now, which does not introduce backward-compatibility concerns.

For the ServerAPI, we could:

• leave it as one big chunk and assume this is more advanced stuff so bear with us, this is how it is :)
• split it the same way in multiple smaller .rst files I guess
• assume this is private stuff and not make it not part of the API documentation - I'm not sure about this one, lack context on client usage

Also I feel too new to make the calls here I don't want to be the new guy who wants to change everything for the sake of changing hence my fear to overstep.

[iLLiCiTiT]

assume this is private stuff and not make it not part of the API documentation - I'm not sure about this one, lack context on client usage

It's not private, from the beginning the focus was that we have to be able to create multiple different connections during single process. Whatever the reason is, different site name, different user, or different server, so ServerAPI is actually very important in that sense. Content of _api.py is just a singleton wrapper for it, that simplifies daily basic pipeline and services work.

Also I feel too new to make the calls here I don't want to be the new guys who wants to change everything for the sake of changing hence my fear to overstep.

We never discussed it and nobody pointed it out, so I think you've raised a very valuable question.

Well maybe we can limit the scope of this ticket to split the documentation per concept then (and sub-organise/order them underneath)

Agree, that is something we will have to do anyways, and api code split would block this as there might be some discussion (maybe?).

[BigRoy]

EDITED NOTE: We can move the code to related files, but at the end I would still like to be able to do just from ayon_api import get_project, get_folders, I usually want to import functions for multiple entities, and I'm lazy, sorry...

I think even if we were to move things into e.g. ayon_api.tasks, etc. that would still be possible by importing it upwards. The documentation would then just show them as part of ayon_api.tasks?

I fully agree it should be done for documentation, by "concept". BTW Even if we would split it, we would still need to make some categorization and order of the functions. For example simplest functions like get, post, put, patch, delete should live next to each other in documentation, and there are much more functions that are semi-related and are not alphabetically correct.

I believe most auto-documenting tools just order by the function order in the python file. I assume, we also want to group it there in the same meaningful way so we can just structure the code in the file itself in the way we'd prefer it in the docs?

Another question will be how to handle ServerAPI (which is the "big boss" implementing the code for those public functions).

This is my biggest issue with AYON API actually - because all calls to ayon_api methods are actually methods on the server api or whatever all type hints and code autocompletes don't actually work as they should. Since they display autocomplete for the 'stub' server api that is in-between instead of the actual functions they call.

Maybe we should improve that by making those nicer 'stubs' (e.g. by just decorating the upstream functions and in the decorator make it also inherit the docstring from the function it's wrapping so that the functions docstring (and type hints?) somehow match. Not sure what is feasible there, but the arguments not being available in quick autocomplete (with type hints) is what I miss most.

For example this is what my IDE thinks currently:

But I suppose that's partially what this is about: #175