Deferred Loader Error When Reverting to State With No Polymorphic Children

[gavmor]

[Context: this is an issue written for the SQLAlchemy-History repository to try and address some frustrations we have encountered.]

SQLAlchemy error: Deferred loader for attribute 'radius' failed to populate correctly

Lost a lot of time to what, ultimately, felt like a pretty silly oversight due to the specificity of this SQLAlchemy message coming up when reverting relations to a transaction before which they exist.

Let's say we have a class composed of polymorphic components:

class Car(Base):
    __tablename__ = "car"
    __versioned__ = {}

    id: Mapped[int] = mapped_column(primary_key=True)
    
    parts: Mapped[List["Part"]] = relationship(
        back_populates="car", 
        cascade="all, delete-orphan"
    )


class Part(Base):
    __tablename__ = "part"
    __versioned__ = {}

    id: Mapped[int] = mapped_column(primary_key=True)
    car_id: Mapped[Optional[int]] = mapped_column(ForeignKey("car.id"))    
    car: Mapped["Car"] = relationship(back_populates="parts")

    type: Mapped[str] = mapped_column(String(50))
    __mapper_args__ = {
        "polymorphic_identity": "part",
        "polymorphic_on": "type",
    }


class Tire(Part):
    __tablename__ = "tire"
    __versioned__ = {}

    id: Mapped[int] = mapped_column(ForeignKey("part.id"), primary_key=True)
    
    radius: Mapped[int]
    width: Mapped[int]

    __mapper_args__ = {
        "polymorphic_identity": "tire",
    }

# # For example, another polymorphic child:
# class Engine(Part):
#     __tablename__ = "engine"
#     __versioned__ = {}

#     id: Mapped[int] = mapped_column(ForeignKey("part.id"), primary_key=True)
#     horsepower: Mapped[int]

#     __mapper_args__ = {
#         "polymorphic_identity": "engine",
#     }
    

When, in our application, we try to revert a Car instance to a previous transaction that predates the addition of any Tire or Engine parts, we encounter the error:

parts = car.revert(relations=["parts"]).parts
tire = parts[0]
tire.radius # Throws the error

Note: We can bring this about sooner by specifying lazy="selectin" in the relationship definition. This forces SQLAlchemy to use a different loading strategy that surfaces the issue more quickly.

To reiterate, this happens because we have no parts at that transaction point.

Note: Is there a simpler use case that produces the same error? (eg. a simple one-to-one relation) Or is this inherently tied to polymorphic relationships?

How might we update SQLAlchemy-History to better handle this scenario? Ideally, reverting to a transaction where no polymorphic children exist should not lead to an error when accessing attributes of those children. Instead, it should gracefully handle the absence of those children, perhaps by returning None or an empty collection, depending on the context. At the very least, the error message could be more descriptive to indicate that the issue arises from attempting to access attributes of non-existent polymorphic children.

Or, how can we adjust our usage of SQLAlchemy-History to avoid or pre-empt this error?

[AbdealiLoKo]

Thanks for the report @gavmor
So far we have no testcases for polymorphic_on as we don't use that in our codebase at Corridor.

While we don't mind adding and maintaining it, I am not sure when we'll try to add that support.
If that is beneficial for you - it would be great if you could contribute support for that with tests!
Very open to PRs