Skip to content

Conversation

@ngoldbaum
Copy link
Contributor

@ngoldbaum ngoldbaum commented Nov 13, 2025

Closes #74.

The strategy here is to lock access to the internal C radix tree struct using critical sections. See https://py-free-threading.github.io/porting-extensions/ for more about this topic. The critical section macros include braces, so I needed to do some minor refactoring in a few places to ensure the control flow exits the critical section in error paths.

@ngoldbaum ngoldbaum marked this pull request as ready for review November 14, 2025 22:30
@ngoldbaum
Copy link
Contributor Author

ngoldbaum commented Nov 14, 2025

Ping @mjschultz, this is ready for review now. Also ping @kumaraditya303 who has been doing a lot of work and code review for similar code in the standard library.

The test I added is imperfect: it doesn't cover all the reader APIs that I modified. Should I make the test more thorough or is this good enough?

I verified that running the test on a TSan-instrumented build of Python using a TSan-instrumented build of py-radix passes with no failures. If I run the same test on current main on the free-threaded build with PYTHON_GIL=0 (e.g. force-disabling the GIL), then I get immediate TSan error reports. See https://py-free-threading.github.io/thread_sanitizer/ for more about setting up thread sanitizer. We could even set up a test run under 3.14t using a TSan build.

}
Py_RETURN_NONE;
Py_END_CRITICAL_SECTION();
Py_XINCREF(ret);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm less familiar with the python c api than i once was, but i assume incrementing the ref count to Py_None is either a no-op or (since it's None) so meaningless that it's irrelevant.

Copy link
Contributor Author

@ngoldbaum ngoldbaum Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true, it was only necessary to do that on CPython 3.12 and older. Here's the definition of Py_RETURN_NONE in the CPython headers:

/* Macro for returning Py_None from a function.
   * Only treat Py_None as immortal in the limited C API 3.12 and newer. */
  #if defined(Py_LIMITED_API) && Py_LIMITED_API+0 < 0x030c0000
  #  define Py_RETURN_NONE return Py_NewRef(Py_None)
  #else
  #  define Py_RETURN_NONE return Py_None
  #endif

(Py_NewRef is a variant of Py_INCREF that returns the object).

There's also an optimization in the implementation of Py_INCREF itself that skips immortal objects.

I restructured the code in this function to always incref (except for NULL return values on errors) because the critical section macros include braces, so it's not syntactically possible to ensure the critical section is closed in an early return block. Instead, all the code paths need to return after the critical section is closed.

IMO it's not worth worrying about optimizing things like avoiding increfs on immortal objects. We should leave it up to CPython to optimize stuff like that and instead aim to make the code as clear and correct as possible.

@ngoldbaum
Copy link
Contributor Author

Thanks @kumaraditya303! I think I caught all the places that needed adjusting.

@ngoldbaum
Copy link
Contributor Author

@mjschultz let me know if you have any further questions or concerns. Also happy to point to further resources.

@ngoldbaum
Copy link
Contributor Author

@mjschultz another ping here - it'd be really nice to have free-threaded support merged and I think this is ready.

@mjschultz
Copy link
Owner

Can you bump the version identifier in https://github.com/mjschultz/py-radix/blob/main/setup.py#L13 to v1.1.0 in this PR please?

@ngoldbaum
Copy link
Contributor Author

to v1.1.0 in this PR please?

Sure, done.

@mjschultz mjschultz merged commit b5e4e93 into mjschultz:main Dec 4, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for the free-threaded build

3 participants