Skip to content

Bazel rules py #3542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
GrayHoang wants to merge 20 commits into
base: master
Choose a base branch
from
Open

Bazel rules py #3542

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
33c20b5
Add files via upload
GrayHoang Aug 2, 2025
d251d7f
Rename image.png to sml-error-msg.png
GrayHoang Aug 2, 2025
7e66688
Merge branch 'master' of github.com:UBC-Thunderbots/Software
GrayHoang Dec 4, 2025
75f220e
remove image
GrayHoang Dec 23, 2025
3911d26
make all thunderscope dependencies hermetic, use rules_py
GrayHoang Dec 23, 2025
98a4131
merge conflict
GrayHoang Dec 23, 2025
bff3e51
remove debugging tools
GrayHoang Dec 23, 2025
45fd8cd
clean unneeded dependencies
GrayHoang Dec 23, 2025
4d48707
more cleaning
GrayHoang Dec 23, 2025
d462dec
clean setup_software
GrayHoang Dec 23, 2025
0d4b3b2
[pre-commit.ci lite] apply automatic fixes
pre-commit-ci-lite[bot] Dec 23, 2025
d8ba7be
fix some bash scripting issue and add some debug
GrayHoang Dec 26, 2025
496a037
retrigger checks
GrayHoang Jan 3, 2026
3c269af
[pre-commit.ci lite] apply automatic fixes
pre-commit-ci-lite[bot] Jan 3, 2026
9fce33a
remove libraries and bugfix
GrayHoang Jan 10, 2026
ee7630e
Merge branch 'master' into bazel_rules_py
GrayHoang Feb 3, 2026
911dc36
[pre-commit.ci lite] apply automatic fixes
pre-commit-ci-lite[bot] Feb 3, 2026
ceb6f6b
merge master
GrayHoang Feb 8, 2026
598a193
Merge branch 'bazel_rules_py' of github.com:GrayHoang/Software into b…
GrayHoang Feb 8, 2026
83a2178
[pre-commit.ci lite] apply automatic fixes
pre-commit-ci-lite[bot] Feb 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions environment_setup/setup_software.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ sudo apt-get install -y software-properties-common # required for add-apt-reposi
sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt-get update
sudo apt autoremove

# Detect if running under WSL
# See https://github.com/microsoft/WSL/issues/4071#issuecomment-1221588337
Expand Down Expand Up @@ -92,10 +93,7 @@ fi

virtualenv_opt_args=""
if [[ $(lsb_release -rs) == "24.04" ]]; then
host_software_packages+=(python3-pyqt6)
host_software_packages+=(pyqt6-dev-tools)
host_software_packages+=(python3-pyqt6.qtsvg)

sudo apt remove "python3-pyqt6" "pyqt6-dev-tools" "python3-pyqt6.qtsvg"
virtualenv_opt_args="--system-site-packages"
Copy link
Contributor

@Lmh-java Lmh-java Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am guessing these will contaminate the dependency tree?

Copy link
Contributor Author

@GrayHoang GrayHoang Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are installations of pyqt6 into the local computer, which we no longer need. I think I need to write an additional line to uninstall an existing version of pyqt6 as well so it doesn't contaminate the tree.

fi

Expand Down
6 changes: 1 addition & 5 deletions environment_setup/ubuntu24_requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
ansible-lint==25.8.1
pyqtgraph==0.13.7
thefuzz==0.19.0
iterfzf==0.5.0.20.0
python-Levenshtein==0.25.1
psutil==5.9.0
PyOpenGL==3.1.6
ruff==0.5.5
pyqt-toast-notification==1.3.2
md-toc==9.0.0
grpcio-tools==1.71.0
python-Levenshtein==0.25.1
typer==0.21.0
3 changes: 2 additions & 1 deletion src/MODULE.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ module(
bazel_dep(name = "googletest", version = "1.15.2")
bazel_dep(name = "platforms", version = "0.0.11")
bazel_dep(name = "pybind11_bazel", version = "2.13.6")
bazel_dep(name = "bazel_skylib", version = "1.7.1")
bazel_dep(name = "bazel_skylib", version = "1.8.1")
bazel_dep(name = "rules_pkg", version = "1.1.0")
bazel_dep(name = "rules_foreign_cc", version = "0.14.0")
bazel_dep(name = "rules_python", version = "1.4.1")
Expand All @@ -22,6 +22,7 @@ bazel_dep(name = "rules_proto", version = "7.1.0")
bazel_dep(name = "yaml-cpp", version = "0.8.0")
bazel_dep(name = "buildifier_prebuilt", version = "8.0.3")
bazel_dep(name = "pybind11_protobuf", version = "0.0.0-20250210-f02a2b7")
bazel_dep(name = "aspect_rules_py", version = "1.6.6")
bazel_dep(name = "bazel_lib", version = "3.1.0")

##############################################
Expand Down
162 changes: 162 additions & 0 deletions src/MODULE.bazel.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions src/software/thunderscope/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
load("@aspect_rules_py//py:defs.bzl", "py_binary", "py_library")
Copy link
Member

@williamckha williamckha Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you be loading py_binary and py_library from aspect_rules_py in every BUILD file? Some BUILD files don't load them in, so I assume they're using the rules_python versions.

Copy link
Contributor Author

@GrayHoang GrayHoang Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think for consistency you are probably correct. I would also assume that if I don't explicitly load it, it uses the rules_python version. I just knew that specifically this was broken, and otherwise I didn't want to touch the other parts that seemed to work fine.

Copy link
Contributor Author

@GrayHoang GrayHoang Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a matter of fact, I believe that once this rule has been loaded, it will use aspect_build in every single BUILD file. This is shown by other BUILD files not compiling when the necessary libraries aren't explicitly listed as requirements, even in files where py_binary and py_library are not loaded. So it is consistent in usage.

load("@rules_python//python:pip.bzl", "compile_pip_requirements")
load("@thunderscope_deps//:requirements.bzl", "requirement")

Expand All @@ -13,6 +14,11 @@ compile_pip_requirements(
py_binary(
name = "thunderscope_main",
srcs = ["thunderscope_main.py"],
env = {
"LD_LIBRARY_PATH": "../.thunderscope_main.venv/lib/python3.12/site-packages/PyQt6/Qt6/lib",
"QTWEBENGINEPROCESS_PATH": "../.thunderscope_main.venv/lib/python3.12/site-packages/PyQt6/Qt6/libexec/QtWebEngineProcess",
},
Comment on lines +17 to +20
Copy link
Member

@williamckha williamckha Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to consider setting these environment variables in Python at runtime so you don't have to copy this for every py_binary and py_test. You'd have to do this before importing PyQt, so maybe at the top of thunderscope.py would be a good place. Something like:

# PyQt6 bundles Qt as native shared libraries and helper binaries that are not
# discoverable when running under Bazel's sandboxed environment. In particular,
# the dynamic linker cannot find Qt's .so files and QtWebEngine cannot locate its
# QtWebEngineProcess helper unless explicitly told where they live. 

# Since the aspect_rules_py virtual env lives in the runfiles tree and its
# location is only known at runtime, we derive these paths from $VIRTUAL_ENV 
# here and set the required env vars before importing PyQt6.

qt_path = (
    Path(os.environ["VIRTUAL_ENV"])
    / "lib"
    / f"python{sys.version_info.major}.{sys.version_info.minor}"
    / "site-packages"
    / "PyQt6"
    / "Qt6"
)

os.environ["LD_LIBRARY_PATH"] = str(qt_path / "lib")
os.environ["QTWEBENGINEPROCESS_PATH"] = str(qt_path / "libexec" / "QtWebEngineProcess")

Copy link
Contributor Author

@GrayHoang GrayHoang Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to work fine as is, without needing to copy paste this to every py_binary and py_test. I think perhaps under the hood aspect_rules is doing the same thing?

package_collisions = "warning",
deps = [
Copy link
Contributor

@Lmh-java Lmh-java Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is managing this virtual environment? Is it bazel?

Copy link
Contributor Author

@GrayHoang GrayHoang Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I believe it is Bazel by way of the aspect_rules_py reworking. Basically, the new aspect_rules_py pylibrary/ pybinary rules build the dependencies in a venv, instead of doing whatever Bazel did before, which was less rigorous and safe.

":config",
":constants",
Expand Down Expand Up @@ -49,6 +55,7 @@ py_library(
"//software/thunderscope/binary_context_managers:game_controller",
"//software/thunderscope/binary_context_managers:simulator",
"//software/thunderscope/binary_context_managers:tigers_autoref",
requirement("pyqt6"),
requirement("numpy"),
requirement("pyqtdarktheme-fork"),
requirement("pyqtgraph"),
Expand Down Expand Up @@ -152,6 +159,7 @@ py_library(
srcs = ["constants.py"],
deps = [
requirement("protobuf"),
requirement("pyopengl"),
],
)

Expand Down
5 changes: 5 additions & 0 deletions src/software/thunderscope/binary_context_managers/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ py_library(
"//software/networking:ssl_proto_communication",
"//software/thunderscope:util",
"//software/thunderscope/common:thread_safe_circular_buffer",
requirement("psutil"),
],
)

Expand All @@ -35,6 +36,7 @@ py_library(
],
data = [
"//software:unix_full_system",
requirement("psutil"),
],
)

Expand All @@ -59,4 +61,7 @@ py_library(
"runtime_loader.py",
"runtime_manager.py",
],
deps = [
requirement("requests"),
],
)
2 changes: 2 additions & 0 deletions src/software/thunderscope/common/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ py_library(
deps = [
":common_widgets",
requirement("netifaces"),
requirement("thefuzz"),
requirement("levenshtein"),
],
)

Expand Down
1 change: 1 addition & 0 deletions src/software/thunderscope/play/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
load("@aspect_rules_py//py:defs.bzl", "py_binary", "py_library")
load("@thunderscope_deps//:requirements.bzl", "requirement")

package(default_visibility = ["//visibility:public"])
Expand Down
11 changes: 10 additions & 1 deletion src/software/thunderscope/requirements.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ numpy==1.26.4
protobuf==6.31.1
pyqtgraph==0.13.7
pyqtdarktheme-fork==2.3.2
PyQt6-Qt6==6.8.1
PyQt6-Qt6==6.10.0
pyqt-toast-notification==1.3.2
qtawesome==1.4.0
PyQt6==6.10.0
PyOpenGL==3.1.6
psutil==5.9.0
thefuzz==0.19.0
python-Levenshtein==0.25.1
<<<<<<< HEAD
requests==2.32.5
=======
>>>>>>> 911dc36b8a1854a4f78e3e6cd46f7e12ec094508
Loading
Loading