Fix: Add load/performance/extension test groups and align env/test ex… #22

Mulily0513 · 2025-12-24T10:18:09Z

…pectations

add bench_test and pxf_extension_test in run_tests.sh, plus matrix entries for bench and pxf_extension in CI
bump surefire heap to 4G to avoid OOM
update gpupgrade expected outputs to new PXF_HOME paths and JSON formatter error text
make ProtocolUtils/HiveBaseTest/JdbcHiveTest/OrcWriteTest/ParquetWriteTest more robust to env defaults (protocol, creds, hive JDBC URL)
keep MultiServerTest running under HDFS with a safe working directory fallback
set distribution key and INSERT pattern for performance test data load

fix #ISSUE_Number

Change logs

Describe your change clearly, including what problem is being solved or what document is being added or updated.

Contributor's checklist

Here are some reminders before you submit your pull request:

Make sure that your Pull Request has a clear title and commit message. You can take the Git commit template as a reference.
Sign the Contributor License Agreement as prompted for your first-time contribution (One-time setup).
Learn the code contribution and doc contribution guides for better collaboration.
List your communications in the GitHub Issues or Discussions (if has or needed).
Feel free to ask for the cloudberrydb team to help review and approve.

MisterRaindrop · 2025-12-25T07:07:22Z

concourse/singlecluster/conf/gphd-conf.sh

-    else
-      export JAVA_HOME=$(readlink -f /usr/bin/java | sed 's:/bin/java::')
-    fi
+    export JAVA_HOME=/usr/lib/jvm/java-8-openjdk


delete Auto-detect Java 8 path for different architectures？

If the /usr/lib/jvm/java-8-openjdk/ directory indeed contains JDK versions for multiple architectures, then the JDK should automatically select the appropriate version based on the architecture.

I mean this commit gphd-conf.sh is deleted automatically, set export JAVA_HOME=/usr/lib/jvm/java-8-openjdk. I mean it shouldn't be removed.

MisterRaindrop · 2025-12-25T07:09:04Z

automation/src/main/java/org/greenplum/pxf/automation/utils/system/ProtocolUtils.java

-        return System.getProperty(AWS_ACCESS_KEY_ID);
+        String access = System.getProperty(AWS_ACCESS_KEY_ID);
+        String result = access != null ? access : System.getenv(AWS_ACCESS_KEY_ID);
+        System.out.println("DEBUG: doInit() method called getAccess  " + result);


this is a debug print?

Yes, this is a debug print. It should be removed.

MisterRaindrop · 2025-12-25T07:16:11Z

automation/src/test/java/org/greenplum/pxf/automation/features/hive/HiveBaseTest.java

+                hdfs.getWorkingDirectory() + "/" + fileName);
        // load to hive table
-        hive.loadData(tableName, hdfsPath, false);
+        hive.loadData(tableName, hdfs.getWorkingDirectory() + "/" + fileName, false);


Sometimes HDFS metadata loading isn't that fast. Would it be better to add hdfs.waitForFile? ensure data load successfully?

ostinru · 2025-12-25T19:30:26Z

.github/workflows/pxf-ci.yml

          - s3
          - features
          - gpdb
+          - bench


bench runs for 5 hours. Not sure that we should run it on every PR.

@tuhaihe bench runs for 5 hours, we need to distinguish between functional testing and performance testing?

5 hours feels too long for regular PR validation. I don't think it’s necessary to run this on every PR.

It might make more sense to separate performance/benchmark testing from functional testing, and run the benchmark as a scheduled job (for example, weekly) instead of blocking day-to-day PRs.

So we can remove it from this PR? How about it?

I can try to further split the load tests and performance tests first and see how long each part takes. Then we can decide the best way to run them.

Since the load test takes about 20 minutes and the performance tests take over 2 hours, we could keep just the load test for regular PR validation and move the performance tests to a separate schedule.

…pectations - add bench_test and pxf_extension_test in run_tests.sh, plus matrix entries for bench and pxf_extension in CI - bump surefire heap to 4G to avoid OOM - update gpupgrade expected outputs to new PXF_HOME paths and JSON formatter error text - make ProtocolUtils/HiveBaseTest/JdbcHiveTest/OrcWriteTest/ParquetWriteTest more robust to env defaults (protocol, creds, hive JDBC URL) - keep MultiServerTest running under HDFS with a safe working directory fallback - set distribution key and INSERT pattern for performance test data load

MisterRaindrop reviewed Dec 25, 2025

View reviewed changes

ostinru reviewed Dec 25, 2025

View reviewed changes

Mulily0513 force-pushed the merge-with-upstream_hlhf branch 4 times, most recently from bc7f101 to d474dce Compare December 29, 2025 07:16

Mulily0513 force-pushed the merge-with-upstream_hlhf branch from d474dce to 17c8ad6 Compare December 29, 2025 08:53

Fix: Add load/performance/extension test groups and align env/test ex… #22

Are you sure you want to change the base?

Fix: Add load/performance/extension test groups and align env/test ex… #22

Conversation

Mulily0513 commented Dec 24, 2025

Change logs

Contributor's checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants