[qt5-webengine] Latest mac, win build support. (fix #47010)

[karino2]

Two major changes for this PR

In this PR, there are two major changes

• python3 update
• modern C++ compiler update

These should have been included in original qt5 repository, but original qt5 is not maintained anymore.
So there is no way other than include these patches in vcpkg, which might be different from vcpkg repogitry poiicy.

But qt6 is not stable for some area (for example, tablet driver related part), and still qt5 is viable options (especially for sustained phase app).

Also, old qt5-webengine builde prerequisite is based on original qt5 build, which is a little different from vcpkg dependent handling.
And these days original prerequisite setup is getting harder and harder (for example, python2 setup, icu version, etc.).

So I make this port more vcpkg manner like Qt6's qtwebengine (like x_vcpkg_get_python_packages).

These PR should have been make separate ones, but current port is already not buildable to windows and mac (see #47010).
So I have to apply all changes to confirm whether each ones is correct.

That's why this PR is bigger than recommendations (though still I try to keep minimum).

Is this PR is acceptable for vcpkg?

PR contents

Build fix for latest macOS (apple silicon) and Windows.
But it requires a lot of following changes:

• modern mac c library fix
• modern clang cpp fix
• support apple silicon
• use python3 instead of python2
• add dependency for perl
• modern windows update (icu requires C++17)

Now, It buildable to MacBookAire M4 and Windows 11.

Basically, these fixes are partial patch from Qt6's qtwebengine.

Fixes #47010

• Changes comply with the maintainer guide.
• SHA512s are updated for each updated download.
• The "supports" clause reflects platforms that may be fixed by this new version.
• Any fixed CI baseline entries are removed from that file.
• Any patches that are no longer applied are deleted from the port's directory.
• The version database is fixed by rerunning ./vcpkg x-add-version --all and committing the result.
• Only one version is added to each modified port's versions file.

[karino2]

@microsoft-github-policy-service agree

[karino2]

Also, I want to know current policy of CI for this ports.
In my understanding, this port is opt out. What should I do for CI checklist item?

[karino2]

I confirm this webengine can load basic page in mac and windows.
I need to copy QtWebEngineProcess.exe from vcpkg/installed/x64-windows/tools/qt5 and some similar works in windows (I guess this is the official way to use qt5-webengine, though there is no documentation for how to use this port).

[dg0yt]

In my understanding, this port is opt out. What should I do for CI checklist item?

Skips and expected cascades/fails are in scripts/ci.baseline.txt and scripts/ci.features.baseline.txt. Related: #45802.

[karino2]

Thank you.

ci.baseline.txt

In the ci.baseline.txt, qt5-webengine related entries are only below.

# Skipped to avoid exceeding the 48 hour time limit in CI
# May also need an older ninja version
qt5-webengine:x64-windows-release=skip
qt5-webengine:x64-windows=skip
qt5-webengine:x86-windows=skip

In the header comment, CI also try arm64-osx. I'm sure currently build for arm64-osx must be fail without this PR, but there is no entry.
Anyway, this PR does not need to add those entry.

ci.feature.baseline.txt

Following line is the related entry

qt5-webengine(!(arm & windows) & !uwp) = fail # Missing prerequisites for CI success

After this PR, osx becomes buildable. So I should change to following, right?

qt5-webengine(!(arm & windows) & !uwp & !osx) = fail # Missing prerequisites for CI success

[karino2]

Now all necessary work is done, make PR ready for reaview.

[SchaichAlonso]

From the PR description:

Build fix for latest macOS (apple silicon) and Windows. But it requires a lot of following changes:

• modern windows update (icu requires C++17)

According to vcpkg.json:24, icu shouldn't be an issue on windows because it's (hopefully) not used there.

If icu causes any compatibiltiy issues on windows it indicates that we do use icu on windows, and accordingly should track it as a dependency.

[SchaichAlonso]

Reviewing portfile.cmake:114, it seems qmake is asked to use the operating system's icu (and webp), which is looked up in the installation prefix, i.e. where vcpkg's one resides, on non-windows systems, and with the PR applied, on non-windows systems that aren't macos either.

As the icu sourcecode is not explicitly deleted from the webengine source distribution, on windows (and macos), qmake will build the bundled icu and use that one instead of vcpkg's icu, which is troublesome if the webengine consumer project has additional dependencies that yield into vcpkg's icu also being passed to the linker. We probably should ask vcpkg to provision icu and webp independend of the target operating systems, and have qmake use the vcpkg provisioned (system) libraries rather then the one it bundles in it's source distribution, but that might be content for a new issue.

[karino2]

In my understanding, in windows environment qt5-webengine use OS icu, which becomes C++17 inevitable (nested namespace) for recent update. This icu update cause compilation failure which include src\3rdparty\chromium\third_party/blink/public/web/web_settings.h.

For mac, building with vcpkg icu and webp cause following errors.

ERROR: Feature 'webengine-system-icu' was enabled, but the pre-condition 'libs.webengine-icu' failed.

ERROR: Feature 'webengine-system-libwebp' was enabled, but the pre-condition 'config.unix && libs.webengine-webp' failed.

I do not yet fully understand what's the real problems of these two, but when I check qtwebengine (qt6's) portfile.cmake, icu handling is different for macOS,

https://github.com/microsoft/vcpkg/blob/master/ports/qtwebengine/portfile.cmake#L93

so I handle macOS differently for these two.
May be I should investigate more for this icu and webp for osx (or do you know the cause and correct fixes for these two ERRORs?).

[BillyONeal]

Tagged vcpkg-team-review on the grounds that the patches are huge.

These should have been included in original qt5 repository, but original qt5 is not maintained anymore. So there is no way other than include these patches in vcpkg, which might be different from vcpkg repogitry poiicy.

This is the main crux of the issue. We are not upstream and do not have the capacity to review / understand substantial patches.

But qt has always been one of the exceptions due to being Important, so going to bring this up with the team. Most everyone is out for the holidays right now so I don't expect clarity until January 5 or later.

[karino2]

OK, thank you for the special treatment. I wait until finishing holiday seasons.
Have a nice holidays!

[dg0yt]

I do not yet fully understand what's the real problems of these two, but when I check qtwebengine (qt6's) portfile.cmake, icu handling is different for macOS,

Actually it is Windows which is handled differently:
Chromium (gn) assumes there is no such thing as pkgconfig on Windows. Without pkgconfig, they are unable to handle "system" libraries (i.e. externally provided dependencies like icu) in particular for static library linkage.
Qt5 (qmake) and Qt6 (cmake) forward this restriction.
(In vcpkg, pkgconfig on Windows works quite well, except for multiconfig.)

BTW the qt6 port doesn't build webengine for static library linkage, only the PDF module.
It is still full of vendored depedencies.

[dg0yt]

And in fact, icu is different again for the Apple systems in qtwebengine. It has been months since I added that. IIRC the portfile only mirrors cmake/gn in this regard.

[karino2]

We need to understand chromium build system and Apple circumustances and qt6 build system and reflect them for qt5 build system...

BTW, in Windows, we need to copy resources and translations folder from vcpkg/installed/x64-windows/share/qt5 to load any page or it will crash with chromium-pdf extension load failure.
I guess these kinds of difficulty (not enough for just call vcpkg install and need manual deployment work) is root from those vendored depedencies.

Anywa, there are many things we should handle differently and some are already done on qtwebengine port.
For this PR, the minimum line would be the situation is not getting worse relative to pre-PR qt5-webengine port.

[karino2]

In qtwebengine, is this mac special handling is necessary for build to pass?

[dg0yt]

In qtwebengine, is this mac special handling is necessary for build to pass?

I already tried to rectify my first comment: I believe I added the condition because the dependencies were simply unused in Qt6's cmake wrapping of chromium/gn.
But this might not apply to Qt5's qmake wrapping of chromium/gn.

[karino2]

OK, thank you.

• In qt5-webengine, the pre-condition fail errors are definitely occure in osx, so I need some fixes.
• I confirm just excluding webengine-system-icu and webengine-system-libwebp erase these ERROR and build pass.
• And qtwebengine, if my understaneding is correct, behaves similarly.

That's all I know and I'm not sure whether this fix is correct or fix things with wrong ways.
If somebody knows better idea for fixing these two erros, I can try it.

Bottom line: this exclusion only affects osx and we can't build pre-PR port in osx anyway.