Release 24_0_0 fails to compile with Portage

[freijon]

I'm packaging fs2_open for Gentoo, but with the recent release 24_0_0 there are compilation errors which I can't seem to solve by myself. The relevant bit in the output is this:

In file included from /var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.cpp:10:
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:50:31: error: ‘uint16_t’ was not declared in this scope
   50 | extern tl::optional<std::pair<uint16_t, uint16_t>>Cmdline_window_res;
      |                               ^~~~~~~~
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:16:1: note: ‘uint16_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
   15 | #include <tl/optional.hpp>
  +++ |+#include <cstdint>
   16 |
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:50:41: error: ‘uint16_t’ was not declared in this scope
   50 | extern tl::optional<std::pair<uint16_t, uint16_t>>Cmdline_window_res;
      |                                         ^~~~~~~~
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:50:41: note: ‘uint16_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:50:41: error: template argument 1 is invalid
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:50:41: error: template argument 2 is invalid
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:50:49: error: template argument 1 is invalid
   50 | extern tl::optional<std::pair<uint16_t, uint16_t>>Cmdline_window_res;
      |                                                 ^~
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:111:8: error: ‘SCP_string’ does not name a type
  111 | extern SCP_string Cmdline_lang;
      |        ^~~~~~~~~~
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.cpp:564:44: error: conflicting declaration ‘tl::optional<std::pair<short unsigned int, short unsigned int> > Cmdline_window_res’
  564 | tl::optional<std::pair<uint16_t, uint16_t>>Cmdline_window_res = tl::nullopt;
      |                                            ^~~~~~~~~~~~~~~~~~
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.h:50:51: note: previous declaration as ‘int Cmdline_window_res’
   50 | extern tl::optional<std::pair<uint16_t, uint16_t>>Cmdline_window_res;
      |                                                   ^~~~~~~~~~~~~~~~~~
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.cpp: In function ‘bool SetCmdlineParams()’:
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cmdline/cmdline.cpp:1803:44: error: request for member ‘emplace’ in ‘Cmdline_window_res’, which is of non-class type ‘int’
 1803 |                         Cmdline_window_res.emplace(static_cast<uint16_t>(width), static_cast<uint16_t>(height));
      |                                            ^~~~~~~

Full log see: https://paste.gentoo.zip/SuScYBnr
I tried to build with gcc and clang, both fail at the same position.
This issue may be specific to Portage, because if I just run cmake ../ and make, there are only a few warnings but no errors. But I never had this issue with older releases, they build fine with Portage. Any hints as to where the issue could be?

[sjnewbury]

I hit this too since I also package fs2open for Gentoo! There are a couple of places where standard c++ header includes are needed, since the compilers have improved their compliance by not including certain headers by default, such as cstdint, your error above suggests such a fix. I haven't dug any deeper yet though, the missing SCP_string type also suggests a missing header, but this time an internal project header, which is odd.

[notimaginative]

This is probably fixable by adding #include "globalincs/pstypes.h" to cmdline.h. That's a rather heavy-handed solution, but if you want to handle it quick and easy, that should do it.

It's possible that simply adding these two will be enough however:

#include <cstdint>
#include "globalincs/vmallocator.h"

I'm not sure why builds work fine except in this situation but I'll do some testing and see if I can figure something out.

[freijon]

When adding the two lines you suggested, the build continues and fails at a later stage with:

/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cutscene/ffmpeg/FFMPEGDecoder.cpp:72:43: warning: 'channel_layout' is deprecated [-Wdeprecated-declarations]
   72 |         paras.channel_layout = stream->codecpar->channel_layout;
      |                                                  ^
/usr/include/libavcodec/codec_par.h:159:5: note: 'channel_layout' has been explicitly marked deprecated here
  159 |     attribute_deprecated
      |     ^
/usr/include/libavutil/attributes.h:100:49: note: expanded from macro 'attribute_deprecated'
  100 | #    define attribute_deprecated __attribute__((deprecated))
      |                                                 ^
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cutscene/ffmpeg/FFMPEGDecoder.cpp:526:23: error: no member named 'thread' in namespace 'std'
  526 |         std::unique_ptr<std::thread> subtitle_thread;
      |                         ~~~~~^
/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cutscene/ffmpeg/FFMPEGDecoder.cpp:531:16: error: no type named 'thread' in namespace 'std'
  531 |                     new std::thread([this, &subtitleDecoder]() { runSubtitleDecoder(subtitleDecoder.get()); }));
      |                         ~~~~~^
1 warning and 2 errors generated.

looks like another missing #include?

[notimaginative]

looks like another missing #include?

Yep, #include <thread> this time.

The ffmpeg deprecation warnings are on my todo list, but hopefully they'll stay warnings and not cause any errors for you in other places.

[freijon]

/var/tmp/portage/games-engines/fs2_open-24.0.0/work/fs2open.github.com-release_24_0_0/code/cfile/cfile.cpp:99:37: error: implicit instantiation of undefined template 'std::array<CFILE, 64>'
   99 | std::array<CFILE, MAX_CFILE_BLOCKS> Cfile_block_list;
      |                                     ^
/usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/tuple:2005:45: note: template is declared here
 2005 |   template<typename _Tp, size_t _Nm> struct array;
      |                                             ^
1 error generated.

is there a good way to identify all these missing inclues at once?

[notimaginative]

I've never tried this myself, but I think you can use the env variable NINJAFLAGS="-k 0". The -k option tells it to stop after a certain number of errors. The default is 1 and 0 means infinity.

I don't know much about gentoo but assuming you're just building on the terminal, export that line and then build. Otherwise you'll probably need to export it from your .bashrc (or whatever for your shell), log out, then log back in a try the build.

Then I think it will try building without stopping at errors. Though some errors might be fatal either way depending on build dependencies. But you should hopefully get a log with most, if not all, of the errors so they can be fixed at one time.

[freijon]

After a lot of trial and error I found with the following additions the code compiles successfully:

code/cmdline/cmdline.h:

#include <cstdint>
#include "globalincs/vmallocator.h"

code/cutscene/ffmpeg:

#include <thread>

code/cfile/cfile.h:

#include <array>

code/graphics/opengl/gropenglopenxr.h:

#include <cstdint>
#include <globalincs/vmallocator.h>

code/libs/discord/discord.h:

#include <globalincs/vmallocator.h>

code/math/floating.h:

#include <algorithm>

code/io/spacemouse.h:

#include <array>

code/network/multi_mdns.h:

#include <array>

code/network/multi_pxo.h:

#include <globalincs/vmallocator.h>

code/network/psnet2.h:

#include <iomanip>

code/osapi/osapi.h:

#include <fstream>

code/scripting/lua/LuaTypes.h:

#include <iostream>
#include <vector>

code/scripting/api/LuaPromise.h:

#include <functional>
#include <memory>

code/mission/missionhotkey.h:

#include <globalincs/vmallocator.h>

code/scripting/missionhotkey.h:

#include <globalincs/vmallocator.h>

code/gamehelp/gameplayhelp.h:

#include <globalincs/vmallocator.h>

[notimaginative]

Thanks for the report, and for putting up with the frustration of failing builds!

I'll start work on fixing those issues. And I really need to figure out what triggered all of these errors for you so that we can catch them early. I have been able to reproduce most of those errors in a container, but not outside of a container so far.

[sjnewbury]

@notimaginative It could be your build is using a different cpp (as in the preprocessor) outside of your container.

[freijon]

I'll start work on fixing those issues

have you made any progress?

[notimaginative]

I have the changes in a local branch, but I'm holding off on submitting the PR. I'd prefer to try and figure out exactly how to reproduce this so I can at least make sure that the CI pipeline will fail in this situation. That way we aren't playing whack-a-mole every few months as new changes/features roll in where this issue isn't caught.

[Goober5000]

I checked with @notimaginative, and he reported that attempts to replicate the problem on "9 different linux docker containers (across 4 different distros) plus macOS and different Windows configs" on current source were unsuccessful. So it's either fixed, or no longer a problem, or something we won't be able to fix.