SDK: Fix so libraries dangling symlinks#294
SDK: Fix so libraries dangling symlinks#2943v1n0 wants to merge 10 commits intoubuntu:gnome-46-2404-sdkfrom
Conversation
3v1n0
force-pushed
the
sdk-fix-dangling-symlinks
branch
2 times, most recently
from
98b91e4 to
772fcd6
Compare
sergio-costas
left a comment
sergio-costas
left a comment
There was a problem hiding this comment.
It looks fine, and even has a check to notify for any left dangling symlink.
|
Before merging this... what happens with the review tools? It seems that they do ensure that no soft link points outside the snap... |
|
So maybe the solution is to add all those links in Core24... |
Oh I didn't check review tools output... But if they don't work with this, then... The only option would be to copy the actual libraries in the sdk too and ensure that they don't get removed as duplicate... |
I still think that they should go in Core24: if you think about it, it IS being used to compile, when you build any snap... |
|
I really dislike having duplicated libraries. |
Yeah, but I feel that's the way... Since I'm pretty sure that core doesn't want to include dev packages contents. While us being a dev sdk we need to provide everything required to build. So if core is missing something, we have to. Implies more maintenance though, I agree. |
|
So... do we copy the required files from CoreXX into the right folder wherever there is a dangling pointer...? |
|
AFAIK is the only way we can... And ensure they get not deleted by duplicates checking tool |
|
Moving from #296:
What do you mean with "build itself from scratch", what is "itself"? In Libtool's case the -dev package (libc6-dev) was added to build-packages. |
Exactly: since Core24 doesn't have the soft link |
|
In fact... why is libpciaccess in gnome-46-2404-sdk if it is already in Core24? |
|
Ok, confirmed my suspicions: just by removing the broken soft link, the The problem is that the So if we remove a library in the gnome SDK because it is already in Core24, we must also remove the corresponding .pc file to avoid this. |
That makes sense. We never built libc6 though. |
|
@3v1n0 So, in my opinion, the true solution is to find which .pc file correspond to each duplicated library in Core24 and remove it. It's not easy, I now, but with some python magic... ;-) |
|
In fact... just checking the "Libs" and "Libs.private" parts should be enough to know which libraries are affected by a .pc file, so if they aren't available in gnome-XX-YY-sdk, then that .pc file can be removed. |
Well, I'm unsure either... These are build dependencies of dependencies we have in the SDK IIRC, I mean if any of these libraries headers/pc files are part of the gnome-sdk, then we should provide them too. For example for the pciaccess one: Since we have stage packages such as It's fine to remove them from the runtime snap, but not from the sdk. |
|
But then, what we have to do is to ensure that using the gnome SDK requires to install the corresponding stage-packages that are provided by CoreXX. |
|
Anyway... you don't necessarily need those dependencies to build something with those libraries... and if you need them for whatever reason, the build error will tell you what you have to add to the |
|
Ok, let's take a step back and see the whole thing... Core24 (and other Cores) are built just by "dumping" an Ubuntu base system, as seen here: https://github.com/canonical/core-base/blob/277049e8bf67fdeadb9afe92428220e02cf95142/snapcraft.yaml#L13 The point is that, when building new snaps against CoreXX, the development files aren't there. Where are they get? And the answer is that Also, there is a list of packages that must not be staged, because they are known to be included from Core... https://github.com/canonical/snapcraft/blob/0157e5b15e66af34a4b12d397e5f56a30eebb13d/snapcraft_legacy/internal/repo/_deb.py#L51 But all that I found at this point is only available in So the point, in my opinion, is to find where those required -dev files are defined, and send a patch for snapcraft to add |
|
Ok, it seems that the list of blocked packages is in craft-parts: https://github.com/canonical/craft-parts/blob/1f163c0ec0691182cf08da52cc1cfc7160618241/craft_parts/packages/deb.py#L57-L283 So... Should we add there |
This reverts commit df43897.
The GNOME SDK should contain all the required dev-dependencies of the libraries we ship, including the core files dependencies. Now, even though core includes already some of the runtime libraries it does not include their *.so counterparts, and then that's now our duty to provide them all and in a working state. Since we can't really symlink to external paths in a snap, we should provide all the libraries here, since there's no way to fix the *.so files that are used at link time. Dropping them would be wrong though, since there may be libraries in the SDK that requires them at link time only, and it's not up to final applications to manage that.
If a dangling *.so file is found, it means that a package it links has not been staged, so fail in such case
We already have code to remove duplicates and these libraries are already dropped as part of the previous lines
3v1n0
force-pushed
the
sdk-fix-dangling-symlinks
branch
from
8baa7ce to
c39eb7b
Compare
|
It seems that krb5-multidev is either required, o removing the dangling pointer |
It looks a bit out of place here TBH. @nteodosio do you know what could have pulled it in? |
|
Yes, libkrb5-dev, which in turn is pulled by Soup3.
Message ID: ***@***.***>
|
|
@nteodosio It fails to build... |
nteodosio
force-pushed
the
sdk-fix-dangling-symlinks
branch
from
ebc13af to
5328b70
Compare
|
Thanks, with the last commit we went from down to So still needing libffi8, libelf1t64, libk5crypto3 and libkrb5support0. |
…krb5support0. They provide the targets of the -dev's symlinks.
nteodosio
force-pushed
the
sdk-fix-dangling-symlinks
branch
from
5328b70 to
75c8986
Compare
|
Those are now gone but eventually more popped up. Probably something needs to be re-worked, we can't go whack-a-moling ad infinitum. |
Well, pkg-config can help us with that... We should just follow the whole dependency chain. |
|
All green, can we merge now? |
Did you build the RUNTIME from this, and tested it to ensure that everything works as expected? |
|
No, but I can. Is this requirement and the corresponding process documented somewhere? |
|
It should be a requirement :-D I use lxcraft to build it, with these changes in the snapcraft.yaml: and this lxcraft.yaml configuration file: (of course, I built the SDK snap at |
|
Here you can see the list of snaps that I use to test each new SDK: #315 |
|
Sorry, my connection is not only slow but today also flaky,
```
\rRetrieving image: metadata: 13% (93.34kB/s)\r \rRetrieving image:
metadata: 14% (96.15kB/s)\r \rRetrieving image: metadata: 14%
(96.15kB/s)\r \r' * Command standard error output: b'Error: Failed
instance creation: read tcp 192.168.0.5:49524->185.125.190.40:443: read:
connection reset by peer\n'
```
I'll have to leave that for tomorrow or give you the honor. :P
Message ID: ***@***.***>
|
|
I'll do it. |
Can we make a CI job that handles this a bit like autopkgtests? So whenever we push to main (at least) or a PR is approved, we check if some snaps are building fine. |
|
I'm preparing a patch for the runtime that allows to easily build it from a local SDK snap. |
|
Good, then can we also use it in CI? |
|
Yes, that's my idea. |
3 participants
libraries .so files provided by -dev packages normally point to the
versioned so file, but if these files are provided by the core snap or
another base snap then we should instead use them, if they're not
provided by the SDK itself.
So, ensure that all the symlinks in the lib directory are actually
pointing to a library file, trying to resolve them with the
core-provided libraries as first try.
This fixes the compile issue we have with gnome-boxes as per the linking issues:
This fixes also these other cases:
UDENG-6680