postgresql: fix cross for FreeBSD #387727

rhelmot · 2025-03-06T20:59:08Z

Fix dependencies
Fix build of plugins in a cross context by providing pg_config_native copied from an equivalent native build and wrapped to provide the correct paths. The postgres source code indicates that there have been plans to do this upstream for a while, but that they are too complex to do "correctly".
Fix the --export-dynamic flag - it is applicable for all lld linkers, and it should be spelled differently.

Pull requests using pg_config_native forthcoming.

Things done

Add a 👍 reaction to pull requests you find important.

drupol · 2025-03-06T21:09:22Z

It should target staging too.

rhelmot · 2025-03-06T21:15:03Z

Hm... darwin mass rebuild makes sense but I was really expecting rebuild-linux: 0.

pkgs/servers/sql/postgresql/generic.nix

wolfgangwalther · 2025-03-06T21:28:29Z

pkgs/servers/sql/postgresql/generic.nix

+          cp ${nativeBuild.dev}/bin/.pg_config-wrapped $dev/bin/.pg_config_native-wrapped
+          # wrapProgram will use the wrong shell. do it ourselves
+          cat >"$dev/bin/pg_config_native" <<EOF
+          #!${lib.getExe buildPackages.bash}
+          exec > >(sed ${
+            lib.concatMapStringsSep " " (
+              output: "-e s@${nativeBuild."${output}"}@${builtins.placeholder output}@g"
+            ) finalAttrs.outputs
+          })
+          exec -a "${builtins.placeholder "out"}/bin/pg_config" "${builtins.placeholder "dev"}/bin/.pg_config_native-wrapped" "\$@"
+          EOF
+          chmod +x "$dev/bin/pg_config_native"


So the idea here is, that you're replacing the native paths with the cross paths, right?

But this is not the only thing that pg_config is returning. It is also returning CFLAGS etc. - and those could be different between native and cross builds, too.

This seems very fragile.

With all the pg_config hackery that we already do elsewhere... we should seriously consider to just write our own shell-script based pg_config implementation doing exactly the right things, which could be run for both native and cross cases.

I agree, it is potentially very fragile. I don't really know how to approach the "correct" solution though, and this does seem to work while at the same time being simple enough that its intent (I can document this inline) is pretty basic and can be easily understood as a hack to be replaced as soon as it gives anyone trouble.

Do you think it's a bad enough hack that it shouldn't be in nixpkgs in order to build what it can?

Do you think it's a bad enough hack that it shouldn't be in nixpkgs in order to build what it can?

It is indeed a very bad hack. I'd like to take a shot at rewriting pg_config as a script first, before judging whether this hack is too bad for right now or not.

I will give it a shot and come back with some results.

I'd like to take a shot at rewriting pg_config as a script first, before judging whether this hack is too bad for right now or not.

Draft in #388807. I like this approach a lot more, it's much cleaner not only for cross, but also for all the other oddities that we have to fight upstream's pg_config on.

pkgs/servers/sql/postgresql/generic.nix

rhelmot · 2025-03-06T22:18:22Z

Fixed all the issues that don't require rewriting pg_config.

- Fix dependencies - Fix build of plugins in a cross context by providing pg_config_native copied from an equivalent native build and wrapped to provide the correct paths. The postgres source code indicates that there have been plans to do this upstream for a while, but that they are too complex to do "correctly". - Add the --export-dynamic flag - it is applicable for all non-macos cross configurations.

wolfgangwalther · 2025-03-09T12:05:06Z

pkgs/servers/sql/postgresql/generic.nix

+      nativeBuild =
+        buildPackages."postgresql_${lib.versions.major version}${lib.optionalString jitSupport "_jit"}";


This approach will probably also break with overrides, right?

So when I do pkgsCross.[freebsd...].postgresql.override { [... toggle some flags ...] }, then the native pg_config will still be taken from a derivation without those flags toggled.

Indeed. This approach is predicated on the assumption that the paths are the only thing that are essential. So far this is correct, but critically, I have been only compiling pgrx (rust) plugins.

wolfgangwalther · 2025-03-09T12:08:00Z

pkgs/servers/sql/postgresql/generic.nix

+        # some version of this flag is required in all cross configurations
+        # since it cannot be automatically detected
+        ++
+          lib.optionals
+            ((!stdenv'.hostPlatform.isDarwin) && (!(stdenv'.buildPlatform.canExecute stdenv.hostPlatform)))
+            [
+              "LDFLAGS_EX_BE=-Wl,--export-dynamic"
+            ]


It would be good to understand why this flag can't be detected automatically. Is it inherent to cross-compilation? Or is just a limitation of the build system?

In general, PostgreSQL's autoconf-based build system is not built for cross-compilation at all. But since v16, they also have a meson based build system. I have a branch locally which builds with meson for 16+. Didn't need that in the end, when enabling the build for pkgsStatic... but maybe it would be time to pick this up again?

Cross-compiling should work much better with meson.

Would support for v16+ be enough for your use-case? I assume that there is no need to build older versions for FreeBSD, because nobody is running those with nix there anyway, yet?

It would be good to understand why this flag can't be detected automatically. Is it inherent to cross-compilation? Or is just a limitation of the build system?

I tried building... and I don't need to add this flag explicitly at all. The build succeeds. So either the flag is not required - or it needs more explanation (run-time? build time for dependencies?)

Wow! I saw this question come in on my phone and I immediately got out of bed in order to type up the actual answer.

The answer is related to plugins, which are shared objects loaded by the postgres binary. These objects try to link against symbols from the postgres binary, which fails without this flag because the linker garbage collection settings are so intense that they completely delete some functions that are only required by these plugins. The flag places the symbols in the dynamic symbol table .dynsym (as opposed to the normal one .symtab), which is a) sufficient to not garbage collect them and b) required (haven't empirically verified this) for dynamic linking against them to work. So I believe the actual test for "is this build good" is nm postgres | grep '\<lowerstr\>', for example.

With respect to the first question in the thread, the postgres m4 files claim that it is inherent to cross compilation. See c-compiler.m4 and configure.ac have some entirely unhelpful comments on the matter and I don't have access to a platform where this flag is unsupported so I'm not super sure how to verify that claim.

The answer is related to plugins, which are shared objects loaded by the postgres binary. These objects try to link against symbols from the postgres binary, which fails without this flag because the linker garbage collection settings are so intense that they completely delete some functions that are only required by these plugins. The flag places the symbols in the dynamic symbol table .dynsym (as opposed to the normal one .symtab), which is a) sufficient to not garbage collect them and b) required (haven't empirically verified this) for dynamic linking against them to work.

Yes, I understand and agree with all of that. That's what I worked out, too, when I fixed the darwin case back then.

So I believe the actual test for "is this build good" is nm postgres | grep '<lowerstr>', for example.

Ah, right. So for the darwin case it was a native build, so it was running the test-suite - which then failed at run-time. But that was still "build time" in terms of "building the nix derivation". Now, with cross, the tests are ofc not run, so I'll not hit that.

Native linux:

% nm /nix/store/m3951xl7r08y7ja0x79b8hy4gkbw07hd-postgresql-17.4/bin/postgres | grep lowerstr 00000000004f3320 T lowerstr 00000000004f31c0 T lowerstr_with_len 0000000000153d4b t lowerstr_with_len.cold

cross freebsd:

% nm /nix/store/y3c1cdannia6wy0ym4k5jg1v9xc06wzb-postgresql-x86_64-unknown-freebsd-13.20/bin/postgres | grep lowerstr 0000000000916950 T lowerstr 0000000000916970 T lowerstr_with_len

Not sure what that means. Probably need another symbol to test against.

With respect to the first question in the thread, the postgres m4 files claim that it is inherent to cross compilation. See c-compiler.m4 and configure.ac have some entirely unhelpful comments on the matter and I don't have access to a platform where this flag is unsupported so I'm not super sure how to verify that claim.

Ah, thanks for digging this out, specifically this part:

In fact, we must actually check that the resulting program runs :-(

I see. So that's essentially exactly the thing that I mentioned about the autoconf-based build system. It depends on this configure-time "build and run" check. The meson build system, does not. It depends on knowledge about the different linkers, see https://github.com/mesonbuild/meson/blob/master/mesonbuild/linkers/linkers.py and grep for export-dynamic.

So I think with meson, this should not be a problem and the flag should be added regardless.

So I think with meson, this should not be a problem and the flag should be added regardless.

Once I'm done with the pg_config stuff, I will dig out my old meson branch again and rebase that. Then we can have a look at whether that would make cross-compiling easier to do.

wolfgangwalther · 2025-03-09T12:09:27Z

pkgs/servers/sql/postgresql/generic.nix

+          cp ${nativeBuild.dev}/bin/.pg_config-wrapped $dev/bin/.pg_config_native-wrapped
+          # wrapProgram will use the wrong shell. do it ourselves
+          cat >"$dev/bin/pg_config_native" <<EOF
+          #!${lib.getExe buildPackages.bash}
+          exec > >(sed ${
+            lib.concatMapStringsSep " " (
+              output: "-e s@${nativeBuild."${output}"}@${builtins.placeholder output}@g"
+            ) finalAttrs.outputs
+          })
+          exec -a "${builtins.placeholder "out"}/bin/pg_config" "${builtins.placeholder "dev"}/bin/.pg_config_native-wrapped" "\$@"
+          EOF
+          chmod +x "$dev/bin/pg_config_native"


Do you think it's a bad enough hack that it shouldn't be in nixpkgs in order to build what it can?

It is indeed a very bad hack. I'd like to take a shot at rewriting pg_config as a script first, before judging whether this hack is too bad for right now or not.

I will give it a shot and come back with some results.

wolfgangwalther · 2025-03-09T12:12:23Z

Fixed all the issues that don't require rewriting pg_config.

One other things that comes to mind: The separate libpq package probably deals with the same problem. Did you try building and using in downstream dependencies, yet?

Ofc, if those dependencies behave well, they will use pkg-config and don't need pg_config. But not all do...

ofborg bot added the 6.topic: cross-compilation Building packages on a different platform than they will be used on label Mar 6, 2025

nix-owners bot requested review from Ma27, wolfgangwalther and thoughtpolice March 6, 2025 21:00

rhelmot force-pushed the freebsd-postgres branch from 15a7c87 to 40fe7d6 Compare March 6, 2025 21:01

github-actions bot added 10.rebuild-darwin: 1001-2500 10.rebuild-darwin: 501+ 10.rebuild-linux: 101-500 labels Mar 6, 2025

rhelmot force-pushed the freebsd-postgres branch from 40fe7d6 to 01a4faa Compare March 6, 2025 21:15

rhelmot changed the base branch from master to staging March 6, 2025 21:16

github-actions bot added 10.rebuild-darwin: 101-500 and removed 10.rebuild-darwin: 1001-2500 10.rebuild-darwin: 501+ labels Mar 6, 2025

wolfgangwalther requested changes Mar 6, 2025

View reviewed changes

rhelmot force-pushed the freebsd-postgres branch from 01a4faa to b504050 Compare March 6, 2025 22:18

rhelmot force-pushed the freebsd-postgres branch from b504050 to db88aa1 Compare March 6, 2025 22:22

github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux and removed 10.rebuild-darwin: 101-500 10.rebuild-linux: 101-500 labels Mar 6, 2025

rhelmot mentioned this pull request Mar 7, 2025

cargo-pgrx: fix cross compilation #387806

Open

13 tasks

wolfgangwalther reviewed Mar 9, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

postgresql: fix cross for FreeBSD #387727

postgresql: fix cross for FreeBSD #387727

rhelmot commented Mar 6, 2025

drupol commented Mar 6, 2025

rhelmot commented Mar 6, 2025

wolfgangwalther Mar 6, 2025

rhelmot Mar 6, 2025

wolfgangwalther Mar 9, 2025

wolfgangwalther Mar 10, 2025

rhelmot commented Mar 6, 2025

wolfgangwalther Mar 9, 2025

rhelmot Mar 10, 2025

wolfgangwalther Mar 9, 2025

wolfgangwalther Mar 10, 2025

rhelmot Mar 10, 2025

wolfgangwalther Mar 10, 2025

wolfgangwalther Mar 10, 2025

wolfgangwalther Mar 9, 2025

wolfgangwalther commented Mar 9, 2025

		nativeBuild =
		buildPackages."postgresql_${lib.versions.major version}${lib.optionalString jitSupport "_jit"}";

postgresql: fix cross for FreeBSD #387727

Are you sure you want to change the base?

postgresql: fix cross for FreeBSD #387727

Conversation

rhelmot commented Mar 6, 2025

Things done

drupol commented Mar 6, 2025

rhelmot commented Mar 6, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhelmot commented Mar 6, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wolfgangwalther commented Mar 9, 2025