Bad user experience (Perl version upgrades break XS modules)

[Paul "LeoNerd" Evans]

Today I can't run a bunch of programs on a server of mine, because they
all complain

lib/meta.c: loadable library and perl binaries are mismatched (got
first handshake key 0xf380080, needed 0xeb80080)

I'm a core perl developer and *I know what I am doing here*, and after
some amount of digging eventually I realised it's because the system
perl has been updated from the 5.38 it was last week to 5.40 now, which
means I'll have to rebuild all my XS modules in $HOME. OK.. fine...

For a general user experience though this is kinda horrible. There's
two things bad about this message itself:

1) The message itself doesn't really point at the problem. A nicer
message would have been

lib/meta.c: loadable library and perl binaries are mismatched
(perl is version 5.40.0, library needed 5.38.2)

This one is probably fairly easy to fix in core, can we take a
look at it?

2) In this specific case that's my `meta` module which because I'm
using Module::Build I put the meta.xs file in its full path in the
lib/ directory of source, so it embeds the full path in the
message. This also comes out in message string of other of my
modules, e.g.

lib/Object/Pad/FieldAttr/Checked.c: ...

It's not *great* as a user experience but it's a start.
Annoyingly, most people use EU:MM or similar and just shove the
.xs file in the root of their source. So that means we get a
message beginning:

XS.c: loadable library and perl binaries ...

Great. That's... not helpful. Which of the potentially-hundreds of
modules whose names end in ::XS does this message refer to?

This I also feel is really not hard to fix, because surely at
compiletime we know the full module name. Can we further make the
message read:

Loadable library for module Some::Package::XS and perl binaries
are mismatched (perl version is 5.40.0, library needed 5.38.2)

Already we're even better helping users here.


Can someone take a look at this? If nobody volunteers and takes a poke
I will probably get around to it "sometime", but really this shouldn't
be a hard one to fix, it's just a message string somewhere. It would
surely be an easy project for someone new(ish) to dip their toes into
and fix, and in the process make a lot of people - new users especially
- very happy. I'd be happy to offer a hand-holding where necessary.

For further context:

https://github.com/Perl/perl5/issues/15861

[Scott Baker]

I support this 1000%. This bites me too

-- Scott

Post by Paul "LeoNerd" Evans
Today I can't run a bunch of programs on a server of mine, because they
all complain
lib/meta.c: loadable library and perl binaries are mismatched (got
first handshake key 0xf380080, needed 0xeb80080)
I'm a core perl developer and *I know what I am doing here*, and after
some amount of digging eventually I realised it's because the system
perl has been updated from the 5.38 it was last week to 5.40 now, which
means I'll have to rebuild all my XS modules in $HOME. OK.. fine...
For a general user experience though this is kinda horrible. There's
1) The message itself doesn't really point at the problem. A nicer
message would have been
lib/meta.c: loadable library and perl binaries are mismatched
(perl is version 5.40.0, library needed 5.38.2)
This one is probably fairly easy to fix in core, can we take a
look at it?
2) In this specific case that's my `meta` module which because I'm
using Module::Build I put the meta.xs file in its full path in the
lib/ directory of source, so it embeds the full path in the
message. This also comes out in message string of other of my
modules, e.g.
lib/Object/Pad/FieldAttr/Checked.c: ...
It's not *great* as a user experience but it's a start.
Annoyingly, most people use EU:MM or similar and just shove the
.xs file in the root of their source. So that means we get a
XS.c: loadable library and perl binaries ...
Great. That's... not helpful. Which of the potentially-hundreds of
modules whose names end in ::XS does this message refer to?
This I also feel is really not hard to fix, because surely at
compiletime we know the full module name. Can we further make the
Loadable library for module Some::Package::XS and perl binaries
are mismatched (perl version is 5.40.0, library needed 5.38.2)
Already we're even better helping users here.
Can someone take a look at this? If nobody volunteers and takes a poke
I will probably get around to it "sometime", but really this shouldn't
be a hard one to fix, it's just a message string somewhere. It would
surely be an easy project for someone new(ish) to dip their toes into
and fix, and in the process make a lot of people - new users especially
- very happy. I'd be happy to offer a hand-holding where necessary.
https://github.com/Perl/perl5/issues/15861

[Felipe Gasper via perl5-porters]

This has bitten me as well and is, to the uninitiated, profoundly irksome. I think effort spending smoothing this over would be well-spent.

(Had I the bandwidth I would volunteer. Alas.)

-FG

Post by Paul "LeoNerd" Evans
Today I can't run a bunch of programs on a server of mine, because they
all complain
lib/meta.c: loadable library and perl binaries are mismatched (got
first handshake key 0xf380080, needed 0xeb80080)
I'm a core perl developer and *I know what I am doing here*, and after
some amount of digging eventually I realised it's because the system
perl has been updated from the 5.38 it was last week to 5.40 now, which
means I'll have to rebuild all my XS modules in $HOME. OK.. fine...
For a general user experience though this is kinda horrible. There's
1) The message itself doesn't really point at the problem. A nicer
message would have been
lib/meta.c: loadable library and perl binaries are mismatched
(perl is version 5.40.0, library needed 5.38.2)
This one is probably fairly easy to fix in core, can we take a
look at it?
2) In this specific case that's my `meta` module which because I'm
using Module::Build I put the meta.xs file in its full path in the
lib/ directory of source, so it embeds the full path in the
message. This also comes out in message string of other of my
modules, e.g.
lib/Object/Pad/FieldAttr/Checked.c: ...
It's not *great* as a user experience but it's a start.
Annoyingly, most people use EU:MM or similar and just shove the
.xs file in the root of their source. So that means we get a
XS.c: loadable library and perl binaries ...
Great. That's... not helpful. Which of the potentially-hundreds of
modules whose names end in ::XS does this message refer to?
This I also feel is really not hard to fix, because surely at
compiletime we know the full module name. Can we further make the
Loadable library for module Some::Package::XS and perl binaries
are mismatched (perl version is 5.40.0, library needed 5.38.2)
Already we're even better helping users here.
Can someone take a look at this? If nobody volunteers and takes a poke
I will probably get around to it "sometime", but really this shouldn't
be a hard one to fix, it's just a message string somewhere. It would
surely be an easy project for someone new(ish) to dip their toes into
and fix, and in the process make a lot of people - new users especially
- very happy. I'd be happy to offer a hand-holding where necessary.
https://github.com/Perl/perl5/issues/15861
--
Paul "LeoNerd" Evans
http://www.leonerd.org.uk/ | https://metacpan.org/author/PEVANS

[Nicolas Mendoza via perl5-porters]

Post by Paul "LeoNerd" Evans
…
XS.c: loadable library and perl binaries ...

I'm still looking for which module is triggering such an error here with
strace, Devel::Trace and what not.

This happens to me once a year when I upgrade my distro and my
local::lib gets out of sync (yeah yeah I'm running system perl locally,
brew perl still doesn't compile due to failing locale test).

[Michael Conrad]

Post by Paul "LeoNerd" Evans
Can someone take a look at this? If nobody volunteers and takes a poke
I will probably get around to it "sometime", but really this shouldn't
be a hard one to fix, it's just a message string somewhere. It would
surely be an easy project for someone new(ish) to dip their toes into
and fix, and in the process make a lot of people - new users especially
- very happy. I'd be happy to offer a hand-holding where necessary.
https://github.com/Perl/perl5/issues/15861

Technical problems aside, the message I'd like to see is

Perl module $full_module_name tried loading shared C library
$full_path_to_lib, but that library was compiled for a different perl
than the current $full_path_to_interpreter.  This is often caused by
updating your installed perl without updating the modules to match, or a
mis-configured library path.  See perldoc $relevant_section.

The linked docs should have helpful suggestions about how upgrading a
Debian package for perl when the user has XS local::lib packages will
cause this, and how users are recommended to use distro-provided
packages  for perl modules, and if they can't do that we recommend they
use perlbrew or plenv to maintain a stable version of perl for their
custom-installed modules.

This message also sidesteps technical issues with determining what perl
the .so *was* for and the fact that two perls of the same exact version
could be incompatible, and just requires a few path lookups and some
inspection of the stack. (or, tweak DynaLoader to localize a global
during operation and save some effort in the C code)

-Mike C

[Paul "LeoNerd" Evans]

On Mon, 28 Oct 2024 12:31:31 +0000

Post by Paul "LeoNerd" Evans
Can someone take a look at this? If nobody volunteers and takes a poke
I will probably get around to it "sometime", but really this shouldn't
be a hard one to fix, it's just a message string somewhere. It would
surely be an easy project for someone new(ish) to dip their toes into
and fix, and in the process make a lot of people - new users
especially
- very happy. I'd be happy to offer a hand-holding where necessary.

Anyone had any thoughts on this at all? It's still being very annoying
for me - and I would imagine everyone else too.

Example today:

$ perlsh
PadWalker.c: loadable library and perl binaries are mismatched (got
first handshake key 0xf380080, needed 0xeb80080)

$ cpan PadWalker
Loading internal logger. Log::Log4perl recommended for better logging
Reading '/home/leo/.cpan/Metadata'
Database was generated on Mon, 20 Jan 2025 18:29:01 GMT
PadWalker is up to date (2.5).

????

This really is a terrible user experience, I'm sure it could be made
much better by just including the $^V in the inc path:

$ perl -V
...
@INC:
/home/leo/lib/perl5/x86_64-linux-gnu-thread-multi

Why does that not say

/home/leo/lib/perl5/x86_64-linux-gnu-thread-multi/5.40.0

I think that one small change would fix almost all of this problem.

Anyone want to tackle it?

[Michael Conrad]

Post by Paul "LeoNerd" Evans
On Mon, 28 Oct 2024 12:31:31 +0000

Post by Paul "LeoNerd" Evans
Can someone take a look at this? If nobody volunteers and takes a poke
I will probably get around to it "sometime", but really this shouldn't
be a hard one to fix, it's just a message string somewhere. It would
surely be an easy project for someone new(ish) to dip their toes into
and fix, and in the process make a lot of people - new users
especially
- very happy. I'd be happy to offer a hand-holding where necessary.

Anyone had any thoughts on this at all? It's still being very annoying
for me - and I would imagine everyone else too.

There was a lot of discussion on the github ticket.  I'm really hoping
people decide to go with adding a new metadata function to each .so, and
I proposed a pseodocode patch to that effect, but didn't see any
approval of that direction by core developers.  I'd be willing to make
that into a real patch if I thought it would get approved.

-Mike