`use VERSION` to imply `use builtin :VERSION`

[Paul "LeoNerd" Evans]

A while ago [1] we discussed the idea of getting e.g.

use v5.40;

to imply

use builtin ':5.40';

so as to import all the stable builtins at that version. The discussion
got a bit derailed in the details of how to do lexical unimport, but I
believe all that is now resolved and we can get back to this.

Now we can do lexical unimport, and we have `no builtin ...` working, I
don't believe there were any significant reasons why not to do this.
The list generally seemed to be in favour on the overall reasons of
further reduced boilerplate, so I think I'm gonna go ahead and
implement this today. Expect a PR inbound..


[1]: https://www.nntp.perl.org/group/perl.perl5.porters/2023/08/msg266808.html

[Paul "LeoNerd" Evans]

On Wed, 17 Jan 2024 11:23:28 +0000

Post by Paul "LeoNerd" Evans
A while ago [1] we discussed the idea of getting e.g.
use v5.40;
to imply
use builtin ':5.40';
so as to import all the stable builtins at that version. The
discussion got a bit derailed in the details of how to do lexical
unimport, but I believe all that is now resolved and we can get back
to this.

Hmmm. In writing the documentation for this I have hit upon a snag.

Implementing the first bit was really easy; it's basically a 3-line
addition into op.c to just activate the builtin bundle. So now this
works:

$ ./perl -e 'use v5.39.6; say "True is ", true'
True is 1

I then looked at updating the documentation about `use VERSION` and
found the wording about what happens when you reset back down to a
lower version number again. It specifically says of the strict,
warnings and feature pragmata, that it resets them down again and
removes whatever shouldn't be visible. If builtin bundles were to work
the same way, then this ought to fail:

$ ./perl -ce 'use v5.39; use v5.20; my $t = true'
-e syntax OK

But, it currently succeeds. The behaviour here ought to be that the
`use v5.20` throws out the imported builtin functions. Currently that
kind of behaviour doesn't exist.

Back in Perl 5.36 we added a trapdoor to `use VERSION` to warn when
downgrading back before the 5.11 boundary (when the implicit 'strict'
flags were enabled), in order to give us some room to get rid of those
flags and free up some hints bits. I wonder if a similar warning or
outright-error should happen if you ask to downgrade over the 5.39
boundary. I.e. that upgrades of the prevailing `use VERSION` are always
permitted, but going back down again is not permitted past that.

This would make theabove example into an outright error:

$ ./perl -ce 'use v5.39; use v5.20;'
Downgrading a use VERSION declaration to below v5.39 is not permitted
at -e line 1.

Since the effects are entirely lexically scoped, you're still perfectly
permitted to have small islands of higher-versions within a file if you
need it, because the scoping rules already handle how to tidy up:

use v5.20;
{
use v5.39.6;
my $t = true; # perfectly fine
}
# now we're back in 5.20-land with none of that visible anyway.

Adding such a trapdoor would simplify a lot of similar situations that
might come up in future, when `use VERSION` enables yet more things.
Not needing to answer the question of "how do you turn it off again?"
generally tidies things up.

In summary, what I'm asking is: How do we feel about disallowing
downgrades of `use VERSION` within a single scope? That the only way to
step backwards is to fall out of an inner scope with higher version,
back outside into a lower version again.

[Paul "LeoNerd" Evans]

On Wed, 17 Jan 2024 19:11:39 +0100 (CET)

I wonder if a similar warning or outright-error should happen if you
ask to downgrade over the 5.39 boundary. ... what I'm asking is: How
do we feel about disallowing downgrades of `use VERSION` within a
single scope?

If you make it only an error when downgrading from v5.39 or higher,
then that can't be anything which somebody is currently relying on in
production code; it can't break anything.

Oh indeed; there's no risk of compatibility problem, it's just a
question of if people find it overly restrictive when writing new code
going forward.

That said I've never wanted to downgrade a `use VERSION` in any
situation ever, so I don't think I can imagine many cases for it to be
a problem.

If you make it an error now, there is always the possibility of
somebody finding a way of making it work in the future and later
changing Perl to allow it. (Whereas allowing it now but making it an
error later wouldn't be acceptable.)
If you make it an error now, we might get a sense of how useful it
actually is in practice to be able to do this — that is, how often do
programmers find themselves trying to do something but getting this
error instead. If it turns out to be hardly ever, then it was the
right choice to make it an error; if it happens fairly often, then
work could perhaps be done at that point, with the advantage of
knowing that it would actually be a useful and requested feature.
So it seems safest all round to go for making it an error.

Yes that's true - if we find a way to implement it later we can allow
it. I've put words to that effect in my PR.

For folks to review:

https://github.com/Perl/perl5/pull/21850

[Paul "LeoNerd" Evans]

On Wed, 17 Jan 2024 17:59:29 +0000

Post by Paul "LeoNerd" Evans
In summary, what I'm asking is: How do we feel about disallowing
downgrades of `use VERSION` within a single scope? That the only way
to step backwards is to fall out of an inner scope with higher
version, back outside into a lower version again.

We discussed this at yesterday's PSC and found to be an unsatisfactory
solution to the problem. So we return to thinking about how to
downgrade the `builtin` bundle by unimporting symbols.

We do have the unimport ability now so it would be possible to have a

use builtin ':VER';

work out what symbols to import or unimport, based on scanning what is
already visible in the pad. Careful checking would be needed to ensure
it doesn't accidentally erase unrelated functions that just happen to
have the same names, but that's not too hard to write.

This would mean that `use VERSION` could do the same thing for builtin
functions as it does for feature names. Any `use VERSION` would leave
enabled only the ones appropriate for that version and anything else
that had been previously enabled is now removed again. While more
effort to implement, it does overall seem like a neater model for users.

There's a bunch of really weird cornercases about this that I'm still
currently unsure of. For example, if you unimport the builtin::true
symbol in an inner scope but you did have an unrelated `my sub true`
outside that, does that become visible again?

I.e. does this test pass?

my sub true () { "true" }
is( true, "true", 'calls outer &true' );

{
use v5.40;
is( true, 1, 'builtin::true looks like 1' );

use v5.36;
is( true, "true", 'calls outer &true' );
## Currently I am unsure if by now true() is entirely absent,
## or just hides the builtin one and leaves the outer again
}

I will continue to take thoughts and comments as people have them...

[G.W. Haywood]

Hi there,

...
... `use VERSION` could ... leave [as] appropriate for that version
... a neater model for users.

+1

... if you unimport the builtin::true symbol in an inner scope but
you did have an unrelated `my sub true` outside that, does that
become visible again?

Yes, obviously. Otherwise the inner scope is trashing the outer scope
and that sort of defeats the object of the scoping.

... does this test pass?

perl -e 'my sub true () {"true"} { my sub true () {"TRUE"} print true (), "\n"; } print true (), "\n";'
TRUE
true

perl -e 'sub true () {"true"} { sub true () {"TRUE"} print true (), "\n"; } print true (), "\n";'
Constant subroutine true redefined at -e line 1.
TRUE
TRUE

I'd expect no less of 'use VERSION'.

[Paul "LeoNerd" Evans]

On Fri, 19 Jan 2024 13:40:19 +0000

Post by Paul "LeoNerd" Evans
We discussed this at yesterday's PSC and found to be an unsatisfactory
solution to the problem. So we return to thinking about how to
downgrade the `builtin` bundle by unimporting symbols.
We do have the unimport ability now so it would be possible to have a
use builtin ':VER';
work out what symbols to import or unimport, based on scanning what is
already visible in the pad. Careful checking would be needed to ensure
it doesn't accidentally erase unrelated functions that just happen to
have the same names, but that's not too hard to write.
This would mean that `use VERSION` could do the same thing for builtin
functions as it does for feature names. Any `use VERSION` would leave
enabled only the ones appropriate for that version and anything else
that had been previously enabled is now removed again. While more
effort to implement, it does overall seem like a neater model for users.

Latest attempt on the PR now has this behaviour.

I've implemented it in the same way as the detail I've observed
behind the 'feature' pragma, even though I don't like it either.

That is to say, that already (ever since perl 5.10 or so I guess),
`use VERSION` and `use feature :VERSION` aren't *quite* the same. The
difference is that a `use feature` bundle only turns on features but
leaves existing ones alone, whereas `use VERSION` resets any existing
ones first. As demonstrated:

$ perl -e 'use feature ":5.16"; use feature ":5.10"; say fc("Hello")'
hello

$ perl -e 'use v5.16; use v5.10; say fc("Hello")'
Downgrading a use VERSION declaration to below v5.11 is deprecated, and will become fatal in Perl 5.40 at -e line 1.
Undefined subroutine &main::fc called at -e line 1.

Here for example, the first `use feature` bundle enabled the 'fc'
feature which the second did not bother to turn off again. But while
the first `use VERSION` had turned it on, the second turned it off
again.

I'm not entirely sure that I like this model, but nevertheless for
consistency I have done the same thing for `builtin` bundles. Namely,
that a `use builtin :VERSION` will import extra things but won't care
about whatever builtins are existing, but a full `use VERSION` will now
scrub the scope of things that shouldn't be there.

$ ./perl -e 'use builtin "is_bool"; use builtin ":5.39"; print is_bool(0) ? "yes\n" : "no\n"'
Built-in function 'builtin::is_bool' is experimental at -e line 1.
no

$ ./perl -e 'use builtin "is_bool"; use v5.39; print is_bool(0) ? "yes\n" : "no\n"'
Undefined subroutine &main::is_bool called at -e line 1.

[Paul "LeoNerd" Evans]

On Fri, 19 Jan 2024 15:50:54 +0000

Post by Paul "LeoNerd" Evans
Latest attempt on the PR now has this behaviour.

Reminder all, the PR is

https://github.com/Perl/perl5/pull/21850

if no further comments by perhaps end of tomorrow, I shall merge it.

[G.W. Haywood]

Hi there,

... I've defined lots of trim functions throughout my projects ...

Ditto.

[Darren Duncan]

I agree with the following. Make it an error now. -- Darren Duncan

I wonder if a similar warning or outright-error should happen if you
ask to downgrade over the 5.39 boundary. ... what I'm asking is: How
do we feel about disallowing downgrades of `use VERSION` within a
single scope?

If you make it only an error when downgrading from v5.39 or higher, then
that can't be anything which somebody is currently relying on in
production code; it can't break anything.
If you make it an error now, there is always the possibility of somebody
finding a way of making it work in the future and later changing Perl to
allow it. (Whereas allowing it now but making it an error later wouldn't
be acceptable.)
If you make it an error now, we might get a sense of how useful it
actually is in practice to be able to do this — that is, how often do
programmers find themselves trying to do something but getting this
error instead. If it turns out to be hardly ever, then it was the right
choice to make it an error; if it happens fairly often, then work could
perhaps be done at that point, with the advantage of knowing that it
would actually be a useful and requested feature.
So it seems safest all round to go for making it an error.
Smylers