Add a Lint for Pointer to Integer Transmutes in Consts

[veera-sivarajan]

Fixes #87525

This PR adds a MirLint for pointer to integer transmutes in const functions and associated consts. The implementation closely follows this comment: #85769 (comment). More details about the implementation can be found in the comments.

Note: This could break some sound code as mentioned by RalfJung in #85769 (comment):

... technically const-code could transmute/cast an int to a ptr and then transmute it back and that would be correct -- so the lint will deny some sound code. Does not seem terribly likely though.

References:

1. https://doc.rust-lang.org/std/mem/fn.transmute.html
2. https://doc.rust-lang.org/reference/items/associated-items.html#associated-constants

[rustbot]

r? @estebank

rustbot has assigned @estebank.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

[veera-sivarajan]

Not sure if this should be Deny or Warn by default.

[rust-log-analyzer]

The job x86_64-gnu-tools failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

   Compiling compiler_builtins v0.1.125
error: pointers cannot be transmuted to integers during const eval
    --> core/src/ptr/mod.rs:1985:32
     |
1985 |     let addr: usize = unsafe { mem::transmute(p) };
     |
     = note: at compile-time, pointers do not have an integer value
     = note: at compile-time, pointers do not have an integer value
     = note: avoiding this restriction via `union` or raw pointers leads to compile-time undefined behavior
     = help: for more information, see https://doc.rust-lang.org/std/mem/fn.transmute.html
     = note: `#[deny(ptr_to_integer_transmute_in_consts)]` on by default
error: could not compile `core` (lib) due to 1 previous error
Build completed unsuccessfully in 0:05:12
  local time: Thu Sep 19 02:44:00 UTC 2024
  network time: Thu, 19 Sep 2024 02:44:00 GMT

[scottmcm]

What's the motivation for this lint? https://doc.rust-lang.org/std/primitive.pointer.html#method.addr and https://doc.rust-lang.org/std/ptr/fn.without_provenance.html are transmutes, and I'd thought that this was considered basically fine now, albeit possibly not what you wanted if you need to preserve provenance.

[veera-sivarajan]

Transmuting a pointer to integer in const context is undefined behavior¹. Usually, the evaluator will abort and emit an error when it comes across such undefined transmutes.

But const functions and associated consts are evaluated only when referenced. This can result in undefined behavior in a library going unnoticed until the function or constant is actually used. Therefore, this lint specifically targets pointer to integer transmutes in const functions and associated consts.

From what I can tell, this lint is not related to the functions you have linked because:

1. addr() is not a const function.
2. without_provenance() transmutes an integer to pointer.

Footnotes

1. https://doc.rust-lang.org/std/mem/fn.transmute.html#transmutation-between-pointers-and-integers ↩

[scottmcm]

Oh, of course; thank you. Brain fart on my part.

Seem hard to lint on when it's not always UB, because things like NonNull::dangling().addr() is a transmute but isn't UB.

I'd be tempted to run it on optimized mir where we can have const-folded a bunch of those cases away already, but then it'd be super inconsistent which we probably don't want either :/