staticdata: Set min validation world to require world (JuliaLang#57381)

When we have implicit binding edges from literal GlobalRefs, these edges
imply a implicit minimum world age of jl_require_age (which is what all
bindings loaded from pkgimages get their definition age set to). In
JuliaLang#57318, the `cpuid_llvm` minimum world age got set to `1` rather than
`jl_require_world`, so codegen would refuse to perform the llvmcall
special codegen, since it couldn't guarantee that the binding would
actually resolve to `llvmcall` in all worlds. Fix that by adjusting
staticdata.jl to set the appropriate minworld.

That said, there are a few complicating factors here:
1. In general, most of our code doesn't handle world ranges with more
than one partition. But for example, codegen could have checked the
current world age and implicitly partitioned the binding at codegen time
(in practice just adding an appropraite error check).

2. The included test case uses a cached inference. However, in the
original issue it appeared that inlining was also creating new
references to this replaced binding, which should not have been
permitted. I have not fully investigated this behavior yet and this
might be another bug.

3. In the original issue, the specialization had its max_world
terminated a few ages past jl_require_world. I do not understand this
behavior yet.

Still, fixes JuliaLang#57318.

Author: Keno

base/loading.jl

@@ -3157,11 +3157,11 @@ This can be used to reduce package load times. Cache files are stored in
 `DEPOT_PATH[1]/compiled`. See [Module initialization and precompilation](@ref)
 for important notes.
 """
-function compilecache(pkg::PkgId, internal_stderr::IO = stderr, internal_stdout::IO = stdout; flags::Cmd=``, reasons::Union{Dict{String,Int},Nothing}=Dict{String,Int}(), loadable_exts::Union{Vector{PkgId},Nothing}=nothing)
+function compilecache(pkg::PkgId, internal_stderr::IO = stderr, internal_stdout::IO = stdout; flags::Cmd=``, cacheflags::CacheFlags=CacheFlags(), reasons::Union{Dict{String,Int},Nothing}=Dict{String,Int}(), loadable_exts::Union{Vector{PkgId},Nothing}=nothing)
     @nospecialize internal_stderr internal_stdout
     path = locate_package(pkg)
     path === nothing && throw(ArgumentError("$(repr("text/plain", pkg)) not found during precompilation"))
-    return compilecache(pkg, path, internal_stderr, internal_stdout; flags, reasons, loadable_exts)
+    return compilecache(pkg, path, internal_stderr, internal_stdout; flags, cacheflags, reasons, loadable_exts)
 end
 
 const MAX_NUM_PRECOMPILE_FILES = Ref(10)

base/staticdata.jl

@@ -70,6 +70,8 @@ function verify_method_graph(codeinst::CodeInstance, stack::Vector{CodeInstance}
     nothing
 end
 
+get_require_world() = unsafe_load(cglobal(:jl_require_world, UInt))
+
 # Test all edges relevant to a method:
 # - Visit the entire call graph, starting from edges[idx] to determine if that method is valid
 # - Implements Tarjan's SCC (strongly connected components) algorithm, simplified to remove the count variable
@@ -81,7 +83,14 @@ function verify_method(codeinst::CodeInstance, stack::Vector{CodeInstance}, visi
             return 0, world, max_valid2
         end
     end
-    local minworld::UInt, maxworld::UInt = 1, validation_world
+    # Implicitly referenced bindings in the current module do not get explicit edges.
+    # If they were invalidated, they'll be in `mwis`. If they weren't, they imply a minworld
+    # of `get_require_world`. In principle, this is only required for methods that do reference
+    # an implicit globalref. However, we already don't perform this validation for methods that
+    # don't have any (implicit or explicit) edges at all. The remaining corner case (some explicit,
+    # but no implicit edges) is rare and there would be little benefit to lower the minworld for it
+    # in any case, so we just always use `get_require_world` here.
+    local minworld::UInt, maxworld::UInt = get_require_world(), validation_world
     def = get_ci_mi(codeinst).def
     @assert def isa Method
     if haskey(visiting, codeinst)
@@ -103,9 +112,8 @@ function verify_method(codeinst::CodeInstance, stack::Vector{CodeInstance}, visi
     # verify current edges
     if isempty(callees)
         # quick return: no edges to verify (though we probably shouldn't have gotten here from WORLD_AGE_REVALIDATION_SENTINEL)
-    elseif maxworld == unsafe_load(cglobal(:jl_require_world, UInt))
+    elseif maxworld == get_require_world()
         # if no new worlds were allocated since serializing the base module, then no new validation is worth doing right now either
-        minworld = maxworld
     else
         j = 1
         while j ≤ length(callees)

test/precompile.jl

@@ -2285,4 +2285,29 @@ precompile_test_harness("Constprop CodeInstance invalidation") do load_path
     end
 end
 
+precompile_test_harness("llvmcall validation") do load_path
+    write(joinpath(load_path, "LLVMCall.jl"),
+        """
+        module LLVMCall
+        using Base: llvmcall
+        @noinline do_llvmcall() = llvmcall("ret i32 0", UInt32, Tuple{})
+        do_llvmcall2() = do_llvmcall()
+        do_llvmcall2()
+        end
+        """)
+    # Also test with --pkgimages=no
+    testcode = """
+        insert!(LOAD_PATH, 1, $(repr(load_path)))
+        insert!(DEPOT_PATH, 1, $(repr(load_path)))
+        using LLVMCall
+        LLVMCall.do_llvmcall2()
+    """
+    @test readchomp(`$(Base.julia_cmd()) --pkgimages=no -E $(testcode)`) == repr(UInt32(0))
+    # Now the regular way
+    @eval using LLVMCall
+    invokelatest() do
+        @test LLVMCall.do_llvmcall2() == UInt32(0)
+    end
+end
+
 finish_precompile_test!()