Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standard Library Header Units and Modules - tracking issue #1694

Open
StephanTLavavej opened this issue Feb 24, 2021 · 1 comment
Open

Standard Library Header Units and Modules - tracking issue #1694

StephanTLavavej opened this issue Feb 24, 2021 · 1 comment
Labels
compiler Compiler work involved modules C++23 modules, C++20 header units

Comments

@StephanTLavavej
Copy link
Member

StephanTLavavej commented Feb 24, 2021

📄 Documentation

🚧 Status

  • Standard Library Header Units are ready for production use in VS 2019 16.11, for MSVC only.
  • Standard Library Modules are ready for production use in VS 2022 17.5, for MSVC only.

We've reported the following compiler bugs; this issue is for tracking the status of these compiler bugs and removing workarounds/adding test coverage as compiler fixes are released.

As /clr C++20 support is a work in progress, we are not yet testing Standard Library Header Units under /clr.

❌ Clang

🐛 EDG (IntelliSense Front-End)

  • EDG is currently investigating test failures. STL test coverage is not yet enabled. See Report EDG bugs #1621.

🪲 Active, UCRT

🐞 Active, C1XX (MSVC Front-End)

❔ Possible Improvements (C1XX/MSBuild)

  • VSO-1330581 "Throughput for Standard Library Header Units"
  • VSO-1469758 "Standard Library Header Units: Possible IFC size reductions?"
  • VSO-1568412 "Standard Library Modules: Enhanced diagnostics for needing import std.compat;, std::printf, or std::cout"
  • VSO-1621267 "Standard Library Modules: Enhanced diagnostics for missing macros"
  • VSO-1922468 "Standard Library Modules: Helpful diagnostic when import std; is seen, but std.ifc/std.obj need to be built"
  • VSO-1922469 "Support C++20 builds of import std; and import std.compat;"
  • VSO-1987212 "[C++ Modules DT perf] STL modules are always scanned in designtime regardless of their usage in code"

🛠️ Upcoming Improvements (C1XX)

🩹 Permanent Workarounds

  • VSO-1308657 "Standard Library Header Units: std::projected::operator*() error LNK2019: unresolved external symbol"
    • Perma-workaround: provide a definition that calls abort().

😻 Fixed, Workaround Removed

  • Fixed in 16.8:
    • DevCom-1160043 VSO-1168062 "Standard Library Header Units: Bogus error C2131 when using compare functions at compile-time"
    • DevCom-1160145 VSO-1180193 "Standard Library Header Units: error LNK2019 'unresolved external symbol' when a function is declared in one header and defined in another"
  • Fixed in 16.9:
    • DevCom-1159869 VSO-1180127 "Standard Library Header Units: Bogus warning C4373 about virtual function overriding"
    • DevCom-1159995 VSO-1180128 "Standard Library Header Units: UDLs don't compile, depending on the order of imports"
    • DevCom-1160260 VSO-1180207 "Standard Library Header Units: Bogus error C2752: 'std::pointer_traits<_Voidptr>': more than one partial specialization matches the template argument list"
    • DevCom-1161187 VSO-1181303 "Standard Library Header Units: Bogus error C2248 'cannot access private member'"
    • DevCom-1162644 VSO-1181305 "Standard Library Header Units: __pragma(warning(disable : 4996)) doesn't take effect"
    • DevCom-1224512 VSO-1236022 "Standard Library Header Units: Bogus error C2027: use of undefined type 'std::char_traits'"
    • VSO-1236034 "Standard Library Header Units: bogus error LNK2005: std::_Yarn<char>::operator=(char const *) already defined"
    • VSO-1236047 "Standard Library Header Units: std::optional, c1xx!FindConversionFunctions() assertion failed: PF_ISSET(ParsingClassTemplateDefn)"
    • VSO-1237804 "Standard Library Header Units: std::variant, c1xx!Module::InterfaceReader::materialize_function() assertion failed"
    • VSO-1237145 "Standard Library Header Units: bogus error C7599: 'std::ranges::operator ==': a trailing requires clause is only allowed on a templated function"
  • Fixed and implemented in 16.10:
    • VSO-1271718 "Standard Library Header Units ICE with C++20 chrono"
    • VSO-1180134 DevCom-1160041 "Standard Library Header Units: deprecated attributes aren't imported"
    • VSO-1273005 "[Modules] c1xx does not fully resolve templates imported from IFC in the context of trailing requirements"
    • VSO-1284279 "Standard Library Header Units: memory_resource ICE: Mapping already has an entry defined at this slot"
    • VSO-1287222 "[modules] transitive header unit imports do not import macros"
    • VSO-1287925 "/headerUnit:angle ICEs"
    • VSO-1307828 "Standard Library Header Units: ICE with queue's friend bool operator=="
    • VSO-1309454 "Standard Library Header Units: CTAD fails when deduction guides are duplicated across header units"
    • VSO-1314139 "Cannot build header units for STL headers with 'Translate Includes to Imports'"
    • VSO-1232145 "EDG ICEs when consuming Standard Library Header Units"
    • Implemented /headerUnit:angle vector=vector.ifc
    • Implemented support for header-units.json via /sourceDependencies:directives and /translateInclude
    • Implemented /exportHeader /headerName:angle algorithm type_traits vector
  • Fixed in 16.11:
    • VSO-1330589 "Standard Library Header Units: Bogus error C2131 with _Nontrivial_dummy_type's constructor"
    • VSO-1330591 "Standard Library Header Units: Bogus error C2676 with type_info's equality operator"
    • DevCom-1440183 VSO-1337327 "Constexpr Function Involving String Does Not Compile When Consuming Modules"
  • Fixed in 17.0:
  • Fixed in 17.1:
    • DevCom-1511903 VSO-1384883 "Standard Library Header Units: Bogus error C2440 when convertible_to constrains a constructor"
    • VSO-1409853 "Standard Library Header Units: <ranges> compiler assertion: previous_element == tokenInputStack.TopOfStack(), aliastemplates.cpp 1010"
    • VSO-1433873 "Standard Library Header Units: Adding template <int = 0> to vformat() emits warnings C4265 and C4365"
  • Fixed in 17.2:
    • VSO-1464637 "Standard Library Header Units: #pragma warning doesn't always suppress warnings in templates"
    • VSO-1466711 "/scanDependencies generates incorrect "logical-name" in JSON output"
    • VSO-1471374 "Standard Library Header Units: Deduplication emits fatal error C1116: unrecoverable error importing module, with <concepts>"
    • VSO-1471376 "Standard Library Header Units: Confusing diagnostic for fatal error C1116: unrecoverable error importing module ''"
    • VSO-1471382 "Standard Library Header Units: Deduplication emits error C2672: 'count_if': no matching overloaded function found"
  • Fixed in 17.3:
    • VSO-1496074 "Standard Library Modules: C++20 chrono::days emits bogus error C2131"
    • VSO-1496084 "Standard Library Header Units: Splitting <chrono> emits fatal error C1116: unrecoverable error importing header unit '<chrono>'"
    • VSO-1496493 "Standard Library Modules: Implementing std.ixx emits fatal error C1011: cannot locate standard module interface"
    • VSO-1497379 "Standard Library Modules: extern "C" function declarations taking extern "C" types emit bogus error LNK2019: unresolved external symbol"
    • VSO-1504872 "Standard Library Modules: extern "C" function mentioning entity with module linkage is being marked with module linkage"
    • VSO-1526632 "Standard Library Header Units: Hidden friend operators cause ICE in Module::InterfaceReader::materialize_function<Module::helpers::MemberBuilder>()"
    • VSO-1513874 "Changing construct_at SFINAE causes Modules bogus error C2660: 'operator new': function does not take 3 arguments"
    • VSO-1533852 "Standard Library Modules: The Mega-Bug"
      • <codecvt>: fatal error C1001: Internal compiler error.
      • <any>: error C2679: binary '!=': no operator found which takes a right-hand operand of type 'const type_info' (or there is no acceptable conversion)
      • <memory>: error C2679: binary '==': no operator found which takes a right-hand operand of type 'const type_info' (or there is no acceptable conversion)
      • <functional>: error C2027: use of undefined type 'type_info'
      • <typeinfo>: error C2872: 'type_info': ambiguous symbol
      • <typeindex>: error C2440: 'initializing': cannot convert from 'initializer list' to 'std::type_index'
      • <regex>: Assertion failed: Nerrors > 0, file D:\msvc\src\vctools\Compiler\CxxFE\sl\p1\c\trees.c
    • VSO-1540220 "Standard Library Modules: /d1ifcInlineFunctions asserts (bits::rep(typeEncoding.basicEncoding_) & 0x0000000F) != 0 for printf(_Format, ...)"
    • VSO-1543660 "Standard Library Header Units: <expected> ICEs with Assertion failed: IsInClassDefn()"
    • VSO-1541639 "Standard Library Modules: error C3083: '_Iterator_base12': the symbol to the left of a '::' must be a type"
    • VSO-1509503 "Standard Library Header Units: Deduplication interferes with ADL for global operator&"
  • Fixed in 17.4:
  • Fixed in 17.5:
  • Fixed in 17.6:
  • Fixed in 17.7:
    • DevCom-10283892 VSO-1749846 "C++ modules std::expected error C2280 attempting to reference a deleted function"
    • DevCom-10404496 "[C++][Modules] Using std::views::iota in member function in module causes ICE"
  • Fixed in 17.8:
    • DevCom-10404498 VSO-1843798 "[C++][Modules] Using std::views::repeat in member functions gives bogus error about lack of compare header"
    • DevCom-10324326 VSO-1780122 "'...': there are no parameter packs available to expand module"
    • VSO-1475786 "Standard Library Header Units: Deduplication emits bogus errors related to numeric_limits"
      • Fixed at some point between VS 2022 17.1 and 17.8.
    • VSO-1593161 "Standard Library Modules: ctype<char>::table_size should be a compile-time constant"
  • Fixed in 17.9:
  • Fixed in 17.10:
    • VSO-1592395 "Standard Library Modules: <chrono> year_month_weekday and year_month_weekday_last emit bogus error C2131: expression did not evaluate to a constant"
    • VSO-1555263 "Standard Library Header Units: Local array with day{31} initializers emits 'not yet implemented' internal compiler error"
    • VSO-1913621 "extern "C++" doesn't avoid all problems when mixing includes and imports"
    • VSO-1914077 "Modules: extern "C++" interferes with ADL"
    • DevCom-10452704 VSO-1880921 "[Feedback][std:c++latest] C++ compile error when both including a header and calling std::filesystem::path::generic_string() in a module unit"
    • DevCom-10542001 VSO-1930627 "error C1116 unrecoverable error importing module 'std' when constructing and exporting a std::chrono::zoned_time object"
      • Duplicate: VSO-1951431 "Standard Library Modules: chrono::time_point formatting emits bogus error C3861: '_Fill_tm': identifier not found"
      • Duplicate: VSO-1951433 "Standard Library Modules: utc_clock::now() emits fatal error C1116: unrecoverable error importing module 'std'."
    • VSO-1538698 "Better handling for non-exported friend function declarations"
    • VSO-1975579 "Standard Library Modules: fatal error C1116: unrecoverable error importing module 'std'. Specialization of 'std::invoke_result_t' with arguments '_Fn, _Ty...'"
    • DevCom-10519854 VSO-1919553 "Using std::views::zip with std::views::iota in C++20 module results in C1001 Internal compiler error"

🏗️ Building Deduplicated Header Units

Here's the part of our Python-powered test harness that builds "deduplicated" header units by topologically sorting them:

# Compiler options, common to both scanning dependencies and building header units.
clOptions = ['/exportHeader', '/headerName:angle', '/translateInclude', '/Fo', '/MP']
# Store the list of headers to build.
allHeaders = getAllHeaders(os.path.join(litConfig.cxx_headers, 'header-units.json'))
# Generate JSON files that record how these headers depend on one another.
if noisyProgress:
print('Scanning dependencies...')
cmd = [test.cxx, *test.flags, *test.compileFlags, *clOptions, '/scanDependencies', '.\\', *allHeaders]
yield TestStep(cmd, shared.execDir, shared.env, False)
# The JSON files also record what object files will be produced.
# IFC filenames and OBJ filenames follow different patterns. For example:
# <filesystem> produces filesystem.ifc and filesystem.obj
# <xbit_ops.h> produces xbit_ops.h.ifc and xbit_ops.obj
# We can easily synthesize IFC filenames, but it's easier to get the OBJ filenames from the JSON files.
# This dictionary powers the topological sort.
# Key: Header name, e.g. 'bitset'.
# Value: List of dependencies that remain to be built, e.g. ['iosfwd', 'limits', 'xstring'].
remainingDependencies = {}
# Read the JSON files, storing the results in objFilenames and remainingDependencies.
for hdr in allHeaders:
with open(os.path.join(outputDir, f'{hdr}.module.json')) as file:
jsonObject = json.load(file)
objFilenames.append(jsonObject['rules'][0]['primary-output'])
remainingDependencies[hdr] = [req['logical-name'] for req in jsonObject['rules'][0]['requires']]
# Build header units in topologically sorted order.
while len(remainingDependencies) > 0:
# When a header has no remaining dependencies, it is ready to be built.
readyToBuild = [hdr for hdr, dep in remainingDependencies.items() if len(dep) == 0]
# Each layer can be built in parallel.
if noisyProgress:
print('Building deduplicated header units:', ' '.join(readyToBuild))
cmd = [test.cxx, *test.flags, *test.compileFlags, *clOptions, *consumeBuiltHeaderUnits, *readyToBuild]
yield TestStep(cmd, shared.execDir, shared.env, False)
# Update remainingDependencies by doing two things.
# hdr, dep is the current key-value pair.
# First, keep `if len(dep) > 0`. (Otherwise, we just built hdr.)
# Second, filter dep, eliminating anything that appears in readyToBuild. (If we're left with
# an empty list, then hdr will be ready to build in the next iteration.)
remainingDependencies = {
hdr: [d for d in dep if d not in readyToBuild]
for hdr, dep in remainingDependencies.items() if len(dep) > 0
}
# Add compiler options to consume the header units that we just built.
for hdr in readyToBuild:
consumeBuiltHeaderUnits += ['/headerUnit:angle', f'{hdr}={hdr}.ifc']

With a bit of work, this can be extracted into a standalone Python script. (With significantly more work, it could be converted into CMake.)

@CodePagesNet
Copy link

Great to see the note that std lib modules are ready for production use (as of VS2022 17.5, MSVC only). Does that mean that the statement shown here in the documentation about not using preview releases in production code, should be removed?
https://learn.microsoft.com/en-us/cpp/cpp/tutorial-import-stl-named-module?view=msvc-170#prerequisites

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler Compiler work involved modules C++23 modules, C++20 header units
Projects
None yet
Development

No branches or pull requests

2 participants