8

The C++ modules TS provides an excellent facility for eliminating the preprocessor, improving compile times, and generally supporting much more robust, modular, code development in C++, for non-template code at least.

The underlying machinery provides control over import and export of symbols in ordinary programs.

However, there is a major problem developing libraries for two kinds of dynamic loading: startup time loading, and run time loading. This problem involves the exporting of symbols from the library, which is often discussed in terms of visibility.

Generally, not all the extern symbols of the translation units used to construct a dynamic link library should be made visible to the user. In addition, with run time loading, especially with a plugin concept, the same symbol must be exported from many concurrently loaded libraries.

On Windows the use of language extensions

 __declspec(dllexport)
 __declspec(dllimport)

attached in source code as attributes of symbols, and more recently on gcc and clang systems on unix platforms, the use of

__attribute__((visibility("default")))
__attribute__((visibility("hidden")))

are intended to support the provision and use of symbols intended to be made public by the library. Using these is complicated and messy: on Windows macros must be used to export the symbols whilst compiling the library, but import them when using it. On the unix platforms, the visibility must be set to default to both export and import the symbols, the compiler deciding itself, based on whether a definition is found or not: the compiler must be invoked with 

-fvisibility=hidden

switch. The export/import attributes are not required for static linkage, and probably should be macro'd out to an empty string. Making code and fiddling the build system so that this all works, especially considering that #includes must have the correct symbol visibility set during compilation of library translation units is very hard, the file structure required in repositories is a mess, the source code is littered with macros, and in general .. the whole thing is a disaster. Almost all open source repositories FAIL to correctly export symbols for dynamic linkage, and most programmers have no idea that dynamic library code structure (using two level namespaces) is quite different to static linkage.

An example of how to do it (hopefully correctly) can be seen here:

https://github.com/calccrypto/uint256_t

This repository used to have 2 headers and 2 implementation files, the user of the built library would see 2 headers. There are now 7 headers and 2 implementation files and the user of the built library will see 5 header files (3 with extension include to indicate they're not to be directly included).

So after that long winded explanation, the question is: will the final C++ modules specification help to solve problems with export and import of symbols for dynamic linkage? Can we expect to be able to develop for shared libraries without polluting our code with vendor specific extensions and macros?

Yttrill
  • 4,531
  • 1
  • 17
  • 28
  • Probably not, c++ doesn't much about how code is linked and knows nothing about static or dynamic linking – Alan Birtles Sep 12 '18 at 07:40
  • The compiler vendors, who have reps on WG21, know lots about it. – Yttrill Sep 12 '18 at 11:10
  • Yes I'm not suggesting that the people writing the c++ standards dont understand linking practices I'm just saying that how code is linked together is outside the scope of the c++ language. c++ is designed to be portable, if they start adding specifications of linking the language will be less portable as some platforms won't be able to implement those linking patterns. – Alan Birtles Sep 12 '18 at 12:01
  • 1
    I don't agree. How code is linked together has, historically, always been a *key* part of C++. Recall C++ invented "mangled names". It invented various new kinds of linkage (such as vague linkage) to support templates. The abstract machine of C++ was extended to include threads, and the abstract machine can be extended to support libraries. The module TS specifically deals with symbol linkage. The Standard talks about the One Definition Rule which is a linkage issue. C++ invented inline functions, a linkage issue. So C++ can definitely be extended so the abstract machine transcends programs. – Yttrill Sep 12 '18 at 20:09
  • 3
    I would say, on the contrary, the C++ committee MUST address dynamic linkage. It has to change the abstract machine so the "code" of a program can vary with time because that is the reality of the modern world. It HAS to address the specification of what is exported from a library, just as the existing Standard addresses what is exported from a translation unit. The problem you mention regarding portability is *precisely* the reason the abstract machinery must be standardised. Obviously, the implementation will depend on the vendor as usual. – Yttrill Sep 12 '18 at 20:14
  • @Yttrill: C++ *itself* doesn’t involve name mangling at all: it talks about [signatures](http://eel.is/c++draft/basic.link#10.3) and how declarations in different translation units that share a name and signature correspond. Name mangling is just a way of expressing such a signature as a flat-string symbol name of the sort commonly used in object files. – Davis Herring Sep 13 '18 at 00:02
  • @Yttrill: WG21 [is considering](http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0275r3.html) dynamic linking, but it’s currently just the loading process, not symbol visibility (which the language doesn’t have at all prior to modules). Maybe there will be a standardization of modules *as* dynamic objects, but it might be bad to force them to have the same granularity. – Davis Herring Sep 13 '18 at 00:06
  • @DavisHerring: thanks for the link! FYI a rough idea is a language extension in which a particular *wrapper module* that contains other modules can be marked as controlling visibility in a DLL. If we do that, then that module can be built as a DLL directly, and exports only the exports of the top level of the module. So syntactically a single keyword used in exactly one place should be enough, eg "library module ...." would be enough. – Yttrill Sep 13 '18 at 04:38
  • 1
    The real issue is that the current C++ abstract machine model CANNOT DEAL WITH DYNAMIC LINKAGE and that is definitely an issue for the ISO Standard involving significant changes to normative rules. With dynamic loading the ODR is nonsense: types (and thus vtables) MUST be present in each DLL that constructs polymorphic types. Function which are used internally in a DLL may be duplicates, for example constructors and destructors of polymorphic classes. At present the vendors cannot get exceptions and dynamic casts to work across DLL boundaries either, because they chose a broken model. – Yttrill Sep 13 '18 at 04:43
  • @Yttrill: The ODR is about [definitions in source](http://eel.is/c++draft/basic.def.odr#12) and has nothing to do with how many (shared) object files contain a function, a vtable, or anything else that isn’t part of the *language*. That said: yes, hilarity involving things like `RTLD_LOCAL` is a pain. – Davis Herring Sep 13 '18 at 12:26
  • @DavisHerring: I know how the Standard works. In fact I worked on the ODR myself. The issue here is not just run time loading. If you use two level namespaces (and that is the only correct way) you can load two versions of the same library safely. Now consider both throw an the same exception, which is caught several levels up. What happens? We have THREE definitions of the type now: one in each of the libraries, and one in the catching code. Clang and gcc at least used to bomb out. – Yttrill Sep 13 '18 at 22:32
  • @Yttrill Any update on this topic? with msvc, how can one export symbols in a module-based-dll, and then import these symbols in another executable? even with macros, the symbols always end up being exported (even in the exe) – Touloudou Oct 05 '21 at 14:00

0 Answers0