build: fix various shared library build issues
Node.js unofficially supports a shared library variant where the
main node executable is a thin wrapper around node.dll/libnode.so.
The key benefit of this is to support embedding Node.js in other
applications.
Since Node.js 12 there have been a number of issues preventing the
shared library build from working correctly, primarily on Windows:
* A number of functions used executables such as `mksnapshot` are
not exported from `libnode.dll` using a `NODE_EXTERN` attribute
* A dependency on the `Winmm` system library is missing
* Incorrect defines on executable targets leads to `node.exe`
claiming to export a number of functions that are actually in
`libnode.dll`
* Because `node.exe` attempts to export symbols, `node.lib` gets
generated causing native extensions to try to link against
`node.exe` not `libnode.dll`.
* Similarly, because `node.dll` was renamed to `libnode.dll`,
native extensions don't know to look for `libnode.lib` rather
than `node.lib`.
* On macOS an RPATH is added to find `libnode.dylib` relative to
`node` in the same folder. This works fine from the
`out/Release` folder but not from an installed prefix, where
`node` will be in `bin/` and `libnode.dylib` will be in `lib/`.
* Similarly on Linux, no RPATH is added so LD_LIBRARY_PATH needs
setting correctly for `bin/node` to find `lib/libnode.so`.
For the `libnode.lib` vs `node.lib` issue there are two possible
options:
1. Ensure `node.lib` from `node.exe` does not get generated, and
instead copy `libnode.lib` to `node.lib`. This means addons
compiled when referencing the correct `node.lib` file will
correctly depend on `libnode.dll`. The down side is that
native addons compiled with stock Node.js will still try to
resolve symbols against node.exe rather than libnode.dll.
2. After building `libnode.dll`, dump the exports using `dumpbin`,
and process this to generate a `node.def` file to be linked into
`node.exe` with the `/DEF:node.def` flag. The export entries
in `node.def` will all read
```
my_symbol=libnode.my_symbol
```
so that `node.exe` will redirect all exported symbols back to
`libnode.dll`. This has the benefit that addons compiled with
stock Node.js will load correctly into `node.exe` from a shared
library build, but means that every embedding executable also
needs to perform this same trick.
I went with the first option as it is the cleaner of the two
solutions in my opinion. Projects wishing to generate a shared
library variant of Node.js can now, for example,
```
.\vcbuild dll package vs
```
to generate a full node installation including `libnode.dll`,
`Release\node.lib`, and all the necessary headers. Native addons
can then be built against the shared library build easily by
specifying the correct `nodedir` option.
For example
```
>npx node-gyp configure --nodedir
C:\Users\User\node\Release\node-v18.0.0-win-x64
...
>npx node-gyp build
...
>dumpbin /dependents build\Release\binding.node
Microsoft (R) COFF/PE Dumper Version 14.29.30136.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file build\Release\binding.node
File Type: DLL
Image has the following dependencies:
KERNEL32.dll
libnode.dll
VCRUNTIME140.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
...
```
PR-URL: https://github.com/nodejs/node/pull/41850
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
2022-02-04 10:12:57 +00:00
|
|
|
#include <Windows.h>
|
|
|
|
#include <algorithm>
|
|
|
|
#include <cstdint>
|
|
|
|
#include <fstream>
|
|
|
|
#include <iostream>
|
|
|
|
#include <memory>
|
|
|
|
#include <vector>
|
|
|
|
|
|
|
|
// This executable takes a Windows DLL and uses it to generate
|
|
|
|
// a module-definition file [1] which forwards all the exported
|
|
|
|
// symbols from the DLL and redirects them back to the DLL.
|
|
|
|
// This allows node.exe to export the same symbols as libnode.dll
|
|
|
|
// when building Node.js as a shared library. This is conceptually
|
2024-10-15 13:59:49 +00:00
|
|
|
// similar to the create_expfile.sh script used on AIX.
|
build: fix various shared library build issues
Node.js unofficially supports a shared library variant where the
main node executable is a thin wrapper around node.dll/libnode.so.
The key benefit of this is to support embedding Node.js in other
applications.
Since Node.js 12 there have been a number of issues preventing the
shared library build from working correctly, primarily on Windows:
* A number of functions used executables such as `mksnapshot` are
not exported from `libnode.dll` using a `NODE_EXTERN` attribute
* A dependency on the `Winmm` system library is missing
* Incorrect defines on executable targets leads to `node.exe`
claiming to export a number of functions that are actually in
`libnode.dll`
* Because `node.exe` attempts to export symbols, `node.lib` gets
generated causing native extensions to try to link against
`node.exe` not `libnode.dll`.
* Similarly, because `node.dll` was renamed to `libnode.dll`,
native extensions don't know to look for `libnode.lib` rather
than `node.lib`.
* On macOS an RPATH is added to find `libnode.dylib` relative to
`node` in the same folder. This works fine from the
`out/Release` folder but not from an installed prefix, where
`node` will be in `bin/` and `libnode.dylib` will be in `lib/`.
* Similarly on Linux, no RPATH is added so LD_LIBRARY_PATH needs
setting correctly for `bin/node` to find `lib/libnode.so`.
For the `libnode.lib` vs `node.lib` issue there are two possible
options:
1. Ensure `node.lib` from `node.exe` does not get generated, and
instead copy `libnode.lib` to `node.lib`. This means addons
compiled when referencing the correct `node.lib` file will
correctly depend on `libnode.dll`. The down side is that
native addons compiled with stock Node.js will still try to
resolve symbols against node.exe rather than libnode.dll.
2. After building `libnode.dll`, dump the exports using `dumpbin`,
and process this to generate a `node.def` file to be linked into
`node.exe` with the `/DEF:node.def` flag. The export entries
in `node.def` will all read
```
my_symbol=libnode.my_symbol
```
so that `node.exe` will redirect all exported symbols back to
`libnode.dll`. This has the benefit that addons compiled with
stock Node.js will load correctly into `node.exe` from a shared
library build, but means that every embedding executable also
needs to perform this same trick.
I went with the first option as it is the cleaner of the two
solutions in my opinion. Projects wishing to generate a shared
library variant of Node.js can now, for example,
```
.\vcbuild dll package vs
```
to generate a full node installation including `libnode.dll`,
`Release\node.lib`, and all the necessary headers. Native addons
can then be built against the shared library build easily by
specifying the correct `nodedir` option.
For example
```
>npx node-gyp configure --nodedir
C:\Users\User\node\Release\node-v18.0.0-win-x64
...
>npx node-gyp build
...
>dumpbin /dependents build\Release\binding.node
Microsoft (R) COFF/PE Dumper Version 14.29.30136.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file build\Release\binding.node
File Type: DLL
Image has the following dependencies:
KERNEL32.dll
libnode.dll
VCRUNTIME140.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
...
```
PR-URL: https://github.com/nodejs/node/pull/41850
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
2022-02-04 10:12:57 +00:00
|
|
|
//
|
|
|
|
// Generating this .def file requires parsing data out of the
|
|
|
|
// PE32/PE32+ file format. Helper structs are defined in <Windows.h>
|
|
|
|
// hence why this is an executable and not a script. See [2] for
|
|
|
|
// details on the PE format.
|
|
|
|
//
|
|
|
|
// [1]: https://docs.microsoft.com/en-us/cpp/build/reference/module-definition-dot-def-files
|
|
|
|
// [2]: https://docs.microsoft.com/en-us/windows/win32/debug/pe-format
|
|
|
|
|
|
|
|
// The PE32 format encodes pointers as Relative Virtual Addresses
|
|
|
|
// which are 32 bit offsets from the start of the image. This helper
|
|
|
|
// class hides the mess of the pointer arithmetic
|
|
|
|
struct RelativeAddress {
|
|
|
|
uintptr_t root;
|
|
|
|
uintptr_t offset = 0;
|
|
|
|
|
|
|
|
RelativeAddress(HMODULE handle) noexcept
|
|
|
|
: root(reinterpret_cast<uintptr_t>(handle)) {}
|
|
|
|
|
|
|
|
RelativeAddress(HMODULE handle, uintptr_t offset) noexcept
|
|
|
|
: root(reinterpret_cast<uintptr_t>(handle)), offset(offset) {}
|
|
|
|
|
|
|
|
RelativeAddress(uintptr_t root, uintptr_t offset) noexcept
|
|
|
|
: root(root), offset(offset) {}
|
|
|
|
|
|
|
|
template <typename T>
|
|
|
|
const T* AsPtrTo() const noexcept {
|
|
|
|
return reinterpret_cast<const T*>(root + offset);
|
|
|
|
}
|
|
|
|
|
|
|
|
template <typename T>
|
|
|
|
T Read() const noexcept {
|
|
|
|
return *AsPtrTo<T>();
|
|
|
|
}
|
|
|
|
|
|
|
|
RelativeAddress AtOffset(uintptr_t amount) const noexcept {
|
|
|
|
return {root, offset + amount};
|
|
|
|
}
|
|
|
|
|
|
|
|
RelativeAddress operator+(uintptr_t amount) const noexcept {
|
|
|
|
return {root, offset + amount};
|
|
|
|
}
|
|
|
|
|
|
|
|
RelativeAddress ReadRelativeAddress() const noexcept {
|
|
|
|
return {root, Read<uint32_t>()};
|
|
|
|
}
|
|
|
|
};
|
|
|
|
|
|
|
|
// A wrapper around a dynamically loaded Windows DLL. This steps through the
|
|
|
|
// PE file structure to find the export directory and pulls out a list of
|
|
|
|
// all the exported symbol names.
|
|
|
|
struct Library {
|
|
|
|
HMODULE library;
|
|
|
|
std::string libraryName;
|
|
|
|
std::vector<std::string> exportedSymbols;
|
|
|
|
|
|
|
|
Library(HMODULE library) : library(library) {
|
|
|
|
auto libnode = RelativeAddress(library);
|
|
|
|
|
|
|
|
// At relative offset 0x3C is a 32 bit offset to the COFF signature, 4 bytes
|
|
|
|
// after that is the start of the COFF header.
|
|
|
|
auto coffHeaderPtr =
|
|
|
|
libnode.AtOffset(0x3C).ReadRelativeAddress().AtOffset(4);
|
|
|
|
auto coffHeader = coffHeaderPtr.AsPtrTo<IMAGE_FILE_HEADER>();
|
|
|
|
|
|
|
|
// After the coff header is the Optional Header (which is not optional). We
|
|
|
|
// don't know what type of optional header we have without examining the
|
|
|
|
// magic number
|
|
|
|
auto optionalHeaderPtr = coffHeaderPtr.AtOffset(sizeof(IMAGE_FILE_HEADER));
|
|
|
|
auto optionalHeader = optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER>();
|
|
|
|
|
|
|
|
auto exportDirectory =
|
|
|
|
(optionalHeader->Magic == 0x20b) ? optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER64>()
|
|
|
|
->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT]
|
|
|
|
: optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER32>()
|
|
|
|
->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT];
|
|
|
|
|
|
|
|
auto exportTable = libnode.AtOffset(exportDirectory.VirtualAddress)
|
|
|
|
.AsPtrTo<IMAGE_EXPORT_DIRECTORY>();
|
|
|
|
|
|
|
|
// This is the name of the library without the suffix, this is more robust
|
|
|
|
// than parsing the filename as this is what the linker uses.
|
|
|
|
libraryName = libnode.AtOffset(exportTable->Name).AsPtrTo<char>();
|
|
|
|
libraryName = libraryName.substr(0, libraryName.size() - 4);
|
|
|
|
|
|
|
|
const uint32_t* functionNameTable =
|
|
|
|
libnode.AtOffset(exportTable->AddressOfNames).AsPtrTo<uint32_t>();
|
|
|
|
|
|
|
|
// Given an RVA, parse it as a std::string. The resulting string is empty
|
|
|
|
// if the symbol does not have a name (i.e. it is ordinal only).
|
|
|
|
auto nameRvaToName = [&](uint32_t rva) -> std::string {
|
|
|
|
auto namePtr = libnode.AtOffset(rva).AsPtrTo<char>();
|
|
|
|
if (namePtr == nullptr) return {};
|
|
|
|
return {namePtr};
|
|
|
|
};
|
|
|
|
std::transform(functionNameTable,
|
|
|
|
functionNameTable + exportTable->NumberOfNames,
|
|
|
|
std::back_inserter(exportedSymbols),
|
|
|
|
nameRvaToName);
|
|
|
|
}
|
|
|
|
|
|
|
|
~Library() { FreeLibrary(library); }
|
|
|
|
};
|
|
|
|
|
|
|
|
bool IsPageExecutable(void* address) {
|
|
|
|
MEMORY_BASIC_INFORMATION memoryInformation;
|
|
|
|
size_t rc = VirtualQuery(
|
|
|
|
address, &memoryInformation, sizeof(MEMORY_BASIC_INFORMATION));
|
|
|
|
|
|
|
|
if (rc != 0 && memoryInformation.Protect != 0) {
|
|
|
|
return memoryInformation.Protect == PAGE_EXECUTE ||
|
|
|
|
memoryInformation.Protect == PAGE_EXECUTE_READ ||
|
|
|
|
memoryInformation.Protect == PAGE_EXECUTE_READWRITE ||
|
|
|
|
memoryInformation.Protect == PAGE_EXECUTE_WRITECOPY;
|
|
|
|
}
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
Library LoadLibraryOrExit(const char* dllPath) {
|
|
|
|
auto library = LoadLibrary(dllPath);
|
|
|
|
if (library != nullptr) return library;
|
|
|
|
|
|
|
|
auto error = GetLastError();
|
|
|
|
std::cerr << "ERROR: Failed to load " << dllPath << std::endl;
|
|
|
|
LPCSTR buffer = nullptr;
|
|
|
|
auto rc = FormatMessageA(
|
|
|
|
FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM,
|
|
|
|
nullptr,
|
|
|
|
error,
|
|
|
|
LANG_USER_DEFAULT,
|
|
|
|
(LPSTR)&buffer,
|
|
|
|
0,
|
|
|
|
nullptr);
|
|
|
|
if (rc != 0) {
|
|
|
|
std::cerr << buffer << std::endl;
|
|
|
|
LocalFree((HLOCAL)buffer);
|
|
|
|
}
|
|
|
|
exit(1);
|
|
|
|
}
|
|
|
|
|
|
|
|
int main(int argc, char** argv) {
|
|
|
|
if (argc != 3) {
|
|
|
|
std::cerr << "Usage: " << argv[0]
|
|
|
|
<< " path\\to\\libnode.dll path\\to\\node.def" << std::endl;
|
|
|
|
return 1;
|
|
|
|
}
|
|
|
|
|
|
|
|
auto libnode = LoadLibraryOrExit(argv[1]);
|
|
|
|
auto defFile = std::ofstream(argv[2]);
|
|
|
|
defFile << "EXPORTS" << std::endl;
|
|
|
|
|
|
|
|
for (const std::string& functionName : libnode.exportedSymbols) {
|
|
|
|
// If a symbol doesn't have a name then it has been exported as an
|
|
|
|
// ordinal only. We assume that only named symbols are exported.
|
|
|
|
if (functionName.empty()) continue;
|
|
|
|
|
|
|
|
// Every name in the exported symbols table should be resolvable
|
|
|
|
// to an address because we have actually loaded the library into
|
|
|
|
// our address space.
|
|
|
|
auto address = GetProcAddress(libnode.library, functionName.c_str());
|
|
|
|
if (address == nullptr) {
|
|
|
|
std::cerr << "WARNING: " << functionName
|
|
|
|
<< " appears in export table but is not a valid symbol"
|
|
|
|
<< std::endl;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
defFile << " " << functionName << " = " << libnode.libraryName << "."
|
|
|
|
<< functionName;
|
|
|
|
|
|
|
|
// Nothing distinguishes exported global data from exported functions
|
|
|
|
// with C linkage. If we do not specify the DATA keyword for such symbols
|
|
|
|
// then consumers of the .def file will get a linker error. This manifests
|
|
|
|
// as nodedbg_ symbols not being found. We assert that if the symbol is in
|
|
|
|
// an executable page in this process then it is a function, not data.
|
|
|
|
if (!IsPageExecutable(address)) {
|
|
|
|
defFile << " DATA";
|
|
|
|
}
|
|
|
|
defFile << std::endl;
|
|
|
|
}
|
|
|
|
|
|
|
|
return 0;
|
|
|
|
}
|