diff --git a/Obtaining-the-Oodle-library-on-Linux.md b/Obtaining-the-Oodle-library-on-Linux.md deleted file mode 100644 index b83a3e2..0000000 --- a/Obtaining-the-Oodle-library-on-Linux.md +++ /dev/null @@ -1,108 +0,0 @@ -The Unreal Engine (currently v5.0.3) includes a version of the Oodle data compression library compiled for Linux (at `Engine/Source/Runtime/OodleDataCompression/Sdks/2.9.5/lib/Linux`). However, this archive was compiled for static linking, and all symbols were marked as hidden. - -To be able to create a shared library from that, follow the steps described in [StackOverflow - Keep all exported symbols when creating a shared library from a static library](https://stackoverflow.com/questions/54664759/keep-all-exported-symbols-when-creating-a-shared-library-from-a-static-library): - -**elf_unhide.py**: -```python -#!/usr/bin/python -# unhide.py - Replace hidden with default visibility on global symbols defined -# in an ELF object file - -import argparse, sys, lief -from lief.ELF import SYMBOL_BINDINGS, SYMBOL_VISIBILITY, SYMBOL_TYPES - -def warn(msg): - sys.stderr.write("WARNING: " + msg + "\n") - -def unhide(objfile_in, objfile_out = None, namedsyms=None): - if not objfile_out: - objfile_out = objfile_in - binary = lief.parse(objfile_in) - allsyms = { sym.name for sym in binary.symbols } - selectedsyms = set([]) - nasyms = { sym.name for sym in binary.symbols if \ - sym.type == SYMBOL_TYPES.NOTYPE or \ - sym.binding != SYMBOL_BINDINGS.GLOBAL or \ - sym.visibility != SYMBOL_VISIBILITY.HIDDEN } - if namedsyms: - namedsyms = set(namedsyms) - nosyms = namedsyms - allsyms - for nosym in nosyms: - warn("No symbol " + nosym + " in " + objfile_in + ": ignored") - for sym in namedsyms & nasyms: - warn("Input symbol " + sym + \ - " is not a hidden global symbol defined in " + objfile_in + \ - ": ignored") - selectedsyms = namedsyms - nosyms - else: - selectedsyms = allsyms - - selectedsyms -= nasyms - unhidden = 0; - for sym in binary.symbols: - if sym.name in selectedsyms: - sym.visibility = SYMBOL_VISIBILITY.DEFAULT - unhidden += 1 - print("Unhidden: " + sym.name) - print("{} symbols were unhidden".format(unhidden)) - binary.write(objfile_out) - -def get_args(): - parser = argparse.ArgumentParser( - description="Replace hidden with default visibility on " + \ - "global symbols defined in an ELF object file.") - parser.add_argument("ELFIN",help="ELF object file to read") - parser.add_argument("-s","--symbol",metavar="SYMBOL",action="append", - help="Unhide SYMBOL. " + \ - "If unspecified, unhide all hidden global symbols defined in ELFIN") - parser.add_argument("--symfile", - help="File of whitespace-delimited symbols to unhide") - parser.add_argument("-o","--out",metavar="ELFOUT", - help="ELF object file to write. If unspecified, rewrite ELFIN") - return parser.parse_args() - - -def main(): - args = get_args() - objfile_in = args.ELFIN - objfile_out = args.out - symlist = args.symbol - if not symlist: - symlist = [] - symfile = args.symfile - if symfile: - with open(symfile,"r") as fh: - symlist += [word for line in fh for word in line.split()] - unhide(objfile_in,objfile_out,symlist) - -main() -``` - -**elf_unhide.sh**: -```bash -#!/bin/bash - -TEMP=$(mktemp -d) - -OLD_ARCHIVE="$(realpath "$1")" -NEW_ARCHIVE="$(realpath "$2")" - -ar --output $TEMP xv $OLD_ARCHIVE - -find $TEMP -type f -exec ./elf_unhide.py --symfile symbols.txt "{}" \; - -ar rcs $NEW_ARCHIVE $TEMP/*.o - -rm -r $TEMP -echo "$(basename $NEW_ARCHIVE) made" -``` - -**Create the shared library**: - -Create the files above, as well as a file `symbols.txt`, where each line lists one symbol you want to make available for linking. -Then run: - -```shell -./elf_unhide.sh liboo2corelinux64.a liboo2corelinux64_out.a -g++ -shared -o liboo2corelinux64.so -Wl,--whole-archive ./liboo2corelinux64_out.a -Wl,--no-whole-archive -``` \ No newline at end of file diff --git a/Obtaining-the-Oodle-library.md b/Obtaining-the-Oodle-library.md new file mode 100644 index 0000000..118342a --- /dev/null +++ b/Obtaining-the-Oodle-library.md @@ -0,0 +1,20 @@ +Darktide ships with a `.dll` for Oodle 2.8, and that could be linked to for DTMT. However, that creates two problems: + +* It's only available on Windows. Linux users would have to run everything through Wine. +* DTMT needs to know the path to the library file. It's hard to make automatic detection reliable, and it's confusing for people to specify it manually. + +Therefore, linking the libraries statically is the better option, but requires additional files on both systems. +To compile for Windows (native or cross), a `oo2core_win_release.lib` file is needed. Compiling for Linux requires an `liboo2corelinux64.a` archive file instead. + +## Generating a `.lib` file + +A `.lib` file can be generated from a `.dll` file. The MSVC command line tools (from the Visual Studio installer) are required. + +1. `dumpbin /EXPORTS oo2core_win_release.dll > oo2core_win_release.def` +2. Add a line `EXPORTS` at the top of the `.def` file +2.1. (Optional) Remove unused function definitions +3. `lib /def:oo2core_win_release.def /machine:x64 /out:oo2core_win_release.lib` + +## Acquiring a `.lib` or `.a` file from Unreal Engine + +The Unreal Engine source code (currently v5.0.3) includes a version of the Oodle data compression library (at `Engine/Source/Runtime/OodleDataCompression/Sdks/2.9.5`). While this is a never version than the one Darktide uses, they are ABI compatible. \ No newline at end of file