Commit graph

141 commits

Author SHA1 Message Date
bcbc005df7
Implement more texture formats
The second compression method found in the game's code seems to be Zlib,
but it doesn't seem to be used in the game files. What does get used is
a compression type of `0`, which appears to be uncompressed data.

For DDS formats, all the ones that are currently used by in the game
files can be emitted as is. Though for some of them, other tools might
not be able to display them.
2025-02-19 11:14:04 +01:00
233389ebb1
sdk: Implement decompiling streamed mipmaps
For now, we only extract the largest mipmap.
2025-02-19 11:13:59 +01:00
30b9a93fa3
Reverse DDSImage::load
Decompiling the game binary shows a rather elaborate algorithm to load
DDS images from binary. Though comparing it to Microsoft's documentation
on DDS, most of it seems to be pretty standard handling.

However, we don't actually need all of it. The part about calculating
pitch and reading blocks only accesses a subset of the `ImageFormat`
struct, so we can strip our implementation to just that.
2025-02-19 11:12:11 +01:00
84cb6ff985
sdk: Add decompiled SJSON texture file
In addition to the actual image file, also write a `.texture` engine
file.
2025-02-19 11:12:09 +01:00
0e2f0d4409
sdk: Add dictionary group for texture categories 2025-02-19 11:12:08 +01:00
ea7886b08f
feat(sdk): Implement partial texture decompilation 2025-02-19 11:12:05 +01:00
a3583b4485
fix(deps): update rust crate thiserror to v2
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
2025-02-19 11:02:02 +01:00
b219e20f3a
chore(deps): update rust crate bindgen to 0.71.0
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
2024-12-06 20:32:37 +00:00
a2bbab1398
Update dependencies
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
2024-08-21 14:33:39 +02:00
d7fa80f471
Merge pull request 'Add tests for hash inversion' (#185) from feat/murmur-tests into master
All checks were successful
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
Reviewed-on: https://git.sclu1034.dev///bitsquid_dt/dtmt/pulls/185
2024-08-14 12:07:31 +00:00
2a1d8d815f
Add tests for hash inversion
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
build/msvc Build for the target platform: msvc
Just a quick round trip test, and an additional assert to demonstrate
that byte order does matter.
2024-08-14 09:22:24 +02:00
7fa08c2efd
dtmt: Implement listing bundle database contents 2024-07-28 22:03:43 +02:00
dbf060032b
sdk: Implement bundle database resource hashes
Algorithm reverse engineered by WhiteGoat.
2024-07-28 14:46:10 +02:00
3a6e954f9a
sdk: Refactor murmur modules and add IdString32
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
build/msvc Build for the target platform: msvc
2024-07-19 11:30:09 +02:00
f1f9a818cc
sdk: Allow any byte stream for hashing dictionary entries
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
build/msvc Build for the target platform: msvc
2024-07-19 09:48:25 +02:00
c997489e18
Add some doc comments 2024-07-19 09:48:23 +02:00
08219f05ba
sdk: Fix reading strings
Fatshark has a few weird string fields, where they provide a length
field, but then sometimes write a shorter, NUL-terminated string into
that same field and adding padding up to the "advertised" length.
To properly read those strings, we can't rely on just the length field
anymore, but need to check for a NUL, too.
2024-07-19 09:48:21 +02:00
74a7aaa6e5
dtmt-shared: Write log lines to stderr
Ideally, I would prefer the usual split per logging level, but that
seems to be somewhat complex with `tracing_subscriber`, so this simply
switches everything over to stderr, so that some of the experiment
commands can write results to stdout.
2024-07-19 09:48:15 +02:00
95fc6c160b
dtmt: Implement name overrides
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
For most of the game files, we don't know the actual name, only the hash
of that name. To still allow building bundles that contain files with
that name (e.g. to override a game file with a custom one), there needs
to be a way to tell DTMT to name a file such that its hash is the same
as the one in the game.
The initial idea was to just expect the file name on disk to be the
hash, but that wouldn't allow for arbitrary folder structures anymore.

So instead, there is now a new, optional setting in `dtmt.cfg`, where
the modder can map a file path to an override name.
2024-07-18 09:50:48 +02:00
b7e26eee57
refactor(sdk): Split BundleFileType into its own file 2024-07-17 11:14:22 +02:00
a47167b735
Fix printing hashes with leading zeroes
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
Closes #179
2024-07-12 15:58:39 +02:00
96a7eeb1e0
Implement faster hex string parsing 2024-07-09 16:16:48 +02:00
189c3199a0
Update bitflags 2024-07-09 16:16:42 +02:00
3546bc8faa
Update bindgen 2024-05-15 21:57:45 +02:00
ae30499a49
Remove unused dependency 2024-05-15 19:19:54 +02:00
647cb1b8bd
Update fastrand 2024-05-15 19:16:34 +02:00
21df6cfc5c
Update reqwest 2024-05-15 18:52:58 +02:00
6030917ade
Update steamlocate
The actual update already happened, but `cargo oudated` cannot handle
the suffix, so we must update the `Cargo.toml` as well.
2024-05-15 16:37:28 +02:00
86ed5c327f
Update crates
All checks were successful
build/msvc Build for the target platform: msvc
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
`steamlocate` changed its API again.
`shlex` deprecated `quote`, but that will be addressed later.
2024-05-15 13:41:44 +02:00
cfee6d9121
Fix clippy lints 2024-05-15 13:41:42 +02:00
4ad30a8a12
Update color-eyre
We no longer need to patch `ansi-parser`.
2024-05-13 16:31:58 +02:00
6f848bb837
dtmm: Implement NXM URI handler
Closes #31.
2023-11-30 17:50:07 +01:00
5bcc4b8c70
Merge branch 'master' into feat/nexus-file-version
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
* master:
  dtmm: Fix writing Nexus image to disk
  dtmm: Fix importing from `.mod` file
  ci: Combine Dockerfiles into multi-stage build
  ci: Rework MSVC image building
  ci: Fix base pipeline
  ci: Improve caching setup for image building
  ci: Create build artifacts for commits on master
  ci: Fix undefined variable
  Add changelog entry
  dtmm: Fetch mod image from Nexus
2023-11-29 15:42:00 +01:00
a0fe5d3f81
nexusmods: Fix File type
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
2023-11-29 15:37:37 +01:00
246564d00f
Fetch file version from Nexus
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
When importing an archive file downloaded from Nexus, the file name does
include a version field. But, presumably for compatibility reasons,
Nexus replaces special characters with `-`, so that this field doesn't
match common schemes like `1.0.0`.
So instead we use the also included update timestamp to find the
corresponding file info from Nexus and use the version data from that.

Closes #131.
2023-11-24 13:54:19 +01:00
0b3c92d19e
dtmm: Fetch mod image from Nexus
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
Closes #129.
2023-11-24 13:06:55 +01:00
3cbf383b18 Use version number from Nexus import
Non-bundled mods come without a `dtmt.cfg`, and therefore without a
version number. But we need a version number at import to compare to
for the Nexus update check.
2023-11-24 11:52:47 +01:00
bd6c236655 Implement deploying non-bundled mods
Closes #113.
2023-11-24 11:52:47 +01:00
e633a571b5 Apply clippy lints 2023-11-24 11:52:47 +01:00
871a54020e Apply formatting 2023-11-24 11:52:47 +01:00
1eae146721 Add mod config option for loose files
Just the field in the config file, for now.
2023-11-24 11:52:47 +01:00
96f3625b7a
Fix submodule refs
Some checks failed
build/msvc Build for the target platform: msvc
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
2023-11-23 13:16:03 +01:00
4ec44720b0
sdk: Don't fail decompilation on LJD error
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
2023-11-17 15:10:20 +01:00
46a61a7473
Use exit status to determine LJD failure
All checks were successful
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
2023-11-17 09:58:53 +01:00
fc759c005d
oodle: Fix building for target Windows
Some checks failed
lint/clippy Checking for common mistakes and opportunities for code improvement
build/msvc Build for the target platform: msvc
build/linux Build for the target platform: linux
2023-11-08 09:43:50 +01:00
5f43fd17ba
Vendor ansi-parser
The fork updates `nom` to version 7, removing rustc's deprecation
warning.
2023-11-08 09:43:18 +01:00
08d417fd5d
Update steamlocate-rs
While this is an unstable dev release, its only significant change is
exactly the one I need: dropping `steam-vdf` and its out-of-date `nom`
dependency.
2023-11-07 15:45:58 +01:00
48f98a43fe
oodle: Fix linking statically
Some checks failed
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
build/msvc Build for the target platform: msvc
2023-11-06 13:42:51 +01:00
7080dcb8de
sdk: Use common ljd flags
Some checks failed
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
build/msvc Build for the target platform: msvc
2023-10-30 14:07:00 +01:00
57224622d9
sdk: Don't wait for cleanup
Some checks failed
lint/clippy Checking for common mistakes and opportunities for code improvement
build/linux Build for the target platform: linux
build/msvc Build for the target platform: msvc
2023-10-30 09:28:15 +01:00