Skip to content

feat(ktx): add initial ktx input/output support. - #5185

Open
walcht wants to merge 45 commits into
AcademySoftwareFoundation:mainfrom
walcht:main
Open

feat(ktx): add initial ktx input/output support.#5185
walcht wants to merge 45 commits into
AcademySoftwareFoundation:mainfrom
walcht:main

Conversation

@walcht

@walcht walcht commented May 5, 2026

Copy link
Copy Markdown

Description

Fixes #2329. Add initial KTX2 format support (closest similar plugin/format in OIIO is DDS).

This KTX plugin support obviously nullifies the benefits of using KTX in the
first place. That being said, this plugin is still useful so that end users
don't have to convert back and forth between KTX <-> supported format (e.g., PNG).
It is also useful to convert to and from KTX2 format using OIIO.

An example usecase would be Blender and its glTf import/export plugin.
(see KhronosGroup/glTF-Blender-IO#1896).

Ideally, at some point in the future, OIIO may introduce a new API to accomodate
texture formats that are mainly used for fast texture uploads to GPUs.

For details on KTX2 and what this PR supports and what it doesn't (alongside a
description of current limitations), see src/ktx.imageio/README.md (copied parts
of it at the end of this).

Tests:

Added initial testsuite but only for the input (i.e., info_command) part of the plugin.
Ideally, after fixing CI issues, the tests will be adjusted for ktxouput.

Checklist:

  • I have read the guidelines on contributions and code review procedures.
  • I have read the Policy on AI Coding Assistants
    => Haven't used any AI coding assistant tools in any capacity whatsoever.
  • I have updated the documentation if my PR adds features or changes
    behavior.
  • I am sure that this PR's changes are tested in the testsuite.
  • I have run and passed the testsuite in CI before submitting the
    PR, by pushing the changes to my fork and seeing that the automated CI
    passed there.
    => The 3 failing MacOS CIs are not caused by this PR (Jxl and OpenCV-related).
  • My code follows the prevailing code style of this project and I
    fixed any problems reported by the clang-format CI test.
  • If I added or modified a public C++ API call, I have also amended the
    corresponding Python bindings. If altering ImageBufAlgo functions, I also
    exposed the new functionality as oiiotool options.
    => No new API is introduced. This is just a plugin.

The sections below are copied from src/ktx.imageio/README.md. These will be updated
as this PR progresses.

Supported Encoders/Decoders

  • Supported/Tested texture kinds:

    • SINGLE_TEXTURE_1D (TODO)
    • SINGLE_TEXTURE_2D
    • SINGLE_TEXTURE_3D
    • CUBEMAP_TEXTURE (TODO)
    • ARRAY_TEXTURE_1D
    • ARRAY_TEXTURE_2D
    • [ ] ARRAY_TEXTURE_3D (not planned)
    • [ ] ARRAY_TEXTURE_CUBEMAP (not planned)
  • Supported/Tested raw VkFormats (decoder + encoder):

    • VK_FORMAT_R8_UNORM
    • VK_FORMAT_R8G8_SRGB
    • VK_FORMAT_R8G8B8_SRGB
    • VK_FORMAT_R8G8B8A8_SRGB
  • Block-compressed formats (decoder + encoder):

  • Basis Universal schemes (encoder + decoder):

    • UASTC
    • ETC1S
  • Supercompression schemes (decompressor + compressor):

    • ZLIB
    • ZSTD

Dependencies (only one - libktx)

  • libktx: for general KTX@ format support (loading of KTX2 files, transcoding
    support, supercompression decompression support, etc.).

    • Commit hash: see OpenImageIO/src/cmake/build_Ktx.cmake
    • License: Many subresources. All permissive except lib/etcdec.cxx's license (ETC is not used in PR).

This PR should only be merged after the release of libktx 5.0.0.

Resources

Add limited support for input and output for the KTX2
format.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented May 6, 2026

Copy link
Copy Markdown
Author

Update: I'm working on integrating BCn decoders/encoders directly within libktx so that this PR gets significantly simpler (e.g., from 12_000 lines most of which are from dependencies to circa 2_000).

walcht and others added 2 commits June 22, 2026 13:27
*Significantly simplify KTX support by relying on libktx to
decode/encode GPU-compressed formats such as BCn, ASTC, and ETC.
Consequently, BCn and ETC dependencies are removed. All future required
encoders/decoders should be implemented in libktx and not here to keep
this is as simple, maintainable, and short as possible.

*Remove ETC sub-dependency from local libktx build.

*Update ktx README.md documentation to reflect that only libktx
dependency is and should be used (i.e., no additional dependencies
should be added).

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented Jun 22, 2026

Copy link
Copy Markdown
Author

Update2: significantly simplified the whole PR. Now only dependency is libktx (since I offloaded most work to libktx via a PR that will be probably merged soon).

Can you please not approve workflows unless I explicitly request a workflow run? (to avoid filling the actions log with CIs that I know will certainly fail).

walcht added 2 commits June 22, 2026 15:20
*IntelLLVM is simply not supported by libktx.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@lgritz

lgritz commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Can you please not approve workflows unless I explicitly request a workflow run? (to avoid filling the actions log with CIs that I know will certainly fail).

Sure, in this case I can do so, but only because you are a a first-time contributor to this project, so admins have to approve workflow runs on the project's main account. Once you have any PR accepted and merged, the workflows will run automatically.

Here is the general solution that you or others can do, in this situation and after you no longer need admin approval:

Let's say you have pushed to your fork's branch "mypr" and submitted a PR to the project from that branch. (Aside: I strongly recommend working in topic branches, and never submitting a PR from your "main".)

After submitting the PR, you realize you need to iterate (you want to add things, or CI is not passing and you need to make repairs, or you have review comments to address), but you don't want to do a series of many many pushes as you go, each of which will trigger a CI run on the main branch as well as sending email alerts to everybody "watching" this project, since pushing to your branch is technically a PR update.

Instead, push to a different branch!

git push origin mypr:test

This pushes your fork's "mypr" branch to your fork in a "test" branch. This will run the CI on your fork in your account (allowing you to see if CI passes for the changes you've made), but since your "test" branch is not associated with the PR, it will not send the rest of us alerts and will not trigger a CI run on the project's main repo account. Then when you finally have it right and are ready for the rest of us to see it, you can git push origin mypr to update "mypr", which the system will know is the branch you are making the PR from.

*Add to_native* conversion to KTX2 output similar to how JPEG and PNG OIIO
outputs are written.

*Apply clang-format-17 as opposed to the previously applied clang-format
v18 which caused the CI to fail.

*Update CMAKE_VERSION in CIs to minimum version required by libktx
(i.e., 3.23).

*Fix uninitialized structs passed to libktx.

*Other misc clean-ups and minor refactoring.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented Jun 29, 2026

Copy link
Copy Markdown
Author

Question about older CMake version that is failing some CIs and icpx:

  1. Some CIs require a relatively old version of CMake (3.18.3) but libktx (the library through which KTX2 support is achieved) requires CMake >= 3.23. (e.g., CI oldest clang11/C++17 py3.9 exr3.1 ocio2.3 / c++17 py3.9).
    In case I may not be able to lower libktx's minimum CMake version (e.g., request refused or simply not possible/requires too many changes) => I couldn't lower CMake version simply because libktx dependencies also require CMake >= 3.20. How should I proceed with this? => Solved. Used older libktx version (4.3.2) for these CIs.

  2. ICPX compiler is not supported by libktx (haven't tried compiling it with it yet but it is definitely not supported as per the main CMakeList.txt of libktx). How should I proceed here? => Solved. I have just followed libuhrd and overridden C/C++ compilers for libktx in ICX CI.

These are the only* remaining CI issues.

*: there is another one that is Intel-MacOS only but there is a PR already to fix it in libktx.

All CI issues are now solved. Will push updates here when sufficient testing is done.

walcht added 19 commits July 4, 2026 17:46
…actoring

*KTX does not support compilation with Intel's C++ compiler ICX hence
why an override option is provided and is used in ICX CI.

*Do all decompression/decoding/deflation in open() rather than in
seek_subimage()

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Due to missing 'alpha' support (which obviously exists in libktx), unit
testing was failing (due to unexpected number of channels).

*Misc cleanup; removal of ktxTexture* member field and only using
ktxTexture2*.

*Remove MSVC/GNU/Clang checks on libktx dependency in externalpackages.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Previously, a buffer allocated via a malloc by libktx was freed using
RAII via std::unique_ptr's delete[] which doesn't match the allocator.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*libktx > 4.3.2 requires CMake >= 3.22. The code changes required for
this will also be applied accordingly (e.g., older libktx versions don't
support direct BCn encoding/decoding).

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Previously, libktx version was hardcoded to v5.0.0 now it is based on
the CMake-set variable Ktx_VERSION.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*malloc has to be matched by a call to free not a call to delete.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Attempting to address old CMake version with 'oldest' CIs.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
*Point to not-yet-merged PR in libktx only for Intel-based MacOS runner.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
This is added so that CIs on older runners/compilers/cmake can pass.
Newer version of libktx require CMake >= 3.22 which these runners cannot
install.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
ktxBasisParams is vastly different starting from libktx >= 5.0.0.

Use smart pointers to handle ktxTexture resources and clean them up.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
To be reverted once the linkage issue for ASTCENC is solved.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
walcht added 17 commits July 10, 2026 18:00
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Prior to this commit, libktx defaulted to using AVX2 on Intel-based
MacOS runners which do not support that SIMD. This caused hard-to-debug
linkage error messages related to ASTCENC whose source files were not
integrated in libktx. For more details, see:
KhronosGroup/KTX-Software#1210

Revert previous temporary commits that were used to figure out
hard-to-locally-reproduce linkage errors with libktx.a.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Remove wrong CMake generator expression. Unsurprisingly, whenever I use
them I mess something up.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Prior to this commit, CMAKE_ARGS was provided as a single string of
"[<-D var=value>...]". This is incorrect as CMAKE_ARGS have to be
provided as a list and not a string.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* Following different approaches to adding Volume and Cubemap support to
  ktx. Expecting major changes so saving a temporary commit in case
  new approach doesn't work.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* Removed because std::from_char on MacOS still doesn't support parsing
  of floating point values.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* CIs with old gcc versions ("oldest" CIs) use an older version of CMake
  than that is required by libktx 5.0.0. This commit attempts to disable
  ktx plugin testing on such old configurations by setting ENABLE_KTX to
  0.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* libktx v4.3.2 is removed from build_Ktx.cmake and, from now on, only
  libktx v5.0.0 or newer is supported.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
* Add initial documentation about ktx plugin and its exposed attributes

* Add more extensive testing

* Misc cleanups

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht
walcht marked this pull request as ready for review July 22, 2026 17:01
walcht added 2 commits July 22, 2026 19:11
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@walcht

walcht commented Jul 22, 2026

Copy link
Copy Markdown
Author

PR is now ready for initial review. There are still some missing tests but at this point I will wait for initial review then complete missing tests as necessary (there are also a lot of introduced "ktx:<attrib>" attributes. I don't know of any other way to configure the high number of parameters of libktx in ktxoutput).

* libktx's ktxTexture_GetRowPitch is wrongly padding the actual correct
  computation to be a multiple of 4. This is only valid for KTXv1 but we
  only use/support KTXv2 and ktxTexture_GetRowPitch simply does not
  distinguish between the two.
  See issue: KhronosGroup/KTX-Software#1228

* Add HDR support for ktxinput alongside UASTC-HDR/uncompressed HDR
  testcases.

* Add support for HDR ASTC vkFormats (ASTC_WxH_SFLOAT_BLOCK).

* Clean-up metadata parsing (KTXScWriterParams was wrongly serialized as
  uint8 array rather than a string). Remove unnecessary/unused metadata.

Signed-off-by: Walid Chtioui <walid.chtioui.main@gmail.com>
@lgritz lgritz added enhancement Improvement of existing/working features. file formats Image file formats, ImageInput, ImageOutput labels Aug 2, 2026
Comment on lines +267 to +270
# libktx 5.0.0 requires CMake >= 3.22. libktx v4.3.2 supports CMake v3.18 which
# is used in "oldest gcc" CIs but supporting two versions of libktx is very
# cumbersome.
if (CMAKE_VERSION VERSION_GREATER_EQUAL "3.22")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a separate PR #5394 that will raise the cmake minimum for OIIO 3.2 to cmake 3.22. So I think you can remove this bit of complexity.

@lgritz lgritz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the general gist of this and I would definitely like to get ktx support in as a 3.2 feature.

Took a preliminary pass over this and commented on an array of topics, not intended to be comprehensive, but enough for you to get started on the obvious things.

Comment thread src/cmake/testing.cmake
Comment on lines +553 to +554
REPO https://github.com/walcht/OpenImageIO-images.git
BRANCH add-ktx2-images)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll want to actually make a PR against OpenImageIO-images that adds the ktx test images, and change this to refer to that.

Comment thread src/ktx.imageio/ktx_pvt.h
@@ -0,0 +1,594 @@
#pragma once

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs our usual boilerplate license we put at the head of all of our files.

Comment on lines +71 to +72
int m_subimage { -1 }; ///< What subimage are we looking at. This is not
///< used anywhere else except in current_subimage()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need any of the doxygen /// style comments here. We only use that for things that are in public APIs where we are doing automatic generation of the documentation. It doesn't hurt anything, but we definitely don't use it here, so regular // comment are fine.

bool read_native_scanline(int subimage, int miplevel, int y, int z,
void* data) override;

bool read_native_scanlines(int subimage, int miplevel, int ybegin, int yend,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are the old versions of these call signatures. It's true that many of the existing reader/writers still use them, and we will eventually get around to fixing them.

But for NEW readers that come along, I'd like us to use the new functions from the start. These are the versions of read_native_* that take span<std::byte> and so have explicit bounds to the buffer being passed where it writes into.

Then the old ones (that just take the void* and not a span) can also be trivially implemented as calling the span-based ones (though that merely assumes that the pointer points to enough space).

Comment on lines +87 to +91
bool internal_readimg(unsigned char* dst, int w, int h, int d);

/// Checks the magic
bool ktx_magic_cmp(const uint8_t* sig, size_t start) const;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer that for new code, we avoid passing raw pointers that have an assumed length, and instead we should be passing span or similar.

// Currently only two colorspaces are tested, linear or REC709 sRGB
const auto colorspace = m_spec.get_string_attribute("oiio:ColorSpace",
"srgb_rec709_scene");
bool is_srgb = colorspace == "srgb_rec709_scene";

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like ColorConfig::equivalent is a better way to ask "is this colorspace XXX?, since it accounts for aliases synonyms.

Comment on lines +233 to +238
if (iequals(supercompression_str, "NONE")) {
m_superCmp = ktxSupercmpScheme::KTX_SS_NONE;
} else if (iequals(supercompression_str, "ZSTD")) {
m_superCmp = ktxSupercmpScheme::KTX_SS_ZSTD;
} else if (iequals(supercompression_str, "ZLIB")) {
m_superCmp = ktxSupercmpScheme::KTX_SS_ZLIB;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use of iequals makes this equivalent, but I think we use lower case ("none", "zstd", "zip" -- not "zlib") almost everywhere else for compression names.

Comment on lines +393 to +394
KtxOutput::write_scanlines(int ybegin, int yend, int z, TypeDesc format,
const void* data, stride_t xstride, stride_t ystride)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as on the input side -- since this is an entirely new writer, let's start from day one with the "span-based" versions of these calls fully implemented so we don't need to come back to it later.

PLUGENTRY(targa);
PLUGENTRY(webp);
PLUGENTRY(zfile);
PLUGENTRY(ktx);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the rest of the list is alphabetical

Comment on lines +443 to +445
#if !defined(DISABLE_KTX)
DECLAREPLUG (ktx);
#endif

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alphabetical

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improvement of existing/working features. file formats Image file formats, ImageInput, ImageOutput

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KTX format

2 participants