Currently if a layer cannot be found it actually throws an error but instead it should just return a nullptr. That way users can check
if (auto lr = findLayerAs<T, GroupLayer>("path"))

This appears to be due to the way we calculate the amount of channels in a file when going from LayeredFile<T> -> PhotoshopFile. It doesnt detect any channels and therefore writes out 0 which isnt valid for Photoshop. What it appears to actually do is just write out 4 channels in RGB mode and write that into the merged image data section

This is the main issue for adding bindings for the PhotoshopAPI. Likely this will be as a module called psapi.

The general workflow can then be:

import psapi

file = psapi.LayeredFile_8bit.read("in_path.psd")
psapi.LayeredFile_8bit.write(file, "out_path.psd")

The initial goal is to support all items referenced in the docs

When writing PSB files for some reason they wont open unless we add padding bytes at the end of the LayerAndMaskInformation section. I have inspected the binaries of both Photoshop generated binaries as well as our generated binaries and cant really see a reason why this would happen. The commit in question that "fixes" this is 5223b4d and the relevant code can be found below

void LayerAndMaskInformation::write(File& document, const FileHeader& header)
{
	....
	// I genuinely have no idea why this is required but when we dont add this the files wont open in Photoshop
	if (header.m_Version == Enum::Version::Psb)
	{
		WritePadddingBytes(document, 2u);	// 4 also appears to be a valid number here
	}
}

While this does temporarily fix the issue I would like for these bytes to have a reason :)

When writing uneven files likely due to how we go from bbox coords to center coords (division) It writes out the extents incorrectly. e.g. when using the following code:

using namespace NAMESPACE_PSAPI;

std::filesystem::path psb_path = std::filesystem::current_path();
psb_path += "\\Document.psb";

const uint32_t width = 63;
const uint32_t height = 32;
LayeredFile<bpp32_t> document = { Enum::ColorMode::RGB, width, height };

std::unordered_map <Enum::ChannelID, std::vector<bpp32_t>> channelMap;
channelMap[Enum::ChannelID::Red] = std::vector<bpp32_t>(width * height, .25f);
channelMap[Enum::ChannelID::Green] = std::vector<bpp32_t>(width * height, .25f);
channelMap[Enum::ChannelID::Blue] = std::vector<bpp32_t>(width * height, .25f);

ImageLayer<bpp32_t>::Params layerParams = {};
layerParams.layerName = "Layer";
layerParams.width = width;
layerParams.height = height;

auto layer = std::make_shared<ImageLayer<bpp32_t>>(
	std::move(channelMap),
	layerParams
);
document.addLayer(layer);

File::FileParams params = { .doRead = false, .forceOverwrite = true };
auto outputFile = File(psb_path, params);
auto psdDocumentPtr = LayeredToPhotoshopFile(std::move(document));
psdDocumentPtr->write(outputFile);

It writes out the bbox as [0, 0, 32, 62] which is incorrect as it should be [0, 0, 32, 63] Please investigate

Currently a call to those functions might look like this:

auto nestedGroupLayer = layeredFile.findLayer("Group/NestedGroup");
if (nestedGroupLayer)
{
	layeredFile.moveLayer(nestedGroupLayer);
}

which isnt inherently bad but there should also be the option to call these functions with a path instead as that would simplify the code some more.

layeredFile.moveLayer("Group/Nested");

The original signature is still going to be it is useful if we already have a ptr to the layer itself

It doesnt fail and all the layers are there but it has data that is misplaced

In order to get the PhotoshopAPI closer to being production ready we should always compile it with /Wall and /fsanitize=address (This is the only sanitizer currently supported out of the box with MSVC). Especially since our test cases cant and wont ever be able to cover all possible issues

This is due to the way we construct our include for the PhotoshopAPI target

target_include_directories(PhotoshopAPI PUBLIC include ${CMAKE_SOURCE_DIR}/thirdparty/c-blosc2/include src src/Util)

where a better approach would be to add an INTERFACE target for the include dirs in our top level build and not rely on ${CMAKE_SOURCE_DIR} but instead ${CMAKE_PROJECT_DIR}

This could look a little something like this:

add_library(blosc2_include INTERFACE)
target_include_directories(blosc2_include INTERFACE thirdparty/c-blosc_2/include )

This information seems to be stored on the 'lspf' tagged block on bit 8 as well as perhaps on the 'flags' of the layer record but this difference might be unrelated

The goal of this endeavour would be to have PhotoshopAPI be hosted on ReadTheDocs where every commit automatically pushes the latest docs. The tech stack would likely be Doxygen + Breathe + Sphinx as outlined in this blog. The comment structure itself likely needs to be updated to reflect this

PhotoshopAPI should have the option to attach a given ICC profile to a LayeredFile which does not apply any transforms but simply tells photoshop what type of data we are dealing with

From the benchmarks presented here zlib-ng/zlib-ng#1486 it appears that even for larger buffers libdeflate outperforms zlib-ng on both compression ratios as well as speed.

It does provide a different api than zlib-ng and also doesnt support streaming which are both not necessarily concerns for our use cases.

This is a rather big issue as Photoshop has a wide range of adjustment layers, all with their different implementations and different levels of completeness in the official documentation.

This change would have to be two-fold; adding support for the tagged blocks in the PhotoshopFile under AdditionalLayerInformation and then actually implementing these as an abstraction of Layer<T> in the LayeredFile<T>.

For this issue support would be greatly appreciated as its a rather large issue but at the same time each adjustment layer is its individual separate object. I will attempt to provide a reference implementation once I get started but feel free to contribute!

Currently our LayeredFile has no well specified way of extracting image data work with it. This should be added

This internally seems to be handled by the 1005 ResolutionInfo ImageResource which is documented in the Photoshop API Guide from Adobe. The structure is as following

struct ResolutionInfo
{
   uint32_t fixedPointHRes;  
   uint16_t hResUnit;    // 1 for Pixels per inch and 2 for pixels per cm
   uint16_t widthUnit;  // 1 = inches, 2 = cm, 3 = points, 4 = picas, 5 = columns
   uint32_t fixedPointVRes;  
   uint16_t VResUnit;    // 1 for Pixels per inch and 2 for pixels per cm
   uint16_t heightUnit;  // 1 = inches, 2 = cm, 3 = points, 4 = picas, 5 = columns
}

You can see the structure for FixedPoint values below

struct FixedPoint
{
   uint16_t value;
   uint16_t fraction;   // .5 would be 65536/2 for example
}

These would require us to fundamentally implement a system for ImageResources similar to the ones present for TaggedBlocks. This would be required either way for #18

Currently our program uses an excessive amount of memory as it stores all the image data in an uncompressed manner which leads to big files (a focus of what this API should be able to handle) taking up loads of memory.

An example file which unfortunately cannot be shared is ~6GB on disk but takes up >80GB of memory when decompressing it in release mode. The file contains 170 Image layers (and some more mask layers) and has a bit depth of 16 with a resolution of 9124x6082

The goal of this change would be to bring memory usage on par with, or slightly above what photoshop files store but without sacrificing on performance too much.

When trying to parse files with Vector masks the PhotoshopAPI breaks down in c-blosc2 due to the PhotoshopAPI expecting to have e.g. 4096 bytes of data for a 64*64 image for the layer upon decompression but we instead get 0 because the data is stored elsewhere in the additional layer info of the layer record. While we dont have support for vector masks we can simply add a check if the channels size does not match what we expect and raise a warning and continue.

I'm using Python 3.10, cmake 3.29.1 and ninja 1.11.1. I usually try first with my own versions to avoid env balkanization.

With meson 1.3.0:

running build_ext
Traceback (most recent call last):
  File "./PhotoshopAPI/setup.py", line 153, in <module>
    setup(
[..]
    ninja_executable_path = Path(ninja.BIN_DIR) / "ninja"
AttributeError: module 'ninja' has no attribute 'BIN_DIR'

After updating to meson 1.4.0:

ERROR Missing dependencies:
	ninja
	cmake>=3.12

Which versions do you recommend?

With 16-bit files we have the expected white mask but for the other color depths that isnt the case. The input files producing this unexpected output:

PhotoshopTest/documents/Groups/Groups_8bit.psb
PhotoshopTest/documents/Groups/Groups_8bit.psd
PhotoshopTest/documents/Groups/Groups_32bit.psb
PhotoshopTest/documents/Groups/Groups_32bit.psd

when running the test case TEST_CASE("Check Roundtripping Groups") in TestRoundtripping.cpp

This would be statically built into the library

CMYK is currently only partially supported but shouldnt be far off from actually fully working within the PhotoshopAPI. Testing this is required and ironing out any kinks + raising warnings if writing a file that would not be Photoshop compliant (i.e. not all adjustment layers are supported in CMYK)

Often times when parsing the AdditionalLayerInfo section a warning like the one below gets raised but reading it does actually still work without problems.

[AdditionalLayerInfo]Read too much data for the additional layer info, was allowed 712 but read 716 instead

It also appears that it is consistently saying it read 4 bytes too many stemming from 16- and 32- bit files with Lr16 and Lr32 Tagged Blocks. This is especially apparent when building the PhotoshopTest target

Steps to reproduce:

• Create a PhotoshopDocument which will create a "Background" locked layer which isnt stored as actual pixel values
• Try to roundtrip the file i.e. read -> write using the LayeredFile Struct like seen below

  auto inputFile = File(fullInPath);
  auto psDocumentPtr = std::make_unique<PhotoshopFile>();
  psDocumentPtr->read(inputFile);
  LayeredFile<bpp8_t> layeredFile = { std::move(psDocumentPtr) };
  
  // Write to disk
  File::FileParams params = { .doRead = false, .forceOverwrite = true };
  auto outputFile = File(fullOutPath, params);
  auto psdOutDocumentPtr = LayeredToPhotoshopFile(std::move(layeredFile));
  psdOutDocumentPtr->write(outputFile);

• This will raise the following error:

  [LayerInfo] Invalid Document encountered. Photoshop files must contain at least one layer

This happens because Photoshop stores this single layer as merged image data rather than a layer

A possible fix would be to warn about this on read already by adding a similar check giving the user some more information

Greyscale is currently only partially supported but shouldnt be far off from actually fully working within the PhotoshopAPI. Testing this is required and ironing out any kinks + raising warnings if writing a file that would not be Photoshop compliant (i.e. not all adjustment layers are supported in Greyscale)

Create a general benchmarking suite that we can use to continuously test the PhotoshopAPI to its limits. This benchmark suite is not meant to test the library but should provide a rather broad range of use cases such as:

• Writing very large files (>2GB) in PSD and PSB with all bit depths
• Reading very large files (>2GB) in PSD and PSB with all bit depths
• Read Complex nested hierarchies
• Create Complex nested hierarchies
• Benchmark Modifying complex nested hierarchies

All of these test cases should profile their speed, ram usage and file size on disk against Photoshop itself and we will run these benchmarks on all the PCs available at home. Their specs can be found below:

• AMD Ryzen 9 5950x || 96GB DDR4 3200MHz
• AMD Ryzen 5 2600x || 32GB DDR4 3200MHz
• ...

PhotoshopAPI could potentially gain from having a custom threadpool struct rather than using std::for_each with std::execution::par to more efficiently manage resource especially in cases where we actually have nested threaded calls.

For example, when decoding the LayerInfo Section we currently parse each layer in sequence with each sub-channel in parallel (usually 4 channels). This means we leave a lot of potential performance on the table by not utilizing cpus with more than 4 cores very efficiently.

In an ideal scenario we would submit tasks to a thread_queue as well as having some utility function which splits the works into discrete blocks such as a parallelizeLoop() function which will split the work across the available cores. This would likely be especially handy for our decompression functions which, in part, contain such highly multithreaded work

Currently the binaries (e.g. the .exe) requires VCRedist to correctly compile which is unfortunate.

This appears to only fail on compilation of our AVX2 intrinsics which we can simply opt out of if we detect arm based machines and in the future we can add arm based intrinsics

When initially pulling the repo, Zlib-NG needs to first be manually built using cmake in the commandline as PhotoshopAPI wouldnt build otherwise. This is likely some conflict with the way zlib-ng is built.