fix(IBA): reorient() cases for transpose and transverse

[luna-y-kim]

Orientations 5 and 7 were swapped: case 5 produced transverse and case 7 produced transpose. Fix both cases.

This adds reorient() tests covering all 8 values to the oiiotool-xform testsuite, and updates the python-imagebufalgo reorient() test to use the new test image for consistency.

Also, add orientation example images to documentation.

Fixes #4661

Checklist:

• I have read the guidelines on contributions and code review procedures.
• I have read the Policy on AI Coding Assistants
  and if I used AI coding assistants, I have an Assisted-by: TOOL / MODEL
  line in the pull request description above.
• I have updated the documentation if my PR adds features or changes
  behavior.
• I am sure that this PR's changes are tested in the testsuite.
• I have run and passed the testsuite in CI before submitting the
  PR, by pushing the changes to my fork and seeing that the automated CI
  passed there. (Exceptions: If most tests pass and you can't figure out why
  the remaining ones fail, it's ok to submit the PR and ask for help. Or if
  any failures seem entirely unrelated to your change; sometimes things break
  on the GitHub runners.)
• My code follows the prevailing code style of this project and I
  fixed any problems reported by the clang-format CI test.
• (N/A) If I added or modified a public C++ API call, I have also amended the
  corresponding Python bindings. If altering ImageBufAlgo functions, I also
  exposed the new functionality as oiiotool options.

[lgritz]

The orientation codes are so inherently confusing to think about in words.
I really think that ultimately, we should illustrate this with a picture as well as the words (not necessarily on you to do in this PR).

I also think, if we don't have it, we really need a testsuite verification of reorient for EACH of the 8 values. I guess we don't, or the code change you made would either be failing tests, or be accompanied by updated reference images? I have to admit I'm afraid to accept this change in behavior without a visual test that proves that the new code definitely does the right thing.

[luna-y-kim]

The orientation codes are so inherently confusing to think about in words.
I really think that ultimately, we should illustrate this with a picture as well as the words (not necessarily on you to do in this PR).

Yes, the wording is super confusing. A visual illustration sounds good. That'd be helpful.

Also, the intro paragraph in the display hints docs does define the ordering convention, but still it requires some eyeballing for me. For values like 6 and 8, I couldn't tell which part of '(right to left, top to bottom)' described what.

[lgritz]

It feels like the right order of operations is:

1. Create an orientation=0 image that is small but clearly and visually shows the orientation. Then transform it to each of the 7 other orientations. We look at them and agree that we are all confident that all 8 exactly match what the TIFF spec says the orientation values mean.
2. Make a testsuite test that, for each of those 8, does a reorient() and ensures that the result exactly matches the orientation 0 we started with.
3. Use that testsuite to validate this PR and merge it.
4. Use those same images to produce a figure that augments the description of orientation in the stdmetadata.rst.

Without the visual example (and multiple people looking it over), I honestly don't trust myself to get this right just using the words and my head. It's really easy to make mistakes when reasoning about it.

[lgritz]

Maybe something that looks like this:

+------------------------+
| O -------------------> |
| |               cols   |
| |                      |
| |r                     |
| |o      [LOGO]         |
| |w                     |
| |s                     |
| |                      |
| v                (1,1) |
+------------------------+

[lgritz]

oh no, did I already mix up rows and columns?

[lgritz]

It's even confusing visually :-)

Does "cols" mean "this line is an example of a column" or does it mean "column indices increase in this direction?"

[lgritz]

Better?

+------------------------+
| O ---row 0-----------> |
| |                width |
| |                      |
| |c                     |
| |o      [LOGO]         |
| |l                     |
| |0                     |
| |                      |
| v height         (1,1) |
+------------------------+

[lgritz]

I want to make something that nobody could possibly look at and still be confused. :-)

[luna-y-kim]

Does "cols" mean "this line is an example of a column" or does it mean "column indices increase in this direction?"

That is exactly what confused me.

                   Top
        +------------------------+
        | <--- col 0 --------- O |
        |                      | |
        |                      r |
        |                      o |
Left    |                      w |    Right
        |                      0 |
        |                      | |
        |                      | |
        |                      ↓ |
 (1,1)  +------------------------+
                 Bottom

An example of Orientation 6.
In addition to your sketch, top/bottom/left/right tags could be helpful.

[luna-y-kim]

It feels like the right order of operations is:

1. Create an orientation=0 image that is small but clearly and visually shows the orientation. Then transform it to each of the 7 other orientations. We look at them and agree that we are all confident that all 8 exactly match what the TIFF spec says the orientation values mean.
2. Make a testsuite test that, for each of those 8, does a reorient() and ensures that the result exactly matches the orientation 0 we started with.
3. Use that testsuite to validate this PR and merge it.
4. Use those same images to produce a figure that augments the description of orientation in the stdmetadata.rst.

Without the visual example (and multiple people looking it over), I honestly don't trust myself to get this right just using the words and my head. It's really easy to make mistakes when reasoning about it.

I think I should've done this before opening the PR. I thought I understood it correctly. Lesson learned. I'm going to work on step 1 and share soon!

[lgritz]

In addition to your sketch, top/bottom/left/right tags could be helpful.

But not IN the image. If we're showing how each one should display, top is always top, etc. but the contents of the image rotate.

OH, I just realized I described the process incorrectly. (See? Confusing!) I think the correct description is:

1. Create a single small image that clearly shows origin, direction of rows and columns, etc., and make 7 more copies with the same pixel order but each of the other 7 "Orientation" metadata values. Shown in iv or some other viewer that tries to honor Orientation, we should clearly see each one in a different orientation and agree that they match what the TIFF spec describes.
2. Make a testsuite test that, for each of those 8 images, does a reorient() that results in an image with Orientation 0, but the pixels have actually been changed so that it matches what we saw in the viewer in (1). (The correct versions of this should be checked in as reference images.)
3. Use that testsuite to validate this PR and merge it.
4. Use those same reference images to produce a figure that augments the description of orientation in the stdmetadata.rst.

[luna-y-kim]

Here is the screenshot of the 8 images. I created an image, duplicated it 7 times, and set the Orientation metadata to each value on the duplicates.

Interestingly, Apple Preview's inspector shows orientation 6 as "rotated 90° CCW" and 8 as "rotated 90° CW" (in the screenshot). I tried hard to wrap my head around it, and I think both wordings "rotated 90° CW" (OIIO) and "rotated 90° CCW" (Preview app) are right and are saying the same thing in different words. Preview app describes how the encoded data is rotated relative to the intended display (that is, in software that doesn't honor orientation metadata, the image appears to be rotated 90° CCW), and OIIO describes how it should be displayed (rotated 90° CW for 6, the configuration suggested by the Orientation metadata). Am I understanding the OIIO part right?

[lgritz]

Love it!

Now I need to stare at these with the TIFF spec open and convince myself that every one is correct.

The nomenclature is really hard. If the starting image is

A B
C D

And you asked me what this image was:

C A
D B

I would say that second image is "rotated 90 degrees clockwise" (relative to the starting image). Does that help?

[luna-y-kim]

OH, YES!! Super helpful. Thanks! The "what this image was" framing is the best part.

[lgritz]

For reference, the meaning of the orientation values, from the 1992 TIFF 6.0 spec:

[lgritz]

I'm very confused by Apple's preview describing orientation 6 as "rotated counter-clockwise". That feels like the opposite of how I conceptualize it. I will look for other explanations of orientation in the other file format specs. (They all ultimately borrowed TIFF's meaning of the flags, but they might explain it in different ways. I notice that TIFF doesn't say anything about rotation, it just says which sides of the view are the 0th row and column.)

[lgritz]

I am convinced that the images you posted (that is, as displayed by Apple Preview) match the descriptions in the TIFF spec.

Now we just need to verify that OIIO is also doing the right thing when displaying with iv, and that reorient() works correctly in all cases (i.e. the reoriented result should LOOK just like those apple images, but say that they are orientation 1).

And then we need to clean up our own documentation to be less confusing/ambiguous.

[luna-y-kim]

Yep, sounds great! I'm going to work on the testsuite so that the results of reorient() look just the same as those intended orientations (the images in the screenshot), but with all images having Orientation=1 in the spec.

Also, I'll check the display in iv (haven't tried it yet, so I'd like to explore).

[luna-y-kim]

iv displays all 8 as expected, and for reference, here is a screenshot of the new reference images for the testsuite.

[luna-y-kim]

Feedback wanted: (Feedback from anyone is welcome!)

For docs, would something like this image work? It makes sense to me, but I'm not sure if it reads as clearly to others.

---

Also, maybe a header like the one below would help?

[lgritz]

I'm not sure if it's easy given the documentation system, but I was imagining that the picture for each could be incorporated directly into a single table. Like

+---+-----------+-------------+
|   |           |             |
| 1 | [picture] | Explanation |
|   |           |             |
+---+-----------+-------------+
|   |           |             |
| 2 | [picture] | Explanation |
|   |           |             |
+---+-----------+-------------+

Also, given how much we have been confused by those textual descriptions, I wonder if it would be better to directly adopt the descriptions used in the TIFF spec. I need to check the various image file format specs (not just TIFF) that incorporate this encoding to see if any of them do a better job with explanations. I'm not sure I remember where I got the wording that we have now.

[luna-y-kim]

You mean something like this?

If this looks like the right direction, I'll update this PR with just this layout change for now, and leave the wording as is until you have a chance to check the other format specs. No rush on my side either way!

[luna-y-kim]

Or, it also works without the box, like this:

[lgritz]

Yes, I like that.

[luna-y-kim]

Great! Just to be sure: the version with the box (first screenshot) or without (second)? (Personally, the second one looks cleaner!)

And about the "right to left, top to bottom" style of wording (if that is what you meant): a very similar phrasing appears in CineonHeader.h and DPXHeader.h, I ended up reading those files while trying to understand the wording for this PR.

[lgritz]

I also prefer the second one.

[lgritz]

And about the "right to left, top to bottom" style of wording (if that is what you meant): a very similar phrasing appears in CineonHeader.h and DPXHeader.h, I ended up reading those files while trying to understand the wording for this PR.

I'm not sure what I like best here, but I note that the wording from the original TIFF spec (which I posted above a few messages) is at least unambiguous, saying things like "the 0th row is the visual top of the image and the 0th column is the visual left side of the image." I'm thinking now that this is a lot better than talking about rotations, which are harder to discern which direction they mean (rotate memory to get appearance or rotate appearance to get the memory layout).

We should also probably explain, right before the table, that "row/column" refers to the order that the pixels are arranged in memory: starting with scanline/row 0, and within each scanline, starting with column 0. The orientation merely describes how that data is intended to be shown visually on a display.

It's tricky, so easy to get confused, and there is a real tension between wanting the explanations to be unambiguous and not open to misunderstanding, but not wanting to end with a long word salad that nobody understands.

[luna-y-kim]

"row/column" refers to the order that the pixels are arranged in memory

I really like the "in memory" wording here. It clears up the vagueness of "0th row", and it also implicitly explains how software that doesn't honor Orientation will show the image.

In addition to this part, maybe worth stating that "this means value 1 is displayed in memory order"? It can give readers a concrete anchor for what 'in memory' means in their mental model. (...Could this be part of a long word salad? or redundant?)

The orientation merely describes how that data is intended to be shown visually on a display.

When I read the TIFF spec for the first time, I was also confused by the wording "the visual right" together with "0th row" because neither term was defined clearly. So I think "intended to be shown visually on a display" is great here.

[lgritz]

In addition to this part, maybe worth stating that "this means value 1 is displayed in memory order"?

I think maybe this lands us right back into ambiguity, since there is no particular reason to think a display will show the pixel with the lowest memory address at the visual "top". Many display and windowing systems over the years have considered the coordinate origin to be at the bottom of the screen.

[luna-y-kim]

Ah, right! I didn't think of bottom-origin coordinate systems. Thanks for pointing that out! There's more to consider than I thought.

[lgritz]

I'm slightly worried about the ways this can fail, since in order to support multiple references when they differ per-platform, we consider a test to pass if an output we are checking matches any of the reference images. So if orientation 2 were to break and started making the image that should be orientation 4 instead, I think this would still pass?

Here is an idea, what do you think?

After you make the reorient?.tif images, assemble them into one big image using oiiotool --mosaic, and compare THAT (not the individual reoriented images) to a reference image. Then any individual reorient images being wrong will certainly be a failure and can't pass by coincidentally matching a different reorientation.