@Brecht Van Lommel (brecht) Do you have an opinion on this?

I think it's best to just always use 'extend' in the specific case of normal baking in tangent space. But I'm not really sure how to communicate that in the UI.

Is there a way to disable one of the options depending on the rest of the settings?

In the Python UI code you would just hide the option depending if tangent normal maps are selected, and then match the same behavior internally.

Can we make tangent normal maps work with Adjacent Faces? At least in principle we can write the same normal to both pixels but encode it in a different tangent space.

Technically probably yes. Though I don't really know enough/ have enough experience with tangent space to really know how. I can sort of imagine what would need to be done (probably rotate the x and y (?) components the same amount as the angle between the corresponding edges? Something like that).

I'd need some help/docs about how *exactly* tangent space works as I have a global conceptual understanding of what it is, but no actual experience with the math.

It would probably be something like:

• look up pixel from adjacent face (Which can actually search through multiple faces for very narrow faces)
• determine the difference in angle between the original edge the margin pixel belongs to and the edge of the polygon it lands in.
• interpret the rgb values as xyz. rotate (around (0,0,1)?) with that angle difference (or the negative), recode in rgb

The normals are baked in world space, and then transformed to tangent space with RE_bake_normal_world_to_tangent.

I don't think you would try to rotate the normal, rather you'd figure out which triangle and barycentric coordinate to use for encoding the tangent space in, and then use that to transform from world to tangent space like any other pixel.

The problem is that the code is structured to first do the tangent space conversion and then do the margin. One way to make this work would be to do the margin first, and ensure it writes an appropriate triangle id + barycentric coordinate for the tangent conversion to work.

Not sure how difficult this is, just disabling this margin method for tangent space is also ok if you don't have the time.

But the adjacent faces margin generation algorithm is purely a pixel-based post processing step after the baking itself.

So I don't really understand what you mean here.

You mean the baking happens in world space and the conversion to tangent space is also a post processing step? If that is true it would probably not be very hard to move the converting to tangent space to after the margin generation. The margin generation code already looks up the 'owning' triangle for each margin pixel.

I can look into it a bit and see if I think it's manageable. I don't think I could do it before 3.2 though, so maybe best to (temporarily) disable it for 3.2 ?

The conversion to tangent space is also a post processing step, yes. And it would be best to have this fixed for 3.2, so temporarily disabling this for tangent bakes may be best then.

I added a diff to disable for now. Doing it properly is not exactly trivial I think. I'll look into that in more detail later.

I also disabled Adjacent Faces mode for UV baking , as it was nonsensical there.

Shall I mark this as resolved?

I'd like a way to keep in mind that the extend method *could* support normal baking if we refactor the code to do the extending first and the converting to adjacent space later. But just keeping this open seems incorrect?