Maniphest T96824

New 3.1 OBJ exporter writes incorrect polygon/vertex groups in some cases
Closed, ResolvedBUG

Assigned To
Aras Pranckevicius (aras_p)
Authored By
thoennes (unreality)
Mar 28 2022, 4:08 AM
Tags
  • BF Blender
  • Import/Export
  • Pipeline, Assets & I/O
Subscribers
Aras Pranckevicius (aras_p)
Holger Hinze (PrefoX)
thoennes (unreality)

Description

System Information
Operating system: Mojave 10.14.6
Graphics card: NVIDIA GeForce GT 750M 2 GB

Blender Version
Broken: Version 3.1.0 (3.1.0 2022-03-09)
Worked: Version 2.93.4

Summary
Blender OBJ import has the option to "keep vertex order". When exporting the Blender Object back out, the v's, g's and f's should be in the same order as they were imported.

Details
The bug is triggered if the input file spreads a single group's faces over multiple g labels. If a group's faces are in a single g label, then the OBJ exports correctly. the OBJ format allows a group's faces to be declared in multiple separate g labels. They are logically connected into a single group. Some applications export OBJs with faces in single groups, some export in internal vert order, adding g labels as needed.

Implications
Some commercial packages use objects as shape keys for morphs and animations. A specific requirement is that vertex order is maintained. This includes faces and group compositions. This has worked correctly up until the new C++ OBJ importer.

Steps
Import an OBJ with the faces of a single group spread over multiple g labels, with the only option being "poly groups" and "keep vertex order". Export with same options. The exported OBJ in 3.1.0 has broken groups. The v order, f structures, and f order are correct. But some of the f's show up in the wrong g's

Note: the attached files contain a commercial figure consisting of proprietary information. Do not use or distribute beyond what is required for this BUG process.





The following script compares the files. The vert count and face count's are equal, but their distribution within groups in the 3.1.0 exported OBJ are incorrect

import re

files = [
	'Dawn.obj',
	'2.93.4-Dawn.obj-import.obj',
	'3.1.0-Dawn.obj-import.obj'
]

raw = {}
obj = {}

print('FILES:', files)
print()

print('{: <30} {: <20} {: <20}'.format(
	'FILENAME',
	'VERT COUNT',
	'FACE COUNT'
))

for filename in files:
	file = open(filename, 'r')
	data = file.readlines()

	raw[filename] = []
	obj[filename] = {'v':[],'g':{}}

	grp = None
	for line in data:
		line = re.sub(r'\n$', r'', line)
		raw[filename].append(line)
		ins = line[:2]
		if ins == 'v ':
			obj[filename]['v'].append(line)
		elif ins == 'g ' :
			grp = line.split(' ')[1]
			obj[filename]['g'].setdefault(grp, [])
		elif ins == 'f ' :
			obj[filename]['g'][grp].append(line)

	print('{: <32} {: <20} {: <20}'.format(
		filename,
		len(obj[filename]['v']),
		sum([len(obj[filename]['g'][x]) for x in obj[filename]['g']])
	))

print()
print('{: <15} {: ^75}'.format('', 'GROUP FACE COUNTS'))
print('{: <15} {: <15} {: <30} {: <30}'.format('', *files))

for grp in obj[files[0]]['g']:
	print('{: <15} {: <15} {: <30} {: <30}'.format(
		grp,
		len(obj[files[0]]['g'][grp]),
		len(obj[files[1]]['g'][grp]),
		len(obj[files[2]]['g'][grp])
	))

output

FILES: ['Dawn.obj', '2.93.4-Dawn.obj-import.obj', '3.1.0-Dawn.obj-import.obj']

FILENAME                       VERT COUNT           FACE COUNT
Dawn.obj                         39096                39326
2.93.4-Dawn.obj-import.obj       39096                39326
3.1.0-Dawn.obj-import.obj        39096                39326

                                             GROUP FACE COUNTS
                Dawn.obj        2.93.4-Dawn.obj-import.obj     3.1.0-Dawn.obj-import.obj
head            10188           10188                          10350
lowerJaw        4582            4582                           4504
lEye            624             624                            624
rEye            624             624                            624
tongue01        72              72                             80
tongue02        72              72                             72
tongue03        72              72                             72
tongue04        312             312                            288
rForeArm        568             568                            645
rCollar         463             463                            536
rShldr          453             453                            380
rHand           691             691                            766
rThumb1         141             141                            133
rThumb2         182             182                            181
rCarpal         318             318                            322
rIndex2         160             160                            200
rIndex3         301             301                            281
rIndex1         120             120                            80
rMid2           160             160                            200
rMid1           141             141                            100
rRing2          160             160                            200
rRing1          120             120                            80
rPinky2         160             160                            200
rPinky1         125             125                            80
rMid3           301             301                            281
rRing3          302             302                            282
rPinky3         298             298                            278
rThumb3         300             300                            280
lForeArm        568             568                            645
lCollar         463             463                            536
lShldr          453             453                            380
lHand           691             691                            766
lThumb1         141             141                            133
lThumb2         182             182                            181
lCarpal         318             318                            322
lIndex2         160             160                            200
lIndex3         301             301                            281
lIndex1         120             120                            80
lMid2           160             160                            200
lMid1           141             141                            100
lRing2          160             160                            200
lRing1          120             120                            80
lPinky2         160             160                            200
lPinky1         125             125                            80
lMid3           301             301                            281
lRing3          302             302                            282
lPinky3         298             298                            278
lThumb3         300             300                            280
rThigh          626             626                            692
rShin           562             562                            565
rFoot           597             597                            605
rToe            689             689                            744
rBigToe         248             248                            228
rIndexToe       308             308                            288
rMidToe         283             283                            263
rRingToe        263             263                            243
rPinkyToe       245             245                            225
lThigh          626             626                            692
lShin           562             562                            565
lFoot           597             597                            605
lToe            689             689                            744
lBigToe         248             248                            228
lIndexToe       308             308                            288
lMidToe         283             283                            263
lRingToe        263             263                            243
lPinkyToe       245             245                            225
neck            338             338                            442
neck2           260             260                            136
chest2          1678            1678                           1614
chest           446             446                            444
abdomen2        256             256                            256
abdomen         320             320                            326
pelvis          618             618                            618
hip             294             294                            160

Revisions and Commits

rB Blender
D14500

Event Timeline

thoennes (unreality) created this task.Mar 28 2022, 4:08 AM
thoennes (unreality) updated the task description.Mar 28 2022, 6:02 AM
Aras Pranckevicius (aras_p) added a subscriber: Aras Pranckevicius (aras_p).Mar 28 2022, 1:24 PM

@thoennes (unreality) does the issue still happen in the latest 3.1.1 nightly builds (https://builder.blender.org/download/daily/)? There were a handful of new OBJ exporter regression fixes there since 3.1.0.

Holger Hinze (PrefoX) added a subscriber: Holger Hinze (PrefoX).EditedMar 28 2022, 7:37 PM

they removed the "keep vertex order" in blender 3.1 at least for me a my friends. that counts for export.

And even in Version 3.1.1 the option is completly missing!!!

Aras Pranckevicius (aras_p) added a comment.Mar 28 2022, 8:28 PM

@Holger Hinze (PrefoX) in 3.1+ the "keep vertex order" is effectively always on, as far as I can tell.

thoennes (unreality) added a comment.Mar 28 2022, 11:58 PM
In T96824#1330966, @Aras Pranckevicius (aras_p) wrote:

@thoennes (unreality) does the issue still happen in the latest 3.1.1 nightly builds (https://builder.blender.org/download/daily/)? There were a handful of new OBJ exporter regression fixes there since 3.1.0.

Yes. The OBJ output, as far as v's, g's, and f's is the same as 3.1.0: the order is still maintained correctly (vertex and winding), but the group composition is incorrect.

thoennes (unreality) added a comment.Mar 29 2022, 12:06 AM
In T96824#1331130, @Holger Hinze (PrefoX) wrote:

they removed the "keep vertex order" in blender 3.1 at least for me a my friends. that counts for export.

And even in Version 3.1.1 the option is completely missing!!!

To be clear, it's removed as an option in export. It's still an option in import. It may be intentional in that selecting it in import; a user would almost certainly want it in export (no other reason to use the option). And the only reason to select it in export is BECAUSE it was selected in import. Essentially, one less option for the user to mess up. That's probably not a bad thought :)

It seems (and I hope it's intentional) that so long as the OBJ was imported with that option, it will export with that option (assuming no changes to vert count and vertex groups).

The help and tip on the "keep vert order" import option should mention that, as the way it's presented in the workflow (with the export option missing) could confuse people looking at *any* reference to the process prior to version 3.x; they would wonder where the heck the export option went

I'm not sure where blender is storing that info. It must be stored with the file since it persists, and it's stored with the obj, since it's copied with copies of the object. Both of those features are useful.

thoennes (unreality) added a comment.Mar 29 2022, 4:50 AM

Uhm... Trying to understand the source code. Is the importer/exporter C++? It looks like it's python.

Am I looking at the wrong place? https://developer.blender.org/diffusion/BA/browse/master/io_scene_obj/__init__.py

class OBJ_PT_export_geometry(bpy.types.Panel):
    bl_space_type = 'FILE_BROWSER'
    bl_region_type = 'TOOL_PROPS'
    bl_label = "Geometry"
    bl_parent_id = "FILE_PT_operator"
    bl_options = {'DEFAULT_CLOSED'}

    @classmethod
    def poll(cls, context):
        sfile = context.space_data
        operator = sfile.active_operator

        return operator.bl_idname == "EXPORT_SCENE_OT_obj"

    def draw(self, context):
        layout = self.layout
        layout.use_property_split = True
        layout.use_property_decorate = False  # No animation.

        sfile = context.space_data
        operator = sfile.active_operator

        layout.prop(operator, 'use_mesh_modifiers')
        layout.prop(operator, 'use_smooth_groups')
        layout.prop(operator, 'use_smooth_groups_bitflags')
        layout.prop(operator, 'use_normals')
        layout.prop(operator, 'use_uvs')
        layout.prop(operator, 'use_materials')
        layout.prop(operator, 'use_triangles')
        layout.prop(operator, 'use_nurbs', text="Curves as NURBS")
        layout.prop(operator, 'use_vertex_groups')
        layout.prop(operator, 'keep_vertex_order')

And yet that's not what the export panel looks like.

Aras Pranckevicius (aras_p) changed the task status from Needs Triage to Confirmed.Mar 29 2022, 2:32 PM
Aras Pranckevicius (aras_p) claimed this task.

@thoennes (unreality) in 3.1.x the OBJ exporter is in C++ (the old exporter python code still exists but is disabled), but the importer is the same as in previous version, i.e. written in Python.

Anyway, I can definitely see the problem with your repro files, thanks! Something is messed up in the polygon vertex group calculation code, it seems, will try to fix it.

Aras Pranckevicius (aras_p) renamed this task from Incorrect groups in exported .obj from imported .obj with "keeping vertex order" option to New 3.1 OBJ exporter writes incorrect polygon/vertex groups in some cases.Mar 29 2022, 2:33 PM
Aras Pranckevicius (aras_p) added projects: Import/Export, Pipeline, Assets & I/O.
Bastien Montagne (mont29) moved this task from Backlog to Bugs on the Pipeline, Assets & I/O board.Mar 29 2022, 5:31 PM
Bastien Montagne (mont29) changed the subtype of this task from "Report" to "Bug".
Aras Pranckevicius (aras_p) closed this task as Resolved by committing rBb32cb0266c61: Fix T96824: New 3.1 OBJ exporter writes incorrect polygon/vertex groups in some….Apr 17 2022, 9:01 PM
Aras Pranckevicius (aras_p) added a commit: rBb32cb0266c61: Fix T96824: New 3.1 OBJ exporter writes incorrect polygon/vertex groups in some….