De l'acquisition à la compression des objets 3D – AC’3D
Juin 2014
Towards Perceptual Quality Evaluation of
Dynamic Meshes
Fakhri TORKHANI, Kai WANG and Jean-Marc CHASSERY
Introduction
3D Mesh, a collection of :
o Vertices
o Edges
o Faces
o V changes over time (Dynamic mesh)
gipsa-lab
1
Introduction
3D mesh
Acquisition
Processing
Reality
3D mesh
Distortion
MRI
3D camera 3D scanner
Imagination
Compression,
Watermarking,
Remeshing,
Etc..
CAD
How to measure the perceptual quality?
gipsa-lab
Visualization
2
Rendering,
Stereoscopic
Display,
Etc..
Introduction
Perceptual quality of 3D meshes:
Subjective quality
• Based on human judgments
Objective-perceptual
quality metrics
• Based on Human Visual System (HVS)
properties
• Established through lab experiments
• The ground truth
• Can be integrated to guide: compression,
watermarking, remeshing, etc.
• Should correlate with subjective opinions
gipsa-lab
3
Outline
Introduction
I.
Perceptual quality assessment of 3D meshes
II.
Dynamic meshes subjective quality database
III.
Objective quality metrics performances on dynamic meshes
IV. Towards a perceptual quality metric for dynamic meshes
Conclusion
gipsa-lab
4
Geometric distance quality metrics
Geometric distance of static meshes
• Root Mean Square error
• Hausdorff distance :
, with:
Geometric distance of dynamic meshes
A=
coordinates
• Karni and Gotsman metric
XFrame1 …. XFrame n
YFrame1 …. YFrame n
ZFrame1 …. ZFrame n
time
gipsa-lab
5
Geometric distance quality metrics
Geometric distance between static meshes: Fail to predict the perceptual quality !
Watermarked models by the
method of [Wang et al.2011]
Original models
gipsa-lab
6
Watermarked models by the
method of [Cho et al. 2007]
Perceptual quality metrics
MSDM2: Mesh Structural Distortion Measure [Lavoué et al. 2006], [Lavoué 2011]
• Based on the 2D image Structural SIMilarity (SSIM) Index [Wang et al. 2004]
o L: curvature comparison (mean curvature)
o C: contrast comparison (curvature standard deviation)
o S: structure comparison (curvature covariance)
(a and b : local windows on meshes A and B)
• Multiscale
• Connectivity independent
gipsa-lab
7
Perceptual quality metrics
DAME: Dihedral Angle Mesh Error [Vasa 2012]
Distorted mesh
Original mesh
STED: Spatio-temporal edge difference (for dynamic meshes): [Vasa 2011]
Spatial error
gipsa-lab
8
Temporal error
Perceptual quality metrics
FMPD: A fast roughness-based mesh visual quality distance (Reduced-reference)
[Wang et al. 2012]
: Global roughness: the sum of local roughness computed on each vertex.
TPDM: Tensor perceptual distance measure: [Torkhani et al. 2012]
, si : local surface area
Local tensor perceptual distance measure:
: local roughness (1-ring neighborhood) , std. deviation of the projected Min./Max. curvature directions
: ratio of the Laplacian of mean curvature in the 1-ring neighborhood and the mean curvature on current vertex
gipsa-lab
9
Existing subjective databases for static meshes
The general-purpose database [LIRIS/EPFL]
Armadillo
40002 vertices
Dinosaur
42146 vertices
RockerArm
40177 vertices
Venus
49666 vertices
Distorted meshes = 88
The masking database [LIRIS]
Bimba
8857 vertices
Armadillo
40002 vertices
Dinosaur
20228 vertices
LionVase
38728 vertices
Distorted meshes = 26
gipsa-lab
10
Existing subjective databases for static meshes
The simplification database [IEETA]
Bones
2154 vertices
Bunny
25103 vertices
Head
11703 vertices
Lung
4811 vertices
Strange
9988 vertices
Distorted meshes = 30
The compression database [UWB]
James
40218 vertices
Bunny
35961 vertices
Jessi
70291 vertices
Nissan
16947 vertices
helix
30534 vertices
Distorted meshes = 68
gipsa-lab
11
Existing subjective database for dynamic meshes
The UWB dynamic meshes database [UWB]
Chicken
3030 vertices
400 frames
Mocap-Dance
14409 vertices
263 frames
Dance
7061 vertices
201 frames
Jump
15830 vertices
222 frames
Cloth
9987 vertices
200 frames
Distorted meshes = 45 meshes
Dynamic meshes were evaluated separately in 5 experiments
Low distorted meshes number by distortion (for some distortions, only 2 impaired meshes were derived)
Low distortions number for each dynamic mesh (9 distortions)
gipsa-lab
12
Subjectively-rated Dynamic Mesh Quality Database
Assessment method
Single-Stimulus with hidden reference presentation (as described in ITU-R Rec.
BT-500)
: difference score of the distorted sequence i assigned by observer j
in session k
Assessment software
gipsa-lab
13
Subjectively-rated Dynamic Mesh Quality Database
Experiment settings
Participants: Each distorted animation was evaluated by 25 naïve observers
Subjective scores: scores range: [0 , 10] , with 5 labels {Bad, Poor, Fair, Good, Excellent}
Users-interactions: zoom/rotation/translation
Rendering :
Flat-shading
gipsa-lab
14
Gouraud
Smooth-shading
(selected rendering)
Subjectively-rated Dynamic Mesh Quality Database
Test materials
Horse
Chicken
8431 vertices
47 frames
3030 vertices
321 frames
chinchilla
43607 vertices
83 frames
Elephant
Dress
42321 vertices
48 frames
41057 vertices
81 frames
Human
Cloth-ball
Balls
Dinosaur
18890 vertices
161 frames
46598 vertices
68 frames
73960 vertices
40 frames
Armadillo
20218 vertices
151 frames
40002 vertices
74 frames
gipsa-lab
15
Subjectively-rated Dynamic Mesh Quality Database
Test materials: distortions
Global distortion
Spatial mask1
Uniform Gaussian
Descriptor-weighted distortion
Spatial mask2
Temporal mask1 Temporal mask2
In rough In smooth In rough In smooth
Horse
3
3
Chicken
3
3
Chinchilla
3
3
Elephant
3
3
Human
3
3
ClothBall
3
3
Balls 16k
3
3
Blue Dress
3
3
Armadillo
dynamic
3
3
3
3
3
3
Dinosaur
dynamic
3
3
3
3
3
3
3
3
3
In slow
In fast
3
3
3
3
3
3
3
3
3
16
3
3
3
3
•Additive Uniform noise
•Additive Gaussian noise
gipsa-lab
3
In fast
compression
Network
FAMC
error
Coddyac
In slow
LD DCT
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
Subjectively-rated Dynamic Mesh Quality Database
Test materials: distortions
Global distortion
Spatial mask1
Uniform Gaussian
Descriptor-weighted distortion
Spatial mask2
Temporal mask1 Temporal mask2
In rough In smooth In rough In smooth
Horse
3
3
Chicken
3
3
Chinchilla
3
3
Elephant
3
3
Human
3
3
ClothBall
3
3
Balls 16k
3
3
Blue Dress
3
3
Armadillo
dynamic
3
3
3
3
3
3
Dinosaur
dynamic
3
3
3
3
3
3
3
3
3
3
In fast
In slow
In fast
3
3
3
3
3
3
3
3
3
compression
Network
FAMC
error
Coddyac
In slow
LD DCT
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
• Spatial masking simulation
• Spatial masking 1: a random scalar generated for each vertex for all frames
• Spatial masking 2: a random scalar generated for each frame for all vertices
• Noise is weighted by a roughness/smoothness descriptor
gipsa-lab
17
Subjectively-rated Dynamic Mesh Quality Database
Test materials: distortions
Global distortion
Spatial mask1
Uniform Gaussian
Descriptor-weighted distortion
Spatial mask2
Temporal mask1 Temporal mask2
In rough In smooth In rough In smooth
Horse
3
3
Chicken
3
3
Chinchilla
3
3
Elephant
3
3
Human
3
3
ClothBall
3
3
Balls 16k
3
3
Blue Dress
3
3
Armadillo
dynamic
3
3
3
3
3
3
Dinosaur
dynamic
3
3
3
3
3
3
3
3
3
3
In fast
In slow
In fast
3
3
3
3
3
3
3
3
3
compression
Network
FAMC
error
Coddyac
In slow
LD DCT
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
• Temporal masking simulation
• Temporal masking 1: a random scalar generated for each vertex for all frames
• Temporal masking 2: a random scalar generated for each frame for all vertices
• Noise is weighted and inversely weighted by a per-vertex relative Speed descriptor
gipsa-lab
18
Subjectively-rated Dynamic Mesh Quality Database
Test materials: distortions
• Global (Uniform/Gaussian) distortions
with
(The same for y and z )
• Descriptor-weighted distortions (spatial and temporal masking)
(Spatial/temporal mask1)
(Spatial/temporal mask2)
gipsa-lab
19
Subjectively-rated Dynamic Mesh Quality Database
Test materials: distortions
• Local roughness descriptor : The Laplacian of Gaussian curvature
oGC: Gaussian curvature approximation :
oD: Cotg-based Laplacian matrix:
• Local velocity descriptor: The relative difference between vertices
positions in a temporal window of 3 frames.
gipsa-lab
20
Subjectively-rated Dynamic Mesh Quality Database
Test materials: distortions
Global distortion
Spatial mask1
Uniform Gaussian
Descriptor-weighted distortion
Spatial mask2
Temporal mask1 Temporal mask2
In rough In smooth In rough In smooth
Horse
3
3
Chicken
3
3
Chinchilla
3
3
Elephant
3
3
Human
3
3
ClothBall
3
3
Balls 16k
3
3
Blue Dress
3
3
Armadillo
dynamic
3
3
3
3
3
3
Dinosaur
dynamic
3
3
3
3
3
3
3
3
3
3
In fast
In slow
In fast
3
3
3
3
3
3
3
3
3
compression
Network
FAMC
error
Coddyac
In slow
LD DCT
3
3
3
3
3
3
22
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
• 3 lossy compressor schemes
• Simulation of transmission errors over unreliable wired IP network
gipsa-lab
3
Subjectively-rated Dynamic Mesh Quality Database
Test materials: distortions
Global distortion
Spatial mask1
Uniform Gaussian
Descriptor-weighted distortion
Spatial mask2
Temporal mask1 Temporal mask2
In rough In smooth In rough In smooth
Horse
3
3
Chicken
3
3
Chinchilla
3
3
Elephant
3
3
Human
3
3
ClothBall
3
3
Balls 16k
3
3
Blue Dress
3
3
Armadillo
dynamic
3
3
3
3
3
3
Dinosaur
dynamic
3
3
3
3
3
3
3
3
3
3
In fast
In slow
In fast
3
3
3
3
3
3
3
3
3
3
23
3
3
3
3
3
3
3 = distortion levels (low-medium-high quality) for each animation
Total = (10 reference + 276 distorted ) mesh to evaluate subjectively
gipsa-lab
compression
Network
FAMC
error
Coddyac
In slow
LD DCT
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
Subjectively-rated Dynamic Mesh Quality Database
Difference Mean Opinion Scores (DMOS) results with 95% CI
gipsa-lab
24
Subjectively-rated Dynamic Mesh Quality Database
Difference Mean Opinion Scores (DMOS) results with 95% CI
gipsa-lab
25
Subjectively-rated Dynamic Mesh Quality Database
Difference Mean Opinion Scores (DMOS) results with 95% CI
gipsa-lab
26
Subjectively-rated Dynamic Mesh Quality Database
Difference Mean Opinion Scores (DMOS) results
Spatial Roughness cannot
hide trembling
Rapid motion can hide
spatial noise
gipsa-lab
27
Perceptual quality metrics performances
Spearman (rank-order) correlation
Metric
Uniform Gaussian Spatial1 Spatial2
Temporal Temporal
Network Coddyac
1
2
FAMC
Overall
Mean
KG
32,24%
27,55%
27,96%
65,19%
45,59%
-0,52%
58,93%
40,67%
51,99%
11,54%
38,85%
STED
45,46%
47,25%
67,10%
85,98%
68,57%
12,53%
62,22%
44,37%
65,52%
57,66%
55,44%
DAME
19,04%
23,17%
33,05%
44,01%
32,67%
-16,94%
33,15%
0,31%
24,40%
23,07%
21,43%
MSDM2 66,48%
52,17%
72,19%
80,41%
40,98%
-3,61%
79,43%
57,40%
66,59%
59,58%
56,89%
FMPD
63,59%
57,29%
67,67%
61,58%
54,00%
25,32%
68,08%
71,25%
63,60%
62,81%
59,15%
TPDM
59,67%
55,06%
48,23%
58,32%
56,56%
13,75%
69,12%
65,64%
66,78%
55,61%
54,79%
gipsa-lab
28
Perceptual quality metrics performances
Spearman correlation for image-based metrics using default parameters:
Metric Uniform Gaussian
Spatial Spatial Temporal Temporal
Network Coddyac FAMC Overall
1
2
1
2
Mean
PSNR
17,74%
22,07%
8,94% -27,66%
32,89%
-7,90%
13,09%
-14,42%
13,35%
6,04%
MSSSIM
19,19%
26,30%
15,77% 12,44%
30,50%
9,16%
14,32%
-14,60% 10,51% 19,78%
13,73%
VIF
19,88%
21,26%
9,11% -25,66%
35,50%
-5,33%
13,72%
-7,21%
12,84%
7,86%
NIQE
-5,87%
-8,92%
-31,88% -28,48%
-3,31%
17,44%
13,56%
0,49%
-10,62% -11,13%
-6,40%
NIQE2
4,72%
-0,07%
-14,31% -12,52%
3,92%
23,66%
23,49%
9,92%
-8,97%
-5,72%
3,31%
VQM
-19,97%
-19,24%
-3,37%
-18,01%
9,16%
-10,44%
10,42% -10,60% -15,41%
-6,38%
gipsa-lab
29
4,61%
9,64%
9,50%
Towards a perceptual quality metric for dynamic meshes
1. Spatial perceptual distance
For each vertex
and its corresponding vertice
on the distorted mesh
• Local perceptual distance:
Local roughness at
Local roughness at
= 0.002 (stability coef.)
• Per-frame perceptual distance:
= 3 (Minkowski power param.)
• Spatial-based perceptual distance for dynamic meshes
and
= 4 (Minkowski power param.)
Metric Uniform Gaussian SM1
SPD
61,32
66,21
88,02
gipsa-lab
30
SM2
82,19
TM1
52,47
TM2
4,57
FAMC CODDYAC
64,74
72,42
Network
64,96
Overall
64,31
Towards a perceptual quality metric for dynamic meshes
2. Speed-weighted spatial-based perceptual distance
• We compute the mean speed (
) of reference mesh vertices in a 5 frames window
• Speed-based weighting for the spatial distance:
• Per-frame spatio-temporal distance.
• Spatio-temporal perceptual distance:
Metric Uniform Gaussian SM1
SPD
61,32
66,21
88,02
SWSPD 73,25
79,88
90,63
gipsa-lab
31
SM2
82,19
82,41
TM1
52,47
57,91
TM2
4,57
2
FAMC CODDYAC
64,74
72,42
69,23
87,48
Network
64,96
75,91
Overall
64,31
73,47
Towards a perceptual quality metric for dynamic meshes
3. Temporal perceptual distance
• Per-vertex (vector-norm-related) motion difference:
motion vector norm at
motion vector norm at
= 0.01 (stability coef.)
• Per-vertex (vector-direction-related) motion difference:
motion vector angle at
motion vector angle at
(stability coef.)
• Temporal vector-norm-related and vector-direction-related distances:
Spatial and temporal Minkowski sum of
gipsa-lab
32
and
Towards a perceptual quality metric for dynamic meshes
4. Final perceptual distance measure
with:
Spearman correlation:
Metric Uniform Gaussian SM1
SPD
61,32
66,21
88,02
SWSPD 73,25
79,88
90,63
DMPD 84,61
82,51
91,5
gipsa-lab
33
SM2
82,19
82,41
97,8
TM1
52,47
57,91
59,21
TM2
4,57
2
88,91
FAMC CODDYAC
64,74
72,42
69,23
87,48
70,11
89
Network
64,96
75,91
82,72
Overall
64,31
73,47
80,03
Conclusion and Future work
• Geometric metrics (RMS, Hd, KG, etc.) fail to predict the perceptual quality of 3D
meshes
• We present a new general-purpose, subjectively-rated dynamic meshes database
• There is no perceptual metric able to predict dynamic meshes quality after
temporal distortions
•The new perceptual metric integrates both spatial and temporal measures to
evaluate the perceptual quality of dynamic meshes
Future work:
A Machine-learning-based metric, reduced-reference metric for dynamic meshes
Textured meshes
Improve the database by adding new distortions
gipsa-lab
34
Thank you for your
attention
gipsa-lab
35
© Copyright 2024 ExpyDoc
