Descattering Transmissive Observation using Parallel High-frequency Illumination Kenichiro Tanaka1, Yasuhiro Mukaigawa1 Yasuyuki Matsushita2, Yasushi Yagi 1 1 Osaka University, Japan 2 Microsoft Research Asia 1 Our purpose Sharpening transmissive image Normal view Metallic part in milky water Our result 2 Transmissive image What is the Transmissive Image? Light observing internal structure Foreign object Applications Inspection Industrial product Food Medium Camera Medical treatment Security 3 Nail in roll cake Vein pattern Problem in getting transmissive images Problem Unclear transmissive image Cherry Light Blurred transmissive image Reason Scattering in media Transmissive rays Descattering is important 4 Scattered rays What is the difference? Our approach Clues Polarization Angle Time Position Transmission Kept Same as incident Fast Same as incident Scattering Gradually lost Spread Delay Spread polarizer 𝑡 analyzer Treibits et al. PAMI ’08 Kim et al. ECCV ’10 Wu et al. CVPR ’12 All scattering components cannot be removed Low resolution Very high speed sensor is necessary Related Work Demerit 5 Static scene only [Nayar et al. SIGGRAPH ’06] High Frequency Illumination (HFI) Feature Projecting dense checker pattern with shifting the phase Separation Direct components Diffuse and Specular reflections Global components Inter-reflection and Scattering Projector Camera Computation Pattern projected scene 6 Direct components Global components [Nayar et al. SIGGRAPH ’06] Principle of the HFI Difference between Direct and Global components Direct components Intensities vary Global components Different Low pass filter Intensities do not vary Separation method 1 max = 𝑑𝑖𝑟𝑒𝑐𝑡 + 2 𝑔𝑙𝑜𝑏𝑎𝑙 1 min = 2 𝑔𝑙𝑜𝑏𝑎𝑙 𝑑𝑖𝑟𝑒𝑐𝑡 = max − min 𝑔𝑙𝑜𝑏𝑎𝑙 = 2 min 7 Projector Camera HFI for transmissive image Cannot work under perspective projection All rays do not overlap in space Rays overlap on captured image Projector Direct (Transmission) Intensities vary →do not vary Global (Scattering) Low pass filter Intensities do not vary Overlapped Same Unable to separate 8 Camera Captured image Our proposal: Parallel HFI Parallel projection for both projector and camera All rays do NOT overlap in space and captured image There is one-to-one correspondence between projector and camera pixels Direct (Transmission) Intensities vary →do not vary →vary Projector Not overlapped Global (Scattering) Different Low pass filter Intensities do not vary Camera Separable Captured image 9 Separation procedure Projector 1. Project a high frequency checker pattern and shift the phase 𝟏 Max = 𝑻𝒓𝒂𝒏𝒔𝒎𝒊𝒔𝒔𝒊𝒐𝒏 + 𝑺𝒄𝒂𝒕𝒕𝒆𝒓𝒊𝒏𝒈 𝟐 𝟏 𝑺𝒄𝒂𝒕𝒕𝒆𝒓𝒊𝒏𝒈 Min = 𝟐 Captured image Camera 2. Compute transmission and scattering at each pixel 𝑇𝑟𝑎𝑛𝑠𝑚𝑖𝑠𝑠𝑖𝑜𝑛 = Max – Min 𝑆𝑐𝑎𝑡𝑡𝑒𝑟𝑖𝑛𝑔 = 2 × Min Computation 10 Captured images with shifting the phase Transmission Scattering Realization of parallel projection Two approaches Paraboloid Telecentric lens or Paraboloidal mirror Focal point Iris Our selection Telecentric lens for both camera and projector Projector development kit with F-mount (Lightcommander by Texas Instrument) 11 F-mount telecentric lens (Edmund Optics) Implementation Camera Telecentric lens Polarizer Infrared projector Telecentric lens Target object Mirror Polarizer 12 Confirmation of co-axial alignment Alignment sensitivity Check the pattern gap and confirm the alignment Co-axial Out of alignment diffuser Same pattern 13 Pattern gap Confirmation of co-axial alignment Capture images at 2 different heights Check the difference 300 250 200 Upper 150 Lower 100 25mm Difference 50 0 -50 profile 14 higher lower difference Effective pattern size Pattern size trade-off Decide the pattern size for our setting In pure water Smaller Size 3x3 6x6 9x9 12x12 15x15 0.912 0.983 0.984 0.982 0.981 Metallic wire Larger Direct image Correlation Sharp Dark Noisy 15 Bright Blurred Descattering result Scene Metallic part in milky water Captured images 16 Visible normal illumination NIR normal illumination NIR parallel HFI (proposed method) Descattering evaluation Scene Metallic wire in milky water Metallic wire Captured images In pure water Cross correlation Intensity profiles along each line Visible normal illumination NIR normal illumination NIR parallel HFI (proposed method) -0.10 0.68 0.95 in pure water visible light NIR light proposed method 17 Results for various densities Density 1.9% 2.2% 2.5% 2.8% 3.1% 0.98 0.68 0.30 0.06 0.01 0.95 0.95 0.84 0.21 0.03 Normal Illumination Correlation Proposed Method Correlation 18 Artifact Improved Experimental result Scene Vein pattern of a leaf Result 19 NIR normal illumination NIR parallel HFI (proposed method) Limitations Refraction breaks parallel projection Rays overlap on captured image Optically dense materials cannot be treated No transmissive ray Refraction 20 No transmission ray Example of optically dense object Summary Parallel high-frequency illumination Separation of transmissive and scattered rays Implementation using telecentric lenses Sharpening transmissive images 21 Fin. 22 Comparison Perspective Projection Parallel Projection Rays are overlapped on image All rays are NOT overlapped Projector Projector Camera Camera Overlapped 23 Captured image Not overlapped Captured image Idea of Parallel HFI Parallel rays Camera Projector Point light Camera High Frequency Illumination Projector 24 Lens Lens Knife edge Schlieren Method Camera Captured image Parallel High Frequency Illumination Preliminary experiment about wavelength Scattering depends on wavelength Blue, Green, Red, and Infrared(IR) Transmissive image of white acrylic board Compare intensity profiles Acrylic board scene 470nm(Blue) Blue Green Red Transmissive images 25 Infrared 525nm(Green) 660nm(Red) 850nm(IR) IR has a strong transmissive effect Related Work: Various HFI method Direct components changes as setup changes direct Nayar et al. 2006 Lamond et al. 2007 Mukaigawa et al. 2011 Diffuse & specular reflection Specular reflection Single scattering Diffuse reflection Multiple scattering global Inter-reflection All scattering Each red and blue rays do not overlap in space and on the captured image 26 IC card Normal IR image 27 Proposed method
© Copyright 2024 ExpyDoc
