Hyperspectral Imaging: General Introduction
Process
Analysis &
Technology
Prof. emer. Dr. R. W. Kessler
Process Analysis and Technology PA&T
Reutlingen University, Germany
Prof. Dr. Rudolf Kessler
Prof. Dipl. Phys. W. Kessler
STZ Technology Transfer Center
Process Control and Data Analysis
72762 Reutlingen, Germany
Further Reading
R. W. Kessler, Perspectives in process analysis.
J. Chemometrics, 2013, 27: 369–378. doi: 10.1002/cem.2549
B. Boldrini, W. Kessler, K. Rebner and R. W. Kessler
Hyperspectral imaging: a review of best practice, performance and pitfalls for inline and
online applications, Journal of Near Infrared Spectroscopy 2012, 20, 438–508
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Agenda
• Concepts and trends in manufacturing
• Taxonomy of spectral imaging techniques
• Sensitivity, selectivity and robustness
• Selected examples
• Focus on wood
• Summary
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Agenda
• Concepts and trends in manufacturing
• Taxonomy of spectral imaging techniques
• Sensitivity, selectivity and robustness
• Selected examples
• Focus on wood
• Summary
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Concepts
Manufacturing and Processing Industry:
• Provides 70% of the wealth of the German society although only around 30 %
of the population work in the manufacturing industry!!!,
• 90% of IT research is financed by the manufacturing industry
•
•
•
•
•
Manufuture of the EU
PAT/QbD of the FDA
BDI 2006: 32 Thesen
Factbook 06 (VCI) „Chemie 2030 - Globalisierung gestalten“
EU’s 20-20-20 goals (20% increase in energy efficiency, 20%
reduction of CO2 emissions, and 20% renewables by 2020)
• Namur road map: Prozess-Sensoren 2015+
• World Manufacturing Forum 2012
• Industrie 4.0
Trends:
Aging of the population: medical systems
Urbanisation and megacities
Personalization of products and goods
„Internet of Things“
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Knowledge Based Production and Causality
Adaptive
Processing
First
Principles
Knowledge
based
Production
Consistent
Output
Variable
Material
Input
&
Mechanistic Models
Causality
Correlative and Descriptive Models
Fixed
Process
Variable
Output
Models
- First principles
- DoE
- Soft modelling
- Molecular markers
(specroscopy)
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
but…….. Culture …….
"Heaven is where the police are British,
the chefs French, the mechanics German
and the lovers Italian
and it is all organised by Swiss”
“Hell is where the chefs are British,
the mechanics French, the lovers Swiss,
the police German
and it is all organised by the Italians."
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Toolbox Multimodal Optical Spectroscopy
wide spectral range
I
MIR/Raman Fluorescence
NIR
UV/VIS
N
N
N
N

Fluorescence
UV
Reflection
NIR
IR
Raman


Transmission
Different
experimental
setup
x
spatial scan (x,y)
y
Hyperspectral imaging
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Key-Issue: Spectroscopy in Scattering Systems
The Basic Idea:
Absorption AND Scatter
= Chemistry and Morphology
 Use
Scatter as Information!!!
 Separate Scatter from Absorption
 Integrate this Information
into your Modelling:
e.g. MCR, SBC, Multiblock
Theory: you need more than 1 measurement e.g.
Kubelka Munk, RTE
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Toolbox Chemometrics: Increase Selectivity!!
Some information
+
redundant
information
Find small portion of
useful information
 Univariate data analysis
 Explorative multivariate
data analysis (PCA, etc.)
Some
information +
no information
 Multivariate regression and
classification (MLR, PLS,
Some information
RBF, Kohonen.......)
+
 Optical principal component
non-specific
analysis
information
 MCR, Multiblock, …
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Measurements
Data
Knowledge
Causality
The Japanese eat very little fat and suffer fewer heart attacks
than the British or the Americans.
The French eat a lot of fat and also suffer fewer heart attacks
than the British or the Americans.
The Japanese drink very little red wine and suffer fewer heart attacks
than the British or the Americans.
The Italians drink a lot of red wine and also suffer fewer heart attacks
than the British or the Americans.
Conclusion:
Eat and drink whatever you like. It's speaking English that kills you.
What is wrong?
To draw conclusions from random or spurious correlations
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Agenda
• Concepts and trends in manufacturing
• Taxonomy of spectral imaging techniques
• Sensitivity, selectivity and robustness
• Selected examples
• Focus on wood
• Summary
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Chemical/Spectral Imaging
CHEMICAL
CHARACTERIZATION
PHYSICAL/
MORPHOLOGICAL
CHARACTERICHEMICAL
ZATION
IMAGING
=
Absorption
+
Scattering
IMAGING
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Chemical Imaging Techniques
Wavelengths:
UV/VIS
2D- Fluorescence with
FLIMS
NIR
IR
Raman
Pushbroom Imaging
Simultaneous (x, )
Sequential (y)
Staring Imaging
Whiskbroom
Simultaneous (x, y)
Sequential ()

Staring
Whiskbroom Imaging
x
Sequential (x, y, )
y
Hyperspectral imaging
Specular and Diffuse Reflectance, Transmittance, Polarisation
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Advantage Whiskbroom: high flexibility
• High Optical Throughput
• High Sensitivity (high S/N), Dark Field, Bright Field
• Fast Scanning System with High Lateral Resolution
• Multimodal Spectroscopy, Transmittance, Reflectance
• Same Sample- Same Location- Different Wavelength- no optical changes
• No Photon Diffusion (Illumination = Detection)
• Easy to calibrate
• all Microscopy Techniques Optional
• High Collection Efficiency
But: time consuming
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Example Whiskbroom Imaging
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
WITEC – PA&T System
FLIM
CCD
Vis
spectrometer
2D Fluorescence
NIR
spectrometer
CCD
brightfield
Raman
spectrometer
darkfield
multi-mode
fiber
CCD
Glioblastoma
edge filter
laser
reflection
single-mode
fiber
holographic
beam splitter
nearfield image
Glioblastoma
white light
objectiv
z-stage
CCD
Glioblastoma
Nearfield unit with Solid immersion lens
(SIL)
piezo
scanner
SNOM
unit
transmission
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Staring Imaging
0.35
0.30
Extinktion
0.25
0.20
0.15
0.10
0.05
1000
advantage
1100
1200
1300
1400
1500
1600
1700
Wellenlänge [nm]
- high lateral resolution
- easy to implement
- high information density
- but: motion stop needed
- but: calibration and focus difficult,
homogeneous illumination may be difficult
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
1800
1900
2000
Staring Imaging:
Contrast enhancement by
absorption
650 nm

550 nm
450 nm
1.200
1.100
1.000
Absorption
0.900
0.800
0.700
0.600
0.500
0.400
0.300
0.200
420
440
460
480
500
520
540
560
580
600
620
640
660
680
700
720
Wellenlänge [nm]
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Laterale Achse
Pushbroom Imaging
Extinktion
Spektrale
Achse
advantage
- flexible inline and real time applications
- no motion stop needed
- good compromise between spatial and wavelength resolution
- but: different optical and spatial resolution in x- und y- direction
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Inline: Pushbroom Imaging Technology
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
SNAP SHOT Imaging
DEMO video available on:
http://vimeo.com/77218620
Data Cube in Spectral Imaging

(2nd spatial dimension)
(time coordinate,
if sample is moving)
y-axis
Spectrum for one pixel
x-axis
(1st spatial dimension)
Distribution map for one wavelength
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Agenda
• Concepts and trends in manufacturing
• Taxonomy of spectral imaging techniques
• Sensitivity, selectivity and robustness
• Selected examples
• Focus on wood
• Summary
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Example Darkfield Glioblastoma Vis Backscattering Light RGB: TP53
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Example Label Free Karyotyping by Backscattering
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Enhanced Selectivity: Derivative Spectroscopy: UV-Vis, NIR…..
concentration A + 2B
concentration A + B
original
1st derivative
concentration A
be aware:
E = h c/λ
200nm = 50 000 cm-1
250nm = 40 000 cm-1
Δ = 50nm, Δ = 10 000 cm-1
2nd derivative
Δ Raman, MIR app.
4000 cm-1!!!!
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Robustness: Specular and Diffuse Reflectance of a Cavity on a Surface
parallell
high
concentration ?
Model System:
cellulose/
dyed cellulose
high lateral
resolution!!
crossed
Specular
Reflection
I0
Δn
low lateral
resolution due to
photon diffusion!!
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Specrograph
light
source
sample moving direction
45R45
light
source
sample moving direction
45R0
CCD
Array
diffuse
light
source
Specrograph
CCD
Array
CCD
Array
Specrograph
Robustness: Inline Illumination – Detection Set Up
sample moving direction
dR0
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Inline Illumination
pushbroom imager
light source
diffusors
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Absorption and Scattering: Penetration of Photons
Real Life ASA tablet
measured
calculated
R
T
Penetration Depth!!!!
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Scale of Scrutiny
0.3
Si-based CCD
mixed???
Penetration depth/cm
0.25
281 Mpa
156 Mpa
0.2
InGas-based detectors
0.15
3rd
overtone
in NIR
0.1
many small
measurement spots
are better than one
large spot in
spectroscopy!!!
0.05
0
400
600
800
1000
1200
1400
1600
1800
2000
wavelength/nm
Optical penetration depth of Theophyllin tablets with different
API concentrations, calculated from S and K
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Agenda
• Concepts and trends in manufacturing
• Taxonomy of spectral imaging techniques
• Sensitivity, selectivity and robustness
• Selected examples
• Focus on wood
• Summary
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Chemical Imaging Plastic Sorting
HELIOS NIR/SWIR
Smart Camera System
• Sensor Data Processing
HELIOS
–
–
Smile-, Keystone-, Black Drift Corr.,
Defectpixel Elim., Calibration, etc...
• Spectra Preprocessing
–
Derivative, Norm., etc...
• Feature Extraction
–
Mean Intens., Baseline Slope, etc..
• Feature Combination
Illumination
–
Metal Sensor
FG / BG Segmentation, etc...
• Classification
–
Spectra, Colour
• Object Processing
–
Colour, Shape, Size, Structure, Material
• Decision
–
Conveyor belt and/or chute
for material transportation
e.g. Control of Air Valves
Hard Real Time Processing of
> 80 000 high-res. Spectra / Second
> 300 high-res. Spectral Images / Second
Valve Block
DL
AW
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
34
e.g. PP/PES in Non Wovens: enhanced Selectivity using Chemometrics
1.2
120
0/100
100
0.6
70/30
predicted
75/25
PC 2
PLS
80
0.0
60
40
50/50
20
100/0
0
-0.6
20/80
0
20
40
60
80
100
120
measured
0.8
PP/PES
-1.2
-2
-1
0
1
PC 1
2
E
0.6
0.4
PP
PES
0.2
~210°
C
0.0
1000
1200
1400
1600
1800
 [nm]
NIR spectra
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
2000
2200
2400
2600
Imaging Thin Films on Aluminium
(b)
100
R [%]
80
60
Messung
Simulation
40
- ein.Winkel = 30°
- dt = 445 nm
- n2 = 1,35 k2 = 0,01
20
0
300
400
500
600
700
Wellenlänge [nm]
250 nm
645 nm
415 nm
705 nm
550 nm
805 nm
brightfield illumination with polarisation, magnification for visualisation 500
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
800
Counterfeit Detection through blister packaging (N.Lewis, Malvern)
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Key Issue: Sample Presentation: Food and Feed (e.g. Bühler)
see also: 60 000 rice kernels/sec!!
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Multiplexing: e.g. Reactive Extrusion resp. Hot Melt Extrusion
Entrance slit with
fiber optic mounting
prism / grating / prism
Camera
λ
x
optics
Pushbroom Imaging System
reactive Extrusion
Hot Melt Extrusion
Reaction
Tomography!!!
Courtesy of Rottendorf Pharma GmbH
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Multipoint Spectroscopy
Extend to many measurement positions
by using several fibre bundles
Very flexible, adjustable to any need
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
2mm thickness
Coated
Tablet
2mm thickness + Coating
I0
 = 80 cm-1
 = 5 cm-1
API
Absorbance
decreases
with coating
thickness
But:
Increases
with
distance
Illumination
Tablet
Illumination
Photon Diffusion Spectroscopy
I0
 = 180 cm-1
 = 2 cm-1
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
!
Reaction Tomography in a Microreactor
Camera
x, s
Absorbance
Pushbromm
Imager
Objective
Microreactor
Microreactor
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Optical Spectroscopy and Reactor Tomography
Illumination
Catalyst
Pushbroom
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Agenda
• Concepts and trends in manufacturing
• Taxonomy of spectral imaging techniques
• Sensitivity, selectivity and robustness
• Selected examples
• Focus on wood
• Summary
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Example: dry- and wet processing of wood
Dry
Process
Drying
Non woven
Moulding press
Wood
chips
Blow line
Defibrator
Hot
press
Vat
Fibre
mat
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Wet
Process
Concept: Multi Information Manufacturing
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
In-Line Control during Manufacturing: Control of Flutter
Diffuse Reflectance Probe and Spectral Imaging
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Morphology and Chemistry
reminder: sensitivity absorption and scatter
Spruce
Beech
Cross-section 20 µm
magnification 100 x
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Spectra 370-600nm, Selection
Absorbance
process variables: DoE at the production plant
• severity steam treatment (temperature-time)
• severity mechanical refining (type, distance, rotation)
• Wood mixture (spruce, spruce with bark, spruce/beech)
original
1st derivative
3rd derivative
2nd derivative
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
PCA of Vis-Spectra of Fibreboards: DoE at the Plant
380-700 nm
reflectance
PCA-Analysis
of Spectra



Clustering in wood mixtures
Classification of fineness
Classification of Severity factor (SFC)
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Distribution of a Resin on Wood Chips OSB
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Summary Spectral Imaging: The Benefit for Society
Agro-Food-Manufacturing-Process
Industry
Health
grain quality inspection
neonatal and fetal brain diagnostic (non invasive and painless )
identification of contaminants in soil, animal
products and food
surgery monitoring
detection of tooth decay
food safety and authenticity
Earth observation
Environment
disaster monitoring
plastics sorting and recycling
water resource management
detection of dangerous waste
climate change
observation
monitoring pollution in
enviromental air and water.
NASA/Goddard
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University
Thank You for Your Attention
Acknowledgement
Many thanks for their active support to our PhD students at Reutlingen
Research Institute, Reutlingen University
Karsten Rebner
Tobias Merz
Barbara Boldrini
Edwin Ostertag
Lieselotte Barac
Sören Hummel
Anita Lorenz
Sabrina Luckow-Markgraf
…. and many others
Steinbeis-University Berlin: Prof. W. Kessler
ILM Ulm: Prof. Dr. Hibst, Prof. Dr. Kienle
University of Tübingen: Prof. Dr. D. Oelkrug
Finacial Support by BMBF, Landesstiftung BW, EU, ... Industry......
© Prof. Dr. Rudolf Kessler, STZ Prozesskontrolle und Datenanalyse, Process Analysis & Technology, Reutlingen University

Precision Metrology Lab.