KERNEL NORMALIZED MIXED-NORM ALGORITHM FOR SYSTEM IDENTIFICATION
Shujian Yu1, Yixiao Zhao1,3, Xinge You2, Xiaopeng Yang3, Yuanyan Tang2,4, C.L. Philip Chen4
1
Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, USA
2
Department of Electronics and Information Engineering, Huazhong University of Science and
Technology, Wuhan, Hubei, China
3
School of Information and Electronics, Beijing Institute of Technology, Beijing, China
4
Faculty of Science and Technology, University of Macau, Macau, China
ABSTRACT
Kernel methods provide an efficient nonparametric model to
produce adaptive nonlinear filtering (ANF) algorithms.
However, in practical applications, standard squared error
based kernel methods suffer from two main issues: (1) a
constant step size is used, which degrades algorithm
performance under non-stationary environment, and (2)
additive noises are assumed to follow Gaussian distribution,
while in practice the noises are in general non-Gaussian and
follow other statistical distributions. To overcome these two
issues simultaneously, this paper proposes a novel kernel
normalized mixed-norm (KNMN) algorithm. Compared to
standard squared error based kernel methods, KNMN
extends linear mixed-norm adaptive filtering algorithms to
Reproducing Kernel Hilbert Space (RKHS), and also
introduces a normalized step size as well as adaptive mixing
parameter. The mean squared convergence analysis is
conducted. We also demonstrate desirable performance of
KNMN in the problem of system identification.
Index Terms— Adaptive nonlinear filtering, kernel
method, mixed-norm, normalized step size, system
identification
1. INTRODUCTION
Adaptive nonlinear filtering (ANF) algorithms have been
extensively investigated in recent years [1]. Different from
conventional nonlinear filtering problems, where all data is
available beforehand, ANF algorithms require update
operation to deal with incoming samples [2].
Among them, kernel based nonparametric modeling
methods [3-5] produce ANF algorithms with inner products
by employing the famed kernel trick [6]. Almost for all
current kernel-based ANF algorithms, additive Gaussian
distributed noises are assumed. However, in practical
applications [7], non-Gaussian distributed noises, such as
short-tailed distributions (uniform distribution, Bernoulli
distribution, etc.) are all commonly encountered [8].
Moreover, for non-stationary environment, the step size is
another important issue for the algorithm performance
evaluation. Meanwhile, the constant step size which is
determined a priori will obviously degrade algorithm
performance under such environment [9]. Previously works
mainly focus on the first issue. The kernel robust mixednorm (KRMN) [7] algorithm was proposed to enhance ANF
with long-tailed statistical distributed noises, while our
previously proposed quantized kernel least mean mixednorm (QKLMMN) algorithm [10] demonstrates superior
performance for ANF under short-tailed noises environment.
However, none of them take into consideration of the nonstationary environment, where a pre-determined constant
step size cannot achieve desirable expectations.
To overcome these two issues simultaneously, a novel
kernel normalized mixed-norm (KNMN) algorithm is
proposed in this paper by extending mixed-norm algorithm
to RKHS while introducing a normalized step size as well as
adaptive mixing parameter.
The rest of the paper is organized as follows. In section
2, the preliminary knowledge is briefly introduced. In
section 3, the KNMN algorithm is derived, and the
mechanism for adaptive mixing parameter is also presented.
Then, the mean square convergence is analyzed in section 4,
and simulation results for system identification are given in
section 5. Finally, section 6 concludes this paper.
2. PRELIMINARY KNOWLEDGE
2.1. Mixed-norm adaptive filtering family
The family of mixed-norm adaptive filtering was initially
introduced in [11], where least mean square (LMS) and least
mean fourth (LMF) was combined to yield a least mean
mixed-norm (LMMN) algorithm. Followed by that, a robust
mixed-norm (RMN) algorithm was proposed in [12] by
combining LMS and least absolute deviation (LAD). The
cost functions for LMMN and RMN are:
=
J (k) λ E{e 2(k)} + (1 − λ ) E{e 4(k)}
(1)
SINGLE IMAGE RAIN STREAKS REMOVAL BASED ON SELF-LEARNING AND
*
STRUCTURED SPARSE REPRESENTATION
Shujian Yu1,†, Weihua Ou2,3,†, Xinge You3, C.L. Philip Chen4, Yuanyan Tang3,4
1
2
Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, USA
School of Mathematics and Computer Science, Guizhou Normal University, Guiyang, Guizhou, China
3
Department of Electronics and Information Engineering, Huazhong University of Science and
Technology, Wuhan, Hubei, China
4
Faculty of Science and Technology, University of Macau, Macau, China
ABSTRACT
Rain streaks removal from single image is a challenging
problem for image processing. This paper proposed a novel
algorithm for rain streaks removal on single image based on
a self-learning framework and structured sparse
representation. More precisely, our algorithm firstly segment
and categorize input image into "rain streaks" regions and
"non-rain geometric" regions via texture analysis.
Meanwhile, we also decompose input image into highfrequency (HF) and low-frequency (LF) parts with bilateral
filtering. Followed that, we introduced our newly proposed
structured dictionary learning to decompose HF part into
"rain texture" details and "non-rain geometric" details,
where patches for training rain and non-rain sub-dictionaries
area automatically selected from "rain streaks" and "non-rain
geometric" regions. Finally, we combine LF part with nonrain geometric details to get rain-streaks-removal image.
Experiments demonstrate the effectiveness and efficiency of
our proposed algorithm.
Index Terms— Rain streaks removal, texture analysis,
structured dictionary learning and sparse representation
1. INTRODUCTION
Algorithms which can handle complex and unpredictable
behaviors caused by different weather conditions in outdoor
situations remain as challenging topics for image processing
[1]. Previous works for reducing the visibility of dynamic
weather effects, such as rain and snow, are mainly based on
video, where physical and photometric properties of
raindrops or snowflakes can be well employed combined
with temporal information [1-4]. Nevertheless, when only a
*This work was supported in part by the National Natural Science
Foundation of China under Grants 61402122 and 61272203.
†
The first two authors contributed equally to this work and should
be regarded as co-first authors.
single image is available, such as images taken by a camera
or downloaded from internet, algorithms for single-imagebased rain or snow removal are required [5, 6].
Problems on single-image-based rain streaks removal
are rarely studied before, and existing algorithms typically
based on image decomposition [5-7] or image filtering [8, 9].
This paper also follows the frame of image decomposition.
Traditional image-decomposition-based rain streaks removal
methods [5-7] always adopt two main steps: (1) image firstly
decomposition with bilateral filtering [16], and (2) image
high-frequency (HF) part decomposition via sparse coding
and dictionary learning [17]. As a result, "rain component"
can be extracted effectively, and the remaining part is the
rain removal image. However, a major drawback for these
methods is the reliance on a knowledgeable human operator
to identify exemplar patches used for dictionary learning.
Although a content-based algorithm for rain region detection
and segmentation with a hieratical saliency model has been
proposed in [9], this method only applicable for images
corrupted with light rain. Apart from that, traditional
methods [5-7] discriminate two learned dictionaries (one for
"rain texture" details, another for "non-rain geometric"
details in HF part) only with HOG [24] features, which
makes the process of dictionary learning lacks strict
mathematical supports and constraints.
To overcome these two issues simultaneously, this
paper proposed a novel framework (see Fig.1) for singleimage-based rain streaks removal. In our proposed
framework, we use a novel generally-applicable mechanism
for rain regions detection and categorization. Meanwhile, we
also introduce our newly developed structured dictionary
learning [15] to decompose HF part of input image based on
mutual incoherence minimization.
The rest of the paper is organized as follows. In section
II, preliminary knowledge is briefly introduced. In Section
III, the framework for our proposed single-image-based rain
streaks removal algorithm is elaborated. Then, experiments
are conducted in section IV. Finally, section V concludes
this paper.
© Copyright 2024 ExpyDoc
