Quality assessment of satellite and radiosonde data

Analyzed RadioSoundings Archive (ARSA)
Presentation GEWEX_GVap Conf. Berlin
9-10 October, 2014
Quality assessment of satellite and radiosonde data
N.A. Scott (*)
On behalf of
A. Chédin, L. Crépeau, J. Pernin,
R. Armante, V. Capelle
(*) Contact @LMD
Analyzed RadioSoundings Archive (ARSA)
ARSA PART 1 :
Why ARSA?
Description and Current Status
WHY ARSA?
To enter the ARA/LMD protocol (*) combining forward modelling and observations for the
characterization of the various types of errors in the processing of L1 to L2 products.
(*) Since mid 1990s and NOAA/NASA TOVS Pathfinder Programme
Off Line Info
GEISA
TIGR
ARSA
4A/OP
« clear sky »
collocations
BT
SIMULATIONS
OBSERVATIONS
IASI, HIRS, AMSU, AIRS,
BT Residuals
Simul-Obsv’d
http://ara.abct.lmd.polytechnique.fr/
And also for 4A/OP:
http://4aop.noveltis.com/
BT instr « i »
- BT instr « j »
Red arrows: stand alone
Green arrows: intercalibration
From 1993 to the current version of ARSA…
Within the frame of the participation of ARA group (Chédin et al (1985) and Scott et al,
(1998)) to the NOAA/NASA Tovs Pathfinder Programme and subsequent works on
instruments of the NOAA series, AIRS/Aqua, IIR/Calipso, IASI/MetOp,…, a description
of the atmospheric and surface states has become necessary to support the level1 and
level2 processing of these instruments.
So, in the mid 1990’s, starts at LMD the
Extraction of Radiosonde reports from the ECMWF Archive  ARSA
Many Thanks to ECMWF for making us available the RS archive
The real world of Radiosondes Reports  The Fully Automatized ARSA quality
control procedures (*)
•
Format problems, redundant RS and levels, unrealistic jumps
• Physically implausible values,
• Internal inconsistencies among variables,
• Climatological outliers,
• Temporal and vertical inconsistencies in temperature, dew point temperatures.
 The detection of any of the above mentioned error can lead to reject
- The value
- The level
- The full Radiosonde Report
•
•
•
- 22 millions of RS processed
5 millions of RS accepted for subsequent steps of QC
- 480 millions of measurements processed 230 millions of measurements accepted for
subsequent steps of QC
10,583 stations from 11,742 stations
(*) See first description in L. Crépeau, ARA/LMD Internal note, November 1999
ARSA : Further work and QC are required on the raw RS to answer the
requirements of the radiative transfer community
Main Requirement: QUALITY + QUANTITY OF INFORMATION
The RS reports kept after a first round of QC are « reliable »
Now, we have to make it fully compatible with the forward radiative transfer
simulations
- Be complete
- Cover the full range of active pressure levels
- Relevant discretization in pressure
Check that every single radiosonde report satisfies
Quality+Quantity requirements to interface the Radiative transfer model (4A in our
case, or any other)
ARSA
Second step: Constraints on
the number and distribution
of the RS pressure levels
Number of
temperature
levels > 20
Number of
h2o levels
> 15
See also L. Crépeau, 1999
yes
Log(pressure)
difference between
2 consecutive pressure
levels
Ln(dp)
Ln(dp)
Ln(dp)
Ln(dp)
Ln(dp)
yes
Top temperature
pressure level
< 30 mb
Top h2o pressure
level
< 300 mb
yes
Surface pressure
Land > 850 mb
Surface pressure
Over sea > 950 mb
yes
<
<
<
<
<
Keep if :
0.15 (1050-800 mb)
0.30 (800-200 mb)
0.40 (200-35 mb)
0.60 (35-1 mb)
0.90 (1-0.01 mb)
Specification of the atmospheric and surface states
Required for the radiative transfer simulations:
Ozone profiles
Before 1999, Mc Peters, (1994)
In 1999: The 1985-1989 Ozone UGAMP climatology (Li et al, 1995).
http://www.badc.rl.ac.uk/cgi-bin/data_browser/mget/badc/ugamp-o3-climatology/data
Since 2010 : ERA_Interim Ozone profiles
- a) starting with a climatology based on ERA_Interim Ozone profiles
- b) closest profile to the radiosonde in a space time window of 100km*3H
Spectacular improvement of the statistics (bias and standard deviation) on the residuals of (simulated-observed) BTs
Extrapolation of RS to the top of the atmosphere
Up to 0.05 hPa: based on the « ara/LMD in-house » extrapolation approach
Since 2011: From ERA_interim based profiles (up to 0.1 hPa)
Above 0.1hPa : ACE_FTS level2 results
Surface state
Surface temperature from ERA_Interim (0.75*0.75): closest value within a 3 hour time window
ARSA
From the raw radiosonde measurements extracted from ECMWF up to the converged ARSA
product and after several (fully automatized) severe quality control and extrapolation
steps.
A 43-level description of the atmosphere between surface and 0.0026 hPa
including P, T, H2O, Ozone profiles, Surface temperature, Geolocation +
date/time
ARSA starts in January 1979 and is extended onwards on a monthly basis @ LMD
ARSA Metadata
•
•
•
•
List of Radiosonde stations, Code, …
Rejected Radiosonde (after QC-1)
Raw Radiosonde reports kept (all levels)
Monthly statistics on all the rejections at the end of the QC process
ARSA is available upon request at LMD First external distribution in April 2010. So far ~ 16 external users.
ARSA profiles : number and location: January 1979 to December 2013
stations in most areas of the globe:
- spatial coverage is most complete in Europe and northern America
- sparsest in northern Canada,
- quite poor in Antarctica, equatorial Africa and America, India and mid-lat western Europe.
VALIDATION OF THE ARSA DATABASE
Interactive and iterative process between Satellite data observations and simulations
IASI Satellite data
Simulated IASI BTs
Residuals
4A/OP Radiative Transfer model
ARSA : Specification of the
surface/atmospheric state
Due to the excellent stability of the IASI radiances and the accuracy of the 4A/OP model, ARSA profiles
are empirically adjusted in order to improve statistics (bias, standard deviation) between simulated and
observed IASI radiances .
It has to be noticed that the spectral resolution (0.50 cm-1, apodized) and continuity (645 to 2760 cm-1) of IASI
spectra helps doing these adjustments in a coherent way.
The comparison to satellite data records is also done for AMSU-B , MHS (microwave spectral domain) and HIRS
(Thermal Infra Red) radiance space.
ARSA profiles combined with ARA_Interim H2O and Temperature: current status
•Above 380 hPa, water vapour ERA_Interim
•!!!! However: a linear correction to the ERA_Interim water vapor profile
is necessary
•170 hPa :
•
270 hPa :
•
400 hPa :
- 0 % EraInterim
- 20 % EraInterim
- 0 % EraInterim
•Above 37hPa : Temperature from Era_Interim
NEXT SLIDES SHOW
THE POSITIVE IMPACT ON IASI BTs RESIDUALS BASED ON ARSA
HAVING COMBINED
RAOB+ERA_INTERIM +
EMPIRICAL ADJUSTMENT ON WATER VAPOR PROFILES
RESULTS FOR IASI
RESULTS FOR HIRS4 MHS
IMPACT OF THE CURRENT VERSION OF ARSA ON THE RESIDUALS OF METOPA/IASI
IASI Band II : WATER VAPOR : 6.3 mm
NEW Blue  ARSA (h2o = RAOB+ERA_INTERIM + empirical correction+ ACE_FTS )
OLD Red  (h2o = RAOB + « OLD » EXTRAPOLATION)
Impact on TBs residuals of two successive versions of ARSA for Water vapor profiles
(Cont’d)  TROPICAL AIR MASS
METOPA/HIRS channel 11 (left) and channel 12 (right)
Time series (July 2007 to March 2009) of monthly mean of calc-obs TBs residuals
LEGEND :
V2.7 Residuals computed with the current version of ARSA (Red)
V2.5 Residuals computed with ARSA, prior to the empirical correction made on ERA_Interim H2O profiles
(Blue)
Nb of Items (Green) – right y-axis
Impact on TBs residuals of two successive versions of ARSA for Water vapor profiles
(contin’d)  TROPICAL AIR MASS
METOPA/MHS channels 3 (left) , 4 (right) , 5 (bottom)
Time series (July 2007 - March 2009) of monthly mean of TBs residuals for Land/Day Case
LEGEND :
•V2.7 Residuals computed with the
current version of ARSA (Red)
•V2.5 Residuals computed with ARSA,
prior to the empirical correction made on
ERA_Interim H2O profiles (Blue)
•Nb of Items (Green) – right y-axis
POSITIVE IMPACT ON THE RESIDUALS BASED ON ARSA
WHEN ARSA COMBINES
RAOB+ERA_INTERIM +
EMPIRICAL ADJUSTMENT ON WATER VAPOR PROFILES
IS THIS STILL TRUE FOR OTHER AIR MASSES THAN THE TROPICAL
AIR MASS?
METOPA/MHS channels 3 (left) , 4 (right) , 5 (bottom right)
Time series (Jan 2008 – Dec 2008) of monthly mean of TBs residuals for Land/Day Case
MID LAT NORTH +30 - +60°
LEGEND :
•V2.7 Residuals computed with the
current version of ARSA (Red)
•V2.5 Residuals computed with ARSA,
prior to the empirical correction made on
ERA_Interim H2O profiles (Black)
•Nb of Items (Green) – right y-axis
Stability of the ARSA database
Other periods, other platforms than IASI and MetOp
ARSA relies upon raw RS reports associated to auxillary data sets
(ERA_Interim, ACE/FTS level2 products).
As a consequence its stability is closely related to the stability of these
individual datasets
A way to assess a “certain part” of its stability is to check wether or not
results based on ARSA also improve the BTs residuals of other
instruments/satellite/periods
NOAA 10
NOAA 11
NOAA 15
May 1989 to April 1991
January 1990 to December 1991
February 2001 to January 2003
Periods and satellites chosen with DWD within the frame of the QUality Assessment of SAtellite and
Radiosonde data (QUASAR) contract
Stability of the Analyzed RadioSoundings Archive (ARSA) database
We have verified that the empirical correction (drying) of the ERA_Interim water vapour profiles
between 350 and 200 hPa, we had found required to improve the quality of the IASI/MetOpA (July
2007  December 2013) residuals in the tropics is also required for other satellites at other
periods: from left to right: NOAA10, NOAA11, NOAA15 (HIRS channels 11 and 12)
Analyzed RadioSoundings Archive (ARSA)
ARSA PART 2: Towards other Applications / Users?
A reference for GEWEX Gvap?
QUality Assessment of SAtellite and Radiosonde data
(QUASAR)
Comparisons between ARSA and
IGRA_Homogenized
EUMETSAT Satellite Application Facility on Climate Monitoring
Marc Schröder (DWD),
SAF_CM_CDOP2_nnn_Quasar_v2.4k
GEWEX Data and
Assessments Panel (GDAP)
QUality Assessment of SAtellite and Radiosonde data (QUASAR)
Comparisons between ARSA and IGRA_Homogenized
The inter-comparison of the ARSA, and homogenised IGRA data records has been
carried out using collocated radiosonde reports over the period January 1979 to
December 2010 .
A large dataset common to the two databases has been identified, covering this long
period of time: more than 2,800000 radiosondes reports of 940 stations.
It also includes collocated raw radiosonde reports at full vertical resolution (from
ECMWF archive)
The IGRA data base is described in (Durre et al., 2006; Durre and Yin, 2008)
and has been homogenized (IGRA_H) as described in Dai et al. (2011).
QUASAR By-Products
940 stations are finally kept for the statistics (mean, standard deviation)
-
All stations, per standard level:
ARSA – IGRA_H
ARSA
IGRA_H
-
Per standard level, per station:
ARSA – IGRA_H
ARSA
IGRA_H
-
Per deep layer, per station:
ARSA – IGRA_H
ARSA
IGRA_H
-
Stats temperature at various levels, in K
Stats on Deep Layers Precip Water All stations
Time series of ARSA and IGRA_homogenised water vapour
Time series of ARSA and IGRA_homogenised water vapour products compared to raw
radiosondes reports
QUASAR : RESULTS
The two bases ARSA and IGRA_homogenised temperature and water vapour products have been
considered
either in stand alone
or in inter-comparison
or also, each of them two in comparison with the raw radiosonde reports.
Preliminary remarks – however obvious –:
• Comparing or inter-comparing products like water vapour with such a high 4-D natural variability (high
standard deviation at each level, for each station, along any period of time) is very difficult.
• Any misunderstanding of the way these products have been obtained may impact the conclusions of the
comparison :
- IGRA_homogenised  Direct reading of standard levels values + homogenisation
- ARSA  BAU processing followed by interpolation to the IGRA_H standard levels
ARSA AND IGRA_homogenised Comparisons : RESULTS
In stand alone, :
ARSA and IGRA_homogenised have very comparable statistics (mean and standard deviations) from the
1000 to ~400 hPa pressure levels.
For the differences occurring above this level, it has to be recalled that ARSA has been extrapolated with
ERA_Interim values and furthermore that its water vapour is the result of an empirical adjustment based on
the study of simulated-observed brightness temperatures values ( IASI/MetOpA residuals).
When inter-compared :
Values:
ARSA and IGRA_homogenised differences occur at two places of the pressure grid: one above 400 hPa the
other near the surface. For the “above 400 hPa” , the explanation given above for ARSA is still valid.
For the pressure region near the surface, interpolation and extrapolation of ARSA profiles near the surface
pressure may be at the origin of discrepancies between the two bases.
Stability: ARSA relies upon raw RS reports associated to auxillary data sets (ERA_Interim, ACE/FTS level2
products). As a consequence its stability is closely related to the stability of these individual datasets
IGRA_homogenised relies upon the homogenisation of temperature and water vapour products
Conclusion on the stability requires further work and discussions with IGRA_homogenised people.
Analyzed RadioSoundings Archive (ARSA)
Presentation GEWEX_GVap Conf. Berlin
9-10 October, 2014
ARSA PART 2: Planned further developments
ARSA: Conclusions and Planned further developments
Main conclusions:
• ARSA plays an important role in the radiative transfer community (forward and inverse) based
on its continuous validation process.
•
Based on the improvements of the values of the BTs residuals in water vapor dependent
channels of diverse instruments/ periods/ platforms, ARSA is a potential reference dataset
•
Interaction of ARSA with homogenisation programmes would reinforce the coherence of the
various reference datasets.
What do we plan to further improve it?
•
Increase the vertical resolution of ARSA near the surface (based on the study of the residuals
of IASI window channels or weakly absorbing channels)
 keep the full resolution of the RS near the surface
•
Make available the METADATA files to other users
•
Correct the spurious trend induced by the assimilation process in ERA_Interim ozone
•
Develop an approach to an ARSA homogenisation, taking advantage of the performance of
the 4A/OP model and of the excellent radiative stability of IASI ( MetOpA and B, and even
over a longer period of time with MetOpC and IASI-NG, …).

Download Report