Operational Weather Forecast Verification at India Meteorological Department B. P. Yadav, Naresh Kumar, L.S. Rathore OUTLINE • Introduction • Operational Forecast Services • Performance of Operational Forecasts • Conclusions Regional Centres Delhi, Kolkata, Guwahati, Nagpur, Mumbai, Chennai NWFC Delhi/ WF Pune Six Regions 19 Meteorological Centers North Region East Region North East Region Central Region West Region South Region Pune Centre for Research, Training, Climatology, Agromet and surface instrumentation Nowcast, City and Tourism Forecast Short Range Forecast & Warnings District /Block level Quantitative Medium Range Forecast Other forecast and warnings Extended Range Prediction (Week 1, Week 2) Long Range Forecast for Southwest & Northeast Monsoon rainfall and winter Precipitation Cyclone track, intensity & landfall forecast Fog Forecast Venue Specific Forecasts Highway, Expedition, pilgrimage forecast Basic measures/scores Correct Non-occurrence Hit Rate, Probability of Detection a H = POD = (a + c) Range: 0 to 1, Perfect score = 1 C − NON = d (b + d ) Range: 0 to 1 Perfect score = 1 False Alarm Rate •Miss Rate, complement score b F= (b + d ) M =1-H= c/(a+c) Range: 0 to 1 Perfect score = 0 Frequency Bias Index (Bias) FBI = B = ( a + b) (a + c) ∞ Range: 0 to Perfect score = 1 Percentage Correct (a + d ) PC = n Range: 0 to 1 Perfect score = 1 Observed Forecast YES NO YES NO a (Hits) b (False Alarms) c (Misses) d (Correct no events) Basic measures/scores Threat Score, Critical Success Index a TS = CSI = (a + b + c) Range: 0 to 1 Perfect score = 1 No skill level = 0 •poorer scores for rarer events Heidke Skill Score 2(ad − bc) HSS = [(a + c)(c + d ) + (a + b)(b + d )] ∞ Range: to 1 Perfect score = 1 No skill level = 0 Observed Forecast YES NO YES NO a (Hits) b (False Alarms) c (Misses) d (Correct no events) Error Structure for City/ Tourism forecast Temperature + 10 C > + 10 C Correct Incorrect Weather Yes/No Intensity of Rainfall Error Structure for Verification of District forecast RAINFALL Diff ≤ 25% of observed Correct 25% of observed < Diff ≤ 50% of observed Usable Diff > 50% of observed Unusable TEMPERATURE + 10 C Correct + 20 C Usable >+ 20 C Incorrect RELATIVE HUMIDITY + 10% Correct + 20% Usable >+ 20% Incorrect Cont… Wind direction + 30 deg Correct + 40 deg Usable >+ 40 deg Incorrect Wind speed + 2 m/s Correct + 4m/s Usable > + 4m/s Incorrect Cloud cover + 2 okta Correct + 3 okta Usable >+ 3 octa Incorrect CRITERIA for Classification of Forecast Good- If forecasts are correct and usable for ≥ 70% of days of the season for individual district. Moderate- If forecasts are correct and usable for 50-70% of days of the season for individual district. Poor- If Forecasts are correct and usable for < 50% of days of the season for individual district. Name FORECAST VERIFICATION AT IMD Theme Dr. S.K. Roy Bhowmik MME (Oral) Y.V. Rama Rao TIGGE, TC (Oral) B.P. Yadav, Naresh, L.S. Rathore SRF , Lead Dr. Kamaljit Ray Nowcast (Oral) V.R. Durai NWP (Oral) Neetha Gopal Aviation (Oral) M.I. Ansari Observation (Oral) Dr. M. Mohapatra Cyclone (Poster) Dr. R.K Jenamani Fog (Poster) D.S. Pai LRF (Poster) B.P. Yadav & Charan Singh ICC Cup (Poster) Dr. D.R. Pattanik ERF (Poster) Soma Sen Roy NWP(Poster) S.D. Kotal NWP(Poster) Dr. S.I. Laskar NWP(Poster) Ananda Kumar Das NWP (Poster) Verification of City rainfall forecast percentage correctness 95 90 85 2010 80 2013 75 70 WINTER PRE MON MONSOON POSTMON Seasons West Bengal Verification of Value added Rainfall Forecast 2013 DAY WISE SKILL SCORE VALUE FOR OBSERVED VS VALUE ADDED R/F SKILL SCORE IN PERCENTAGE 92 90 88 day1 86 day2 84 day3 82 day4 80 day5 78 76 74 CHB DJG MLD NAD E.MDP W.MDP Multi-model ensemble (ECMWF, NCEP CFSv2 and JMA) 16 Jan, 2010 17 Extended range Forecast skill for All India Rainfall during last 4 years (2010-2013) Correlation Coefficient (CC) 1.0 0.8 0.6 0.4 0.2 0.0 2010 (MME) 2011 (MME) 2012 (MME) 2013 (MME) -0.2 Week 1 (5-11 Days) Week 2 (12-18 Days) Forecast Days Week 3 (19-25 Days) Performance of All India Rainfall LRF (1988-2012): During 7 years error was ≥ 10% with highest during 2002 (20%), 1994 (18%). Average Abs Error of LPA Monsoon 2013 1988-2012 = 7.95% Forecast 98 ± 4 % 1993-2002 = 9.3% Observed 106% 2003-2012 =6.6% During 1993-2002, forecast was within ±4% of actual values during 2 years. During 2003-2012, forecast was within ±4% of actual values during 5 years. Forecasting date of Monsoon onset over Kerala (Model error = 4 days) Year Actual Onset Date Forecast Onset Date 2005 7th June 10th June 2006 26th May 30th May 2007 28th May 24th May 2008 31st May 29th May 2009 23rd May 26th May 2010 31st May 30th May 2011 29th May 31st May 2012 5th Jun 1st Jun 2013 1st Jun 3rd Jun 21-Mar-14 Track forecast error (km) Average error in km (2009-13) Lead Error 12 hr – 68.5 36 hr – 163.8 60 hr – 233.8 550 500 450 400 350 300 250 200 150 100 50 0 Lead Error 24 hr- 124.1 km, 48 hr- 202.1 km 72 hr- 268.2 km Trend (km/year) in improvement in track forecast (2003-13) Lead Error Lead Error 12 hr- -4.8 24 hr- -6.9 36 hr- -30.9 48 hr- -59.0 60 hr- -78.8 72 hr- -78.8 12 hr forecast error(km) 24 hr forecast error (km) 36 hr forecast error (km) 48 hr forecast error (km) 60 hr forecast error (km) 72 hr forecast error (km) Linear (12 hr forecast error(km)) 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Track forecast skill (%) Average Skill (%) - (2009-13) Lead Error Lead Error 12 hr- 31.2 24 hr- 35.9 36 hr- 43.9 48 hr- 52.6 60 hr- 58.1 72 hr- 61.8 100 90 80 70 60 50 40 30 20 10 0 -10 -20 -30 -40 -50 -60 -70 Trend (% per year) in track forecast skill (2003-13) Lead Error Lead Error 12 hr- 7.6 24 hr- 3.7 36 hr- 15.3 48 hr- 14.5 60 hr- 14.1 72 hr- 15.0 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 12 hr track forecast skill (%) 24 hr track forecast skill (%) 36 hr track forecast skill (%) 48 hr track forecast skill (%) 60 hr track forecast skill (%) 72 hr track forecast skill (%) Mean landfall point forecast error Average error in km (2009-13) Lead Error Lead Error 12 hr - 38.8 24 hr- 75.0 km, 36 hr - 94.5 48 hr- 97.5 km 60 hr – 83.8 72 hr- 123.7 km 600 500 12 hr 48 hr Linear (12 hr) Trend (km/year) in improvement in Landfall time forecast (2003-13) Lead Error Lead Error 12 hr- -14.5 24 hr- -31.2 36 hr- -16.6 48 hr- -6.5 60 hr- -04.9 72 hr- -09.1 24 hr 60 hr Linear (24 hr) 36 hr 72 hr 400 300 200 100 0 -100 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 Accuracy of Dense Fog Forecast PC- January 2014 PC- January 2014 PC- January 2014 FOG Day-1 FOG Day-2 FOG Day-3 Region No. of Dense Fog Days Region <10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 >90 NA No. of Dense Fog Days East U.P. 26 Bihar 21 West U.P. 28 West M.P. 22 Haryana 30 East M.P. 21 Punjab 21 Jharkhand 05 West Rajasthan 14 SHWB 22 East Rajasthan 18 GWB 05 Skill of Cold Day Forecast PC - January 2014 PC - January 2014 Cold Day - Day-1 Cold Day - Day-2 <10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 >90 NA <10 FAR - January 2014 FAR - January 2014 Cold Day Day-1 Cold Day Day-2 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 >90 NA Skill of Short Range Forecast and Warnings 36 Meteorological Sub-Divisions Content of Bulletin Terminology Threshold limits of all Observations, Satellite, extreme weather warnings Radar, Synoptic defined Monsoon, Thunderstorm, Cold, Fog, Heat, Cyclone The words like “likely”… done away Watch Probabilistic words like Textual as well as Graphical “could, may, would, will” are used with definition: Day-wise 3 Days Category Probability (%) Forecast and 3 Days warnings Could occur 1-25 4 days outlook May occur 26-50 Would occur 51-75 Updated 4 times a day Will occur 76-100 Spatial distribution of Met. sub-divisional rainfall Spatial distribution Area of the subdivision getting rainfall Dry Mainly dry No rain in the subdivision < 2.5 mm reported in any station Isolated Scattered Fairly widespread 1- 25% stations report rainfall 26% - 50% stations report rainfall 51% - 75% stations report rainfall Widespread 76% - 100% stations report rainfall Verification of spatial distribution of rainfall Observed Range Dry/MD Isolated Scattered Fairly widespread Wide spread Total Forecast range Dry/MD Isol Scat FWS WS Total a f k p u O b g l q v P c h m r w Q d i n s x R e j o t y S J K L M N T PC = ((a+g+m+s+y)/T)*100 CSI = a/(J+O-a), g/(K+P-g), m/(L+Q-m), s/(M+R-s), y/(N+S-y) HSS= (a+g+m+s+y-(JO+KP+LQ+MR+NS)/T)/ (T-(JO+KP+LQ+MR+NS)/T) Spatial Distribution PC - Monsoon 2013 5 Category PC - Monsoon 2013 Yes/No Category PC - Monsoon 2013 3 Category <10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 Spatial distribution rainfall Dry No rain Mainly dry < 2.5 mm Isolated 1- 25 % Scattered 26% - 50% Fairly widespread 51% - 75% Widespread 76% - 100% Spatial distribution rainfall Dry/ MD < 2.5 mm Isol / Scat 1- 50 % FWS/ WS 51% - 100% 81-90 >90 NA YES/NO FORECAST Heidke Skill Score INTENSITY OF RAINFALL Intensity Amount of rainfall Dry Very light Light Moderate Rather heavy Heavy Very heavy Extremely heavy No rain in the subdivision < 2.5 mm 2.5 – 7.4 mm 7.5 – 24.4 mm 24.5 – 64.4 mm 64.5 mm – 124.4 mm 124.5 mm – 244.4 mm 245 mm or more Exceptionally heavy If the rainfall is higher than or nearer to the earlier record over the station. However, the rainfall should be 13 cm or more Heavy Rainfall during Southwest Monsoon in India Weather systems Regions Off-Shore Trough/ Vortex, LLJ West Coast ITCZ & NWly moving Depressions/ Lows/ Cycirs (Active monsoon) East, Central, North Peninsular, NW India Mid-tropospheric circulation Mainly Gujarat Interaction of Wly and Ely Systems Western Himalayas & Northwest plains ITCZ close to Foothills, N-S trough ( Break monsoon) Northeastern States & adjoining East India Secondary ITCZ with Wly moving Cycirs Southeast Peninsula CONTINGENCY TABLE Observed Forecast YES YES NO a (Hits) b (False Alarms) c (Misses) NO d (Correct no events) forecast M observations CR H F Heavy rainfall events during the southwest monsoon season. HEAVY RF > 10 CM Very HEAVY RF > 15 CM Rajeevan et al. 2008, Geophys. Res. Letters FREQUENCY OF VERY HEAVY RAINFALL EVENTS (R >=15 cm) OVER CENTRAL INDIA (20-29N, 72-88E) 45 Mean (1901-1975) = 12.3 Mean (1976-2007) = 18.1 40 35 25 20 15 10 5 2001 1997 1993 1989 1985 1981 1977 1973 1969 1965 1961 1957 1953 1949 1945 1941 1937 1933 1929 1925 1921 1917 1913 1909 1905 0 1901 Frequency 30 YEAR Rajeevan et al. 2008, Geophys. Res. Letters HEAVY RAINFALL TREND OVER CENTRAL INDIA (1°-BY-1°GRIDDED DATA OF IMD BASED ON 1803 STATIONS (90% DATA AVAILABILITY) FOR 1951–2000) HEAVY RAINFALL ( ≥64.5 MM) TREND OVER INDIA (1901-2005) Increasing trend Decreasing trend Significant level 95% Guhathakurta et al (2011), J. Earth Syst. Sci. Increasing trend Decreasing trend Significant level 99% Heavy Rainfall Days during MONSOON 2013 26 Jammu & Kashmir Himachal Pradesh 37 58 Arunachal Pradesh Punjab Uttarakhand 34 25 West Rajasthan 48 52 68 62 74 40 66 59 East Madhya Pradesh Chhattisgarh Vidarbha 20 73 64 Marathawada Madhya Maharashtra 61 66 Jharkhand 48 49 87 Orissa 47 41 Gangetic West Bengal Coastal Andhra Pradesh Rayalaseema South Interior Karnataka 76 19 41 45 Tamilnadu & Pudducherry Lakshadweep 14 72 Kerala Year % dep. 2010 2 2011 2 2012 -7 2013 6 Nagaland Manipur Mizoram & Tripura 46 34 85 52 Telangana North Interior Karnataka Coastal Karnataka 90 Assam & Meghalaya Bihar Gujarat Region West DD&N Madhya Pradesh Havelli Konkan & Goa 79 East Uttar Pradesh East Rajasthan Saurashtra Kuchh & Diu Sub-Himalayan West Benga & Sikkiml 47 Haryana Chd. & West Delhi Uttar Pradesh Andaman & Nicobar Islands Date 24 hrs Chief Rainfall amounts (in Cms) 22 16 - Quant (Gujarat Region) 23 36 - Sagbara ,35 - Umerpada ,30 - Baruch ,20 - Jhagadia, Quant ,19 Jetpur Pavi ,18 - Sankheda, Sankheda Arg, Tilakwada Arg, 15 - Naswadi, Waghai Arg (All Gujarat Region) 24 43 - Quant ,38 - Rajpipala, Vansda, 35 - Tilakwada Arg , 32 - Nandod 30 Karjan , 28 - Sankheda, Tilakwada, Waghai Arg , 26 - Silvassa ,25 - Sinor Arg (All Gujarat Region) 25 26 27 37 - Umerpada ,32 - Surat Aws ,26 - Bardoli ,24 - Karjan, Silvassa, Vagra ,23 - Choryasi ,22 - Palsana ,20 - Jalalpor (All Gujarat Region) 39 – Rajkot, 34 - Visavadar , 28 – Babra, 23 - Jasdan ,22 - Chotila , 21 Gadhda, Jamjodhpur , 20 - Kotdasangani (All Saurashtra & Kutch) 25 - Kamrej , 24 – Olpad, 22 - Choryasi, Hansot Arg (All Gujarat Region), 45 - Khambhalia ,35 - Okha ,29 - Jamnagar Iaf , 24 - Dhrol Arg ,23 Kalyanpur , 22 - Kalavad, Naliya, Upleta (All Saurashtra & Kutch) 28 31 – Bhachau, 28 - Anjar ,20- Bachau Aws (Saurashtra & Kutch) 26 - Bhabhar ,23 – Dhanera, 22 - Deesa (Gujarat Region) 20 - Sanchore (west Rajasthan) 29 27 - Mount Abu, 22- Mounntabu Tehsil, 17- Devel, Dungarpur Tehsil, Kherwara, Pisagan (east Rajasthan) , 17- Merta City (west Rajasthan) 23 – Dantiwada, 17- Deesa (Gujarat Region) 48 hour Forecast 26 Feb 27 Feb EXTREME WEATHER WARNIGS 2014 9 March 11 March HEAVY RAINFALL WARNINGS (PHAILIN ) 12/10/2013 Day 1 Day 2 Day 3 Monsoon 2013 Heavy Rainfall Forecast Cont… Cont… Skill of Heavy Rainfall Warnings Contd.. Contd.. Contd.. Improvement in Skill of Heavy Rainfall Warning Improvement in Skill of Heavy Rainfall Warning (Number of Met. Sub Divisions) Skill of heavy rainfall warning in recent past Programme for improving forecast skills CTCZ PROGRAMME FDP Fog Friday, March 21, 2014 SAARC STORM PROJECT SEVERE WEATHER FORECAST DEMONSTRATION PROJECT (SWFDP) – BAY OF BENGAL FDP Cyclone Conclusion In recent times, operational weather forecasts & warnings of IMD have improved significantly. However, there is need for further improvement: Standardize the verification methods Standardize the threshold limits of scores Standardize region specific impact of extreme events Skill beyond day 3 Skill of extreme and rare events Communicating the uncertainty Focus on PWS Critic’s View We have done extremely well in improving the weather forecast and climate prediction during the past 5 years but have not been that successful in extracting the full value of the more accurate forecast for societal benefit How should we communicate with the public about weather Hazards? PROF. J.SRINIVASAN DIVECHA CENTRE FOR CLIMATE CHANGE INDIAN INSTITUTE OF SCIENCE
© Copyright 2024 ExpyDoc
