Jump to content

Purging of Climate Records


Recommended Posts

It has been talked about before, but it drives me nuts how they purge old records in their quality control process. Is there any way we can stop this or at least view the original data? I have noticed a lot of the cold records from December 1919 and 1930 show up as M now on the Utah State site. The -6 at SLE in 1919 is no longer listed as a record on the WRCC site. 

 

Also from more recent times the -10 at Eugene on 12/8/13 is now shown as M on the Utah State site. 

 

Even more shocking. The -12 at Eugene and Salem from 12/8/1972 has been scrubbed from the records! In fact the WRCC lists the record for the date at Salem as 8 set in 2013. Though oddly they list the -10 from 2013 as the record at Eugene. So disappointing. 

 

 

Snowfall                                  Precip

2022-23: 95.0"                      2022-23: 17.39"

2021-22: 52.6"                    2021-22: 91.46" 

2020-21: 12.0"                    2020-21: 71.59"

2019-20: 23.5"                   2019-20: 58.54"

2018-19: 63.5"                   2018-19: 66.33"

2017-18: 30.3"                   2017-18: 59.83"

2016-17: 49.2"                   2016-17: 97.58"

2015-16: 11.75"                 2015-16: 68.67"

2014-15: 3.5"
2013-14: 11.75"                  2013-14: 62.30
2012-13: 16.75"                 2012-13: 78.45  

2011-12: 98.5"                   2011-12: 92.67"

It's always sunny at Winters Hill! 
Fighting the good fight against weather evil.

 

Link to comment
Share on other sites

  • Replies 18
  • Created
  • Last Reply

Top Posters In This Topic

Popular Days

Sounds like they consider the readings to be suspect.  Somebody needs to let them know that is not the case.  In Utah they should understand what cold pockets are all about.

Death To Warm Anomalies!

 

Winter 2023-24 stats

 

Total Snowfall = 1.0"

Day with 1" or more snow depth = 1

Total Hail = 0.0

Total Ice = 0.2

Coldest Low = 13

Lows 32 or below = 45

Highs 32 or below = 3

Lows 20 or below = 3

Highs 40 or below = 9

 

Link to comment
Share on other sites

The issue is over at NCEI, which has the database that everyone else pulls from (NCDC, WRCC, Utah Climate Center, etc). 

 

Their QC algorithm leaves a lot to be desired.

 

I know you have done some follow up on this in the past. Is their any hope they will fix the algorithm? Also once these various records are purged are they lost to posterity or is their still some kind of original data set kept somewhere that is publicly available?

Snowfall                                  Precip

2022-23: 95.0"                      2022-23: 17.39"

2021-22: 52.6"                    2021-22: 91.46" 

2020-21: 12.0"                    2020-21: 71.59"

2019-20: 23.5"                   2019-20: 58.54"

2018-19: 63.5"                   2018-19: 66.33"

2017-18: 30.3"                   2017-18: 59.83"

2016-17: 49.2"                   2016-17: 97.58"

2015-16: 11.75"                 2015-16: 68.67"

2014-15: 3.5"
2013-14: 11.75"                  2013-14: 62.30
2012-13: 16.75"                 2012-13: 78.45  

2011-12: 98.5"                   2011-12: 92.67"

It's always sunny at Winters Hill! 
Fighting the good fight against weather evil.

 

Link to comment
Share on other sites

I know you have done some follow up on this in the past. Is their any hope they will fix the algorithm? Also once these various records are purged are they lost to posterity or is their still some kind of original data set kept somewhere that is publicly available?

 

Yeah, I had an email conversation with a WRCC climatologist about this issue last year. They're frustrated too since it's not up to them which records get deleted. I have no idea how/when this will be resolved. 

 

The best source for original, unadulterated data are the monthly and annual climate reports. They're available through the NCDC. All the original data can be found here:

 

https://www.ncdc.noaa.gov/IPS/cd/cd.html

Link to comment
Share on other sites

Yeah, I had an email conversation with a WRCC climatologist about this issue last year. They're frustrated too since it's not up to them which records get deleted. I have no idea how/when this will be resolved. 

 

The best source for original, unadulterated data are the monthly and annual climate reports. They're available through the NCDC. All the original data can be found here:

 

https://www.ncdc.noaa.gov/IPS/cd/cd.html

 

Thanks for the link!

Snowfall                                  Precip

2022-23: 95.0"                      2022-23: 17.39"

2021-22: 52.6"                    2021-22: 91.46" 

2020-21: 12.0"                    2020-21: 71.59"

2019-20: 23.5"                   2019-20: 58.54"

2018-19: 63.5"                   2018-19: 66.33"

2017-18: 30.3"                   2017-18: 59.83"

2016-17: 49.2"                   2016-17: 97.58"

2015-16: 11.75"                 2015-16: 68.67"

2014-15: 3.5"
2013-14: 11.75"                  2013-14: 62.30
2012-13: 16.75"                 2012-13: 78.45  

2011-12: 98.5"                   2011-12: 92.67"

It's always sunny at Winters Hill! 
Fighting the good fight against weather evil.

 

Link to comment
Share on other sites

Thanks for the link!

 

Wallace Orchard hitting -14 in December 1919 is pretty impressive as I believe that is essentially West Salem in modern times. That is one of my favorite now defunct stations, along with Mt. Angel, and Sundown Ranch. 

Snowfall                                  Precip

2022-23: 95.0"                      2022-23: 17.39"

2021-22: 52.6"                    2021-22: 91.46" 

2020-21: 12.0"                    2020-21: 71.59"

2019-20: 23.5"                   2019-20: 58.54"

2018-19: 63.5"                   2018-19: 66.33"

2017-18: 30.3"                   2017-18: 59.83"

2016-17: 49.2"                   2016-17: 97.58"

2015-16: 11.75"                 2015-16: 68.67"

2014-15: 3.5"
2013-14: 11.75"                  2013-14: 62.30
2012-13: 16.75"                 2012-13: 78.45  

2011-12: 98.5"                   2011-12: 92.67"

It's always sunny at Winters Hill! 
Fighting the good fight against weather evil.

 

Link to comment
Share on other sites

The QC'ing issue is especially noticeable at stations like The Dalles. Their cold records have been gutted. Missing are the -30 from December 1919, the -25 from January 1930, and the -21 from February 1950. If you look at their daily records now, the coldest January reading shown is just -11, the coldest February reading shown is -12, and the coldest December reading is just -6. 

 

https://wrcc.dri.edu/cgi-bin/cliMAIN.pl?or8407

 

The end result is essentially a fake record book. Completely unrepresentative of the actual capabilities of that climate over the period of record (1893-present). 

Link to comment
Share on other sites

Their QC algorithm leaves a lot to be desired.

 

 

No doubt.  Here's an example of it not working.

 

I noticed that the -50 at Woodruff Utah on 2/6/1899 was purged.  This is the old state record if you discount the un-populated areas such as Peter and Middle Sinks and is probably accurate.

 

Meanwhile the bogus high records of 115 on 8/2/1907 and 111 on 9/4/1907 were left intact.   There's no way that it was that hot in Woodruff (elevation 6350 feet).

Link to comment
Share on other sites

The QC'ing issue is especially noticeable at stations like The Dalles. Their cold records have been gutted. Missing are the -30 from December 1919, the -25 from January 1930, and the -21 from February 1950. If you look at their daily records now, the coldest January reading shown is just -11, the coldest February reading shown is -12, and the coldest December reading is just -6. 

 

https://wrcc.dri.edu/cgi-bin/cliMAIN.pl?or8407

 

The end result is essentially a fake record book. Completely unrepresentative of the actual capabilities of that climate over the period of record (1893-present). 

 

Totally ridiculous.  The West is capable of great anomalies and the idiot algorithm should know that.

Death To Warm Anomalies!

 

Winter 2023-24 stats

 

Total Snowfall = 1.0"

Day with 1" or more snow depth = 1

Total Hail = 0.0

Total Ice = 0.2

Coldest Low = 13

Lows 32 or below = 45

Highs 32 or below = 3

Lows 20 or below = 3

Highs 40 or below = 9

 

Link to comment
Share on other sites

I was thinking about this a little more on the drive into work this morning. It seems like cold records are most effected by this. Perhaps it is perception, but I don't see July 2009 being purged...Is that likely because extreme cold episodes in our climate are more outliers when compared to the historical norm? PDX has been in the 104-107 range may times, but a -30 at The Dalles, though legit is really a once in a century type event without a lot of other events that even approached it? 

Snowfall                                  Precip

2022-23: 95.0"                      2022-23: 17.39"

2021-22: 52.6"                    2021-22: 91.46" 

2020-21: 12.0"                    2020-21: 71.59"

2019-20: 23.5"                   2019-20: 58.54"

2018-19: 63.5"                   2018-19: 66.33"

2017-18: 30.3"                   2017-18: 59.83"

2016-17: 49.2"                   2016-17: 97.58"

2015-16: 11.75"                 2015-16: 68.67"

2014-15: 3.5"
2013-14: 11.75"                  2013-14: 62.30
2012-13: 16.75"                 2012-13: 78.45  

2011-12: 98.5"                   2011-12: 92.67"

It's always sunny at Winters Hill! 
Fighting the good fight against weather evil.

 

Link to comment
Share on other sites

It is also weird some data gets purged and some doesn't. For instance the 0 at Silver Falls in December 1983 has been purged, but the -2 from 1990 or the multiple sub-zero days from 1972 remain. It doesn't seem to have much of a reason, except perhaps because 83' had the low of 0 and then the other lows were around 10, so it seemed like a big outlier in comparison to even the other days around it?

Snowfall                                  Precip

2022-23: 95.0"                      2022-23: 17.39"

2021-22: 52.6"                    2021-22: 91.46" 

2020-21: 12.0"                    2020-21: 71.59"

2019-20: 23.5"                   2019-20: 58.54"

2018-19: 63.5"                   2018-19: 66.33"

2017-18: 30.3"                   2017-18: 59.83"

2016-17: 49.2"                   2016-17: 97.58"

2015-16: 11.75"                 2015-16: 68.67"

2014-15: 3.5"
2013-14: 11.75"                  2013-14: 62.30
2012-13: 16.75"                 2012-13: 78.45  

2011-12: 98.5"                   2011-12: 92.67"

It's always sunny at Winters Hill! 
Fighting the good fight against weather evil.

 

Link to comment
Share on other sites

Okay...One more...

 

Going back to December 1990. There is missing data in the Utah State data set for maximums on the 20th, 25th, and 29th. So it shows the average high at Silver Falls being 37.3 for the month.

 

The missing data for those dates is in the NCDC report.

 

20: 18

25:  33

29: 23

 

The average high for the month was 35.6.

 

The actual mean temp for the month was 30.5, not 31.8. Either way it is the coldest December on record at Silver Falls, but that is a big difference. 

Snowfall                                  Precip

2022-23: 95.0"                      2022-23: 17.39"

2021-22: 52.6"                    2021-22: 91.46" 

2020-21: 12.0"                    2020-21: 71.59"

2019-20: 23.5"                   2019-20: 58.54"

2018-19: 63.5"                   2018-19: 66.33"

2017-18: 30.3"                   2017-18: 59.83"

2016-17: 49.2"                   2016-17: 97.58"

2015-16: 11.75"                 2015-16: 68.67"

2014-15: 3.5"
2013-14: 11.75"                  2013-14: 62.30
2012-13: 16.75"                 2012-13: 78.45  

2011-12: 98.5"                   2011-12: 92.67"

It's always sunny at Winters Hill! 
Fighting the good fight against weather evil.

 

Link to comment
Share on other sites

Okay...One more...

 

Going back to December 1990. There is missing data in the Utah State data set for maximums on the 20th, 25th, and 29th. So it shows the average high at Silver Falls being 37.3 for the month.

 

The missing data for those dates is in the NCDC report.

 

20: 18

25:  33

29: 23

 

The average high for the month was 35.6.

 

The actual mean temp for the month was 30.5, not 31.8. Either way it is the coldest December on record at Silver Falls, but that is a big difference. 

 

It kills people like you and me, but the problem is that not a whole lot of other people care about this stuff. It was probably a lot cheaper for the gov't to develop the algorithm the way it is than to fine tune it. I'm guessing their goal was to knock out a bunch of clearly erroneous data in the COOP record (and there was PLENTY) in order to make the data more useful for scientific study. I'm not even sure if anyone in charge cares about the fact that so many legitimate records were knocked out. Most daily observations and records are still there...so they might run a report and say "well, 96% of daily data remains and we've statistically ascertained that it's all legitimate." Of course the other 4% are all the climatologically significant records that we as enthusiasts care about, in addition to whatever erroneous data that got knocked out.

Link to comment
Share on other sites

This is why I think the management of climate records (and weather/climate modeling in general) should be turned over to the private sector. Preferably to multiple institutions.

Link to comment
Share on other sites

Today's generation is all about automation. They do NOT believe in public feedback and that itself is damaging to our system.  Sure a lot of them will say you can email them but they just send it straight to the rubbish bin.   Today's generation is running blindly the wrong way meanwhile the countries that have experienced true dictatorship like China have slowly realized it and are turning slowly but surely towards freedom more and it's worked out the moment they first tried in the 1990s though still clearly have a ways to go yet.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Unfortunately, your content contains terms that we do not allow. Please edit your content to remove the highlighted words below.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

  • Popular Contributors

  • Activity Stream

    1. 47

      Weather Forums March Madness 2024

    2. 641

      Middle East Conflict of 2023

    3. 379

      March 2024 Observations and Discussion

    4. 5830

      March 2024 Weather in the PNW

    5. 5830

      March 2024 Weather in the PNW

×
×
  • Create New...