Snow forecasters comparison daily

Should I make the website (read comment below before answering)

  • Yes

    Votes: 21 91.3%
  • No

    Votes: 2 8.7%

  • Total voters
    23
  • Poll closed .

Rat trap bindings

One of Us
Ski Pass
Jul 17, 2017
449
1,036
263
Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

5C3D2481-62E7-41C1-86FB-A39379F0780E.jpeg
 

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

5C3D2481-62E7-41C1-86FB-A39379F0780E.jpeg

So would you like me to post the daily forecasts per forecaster for the next 7 days as well as today? I have already got this in my graph, I just haven’t put it here, but I can.
 
Remove ads with a
Ski Pass

zzaaarp

Hard Yards
Jun 20, 2006
26
49
68
38
Melbourne
Forecasts for the 14th july, very big jump in the 7 day weighted average today. Janes weather maintains 100% accuracy. Currently working on adding 7 day per/day forecasts.
Data for th July (5).png
Ranks (1).png

I'm guessing the 100% accuracy hot start for Jane has been due to a couple of zeros. i.e. forecast nothing, get nothing, job done.

I think the 'average' would be more interesting if you discard such data (here in Aus is probably the bulk of the days). We then see the forecast performances in and around weather events, i.e. when (a) it snows but by a different amount to predicted (b) it snowed when no snow predicted (c) it didn't snow when predicted, and disregard (d) no snow when no snow predicted.
 
  • Like
Reactions: Star_Hawk

Rat trap bindings

One of Us
Ski Pass
Jul 17, 2017
449
1,036
263
So would you like me to post the daily forecasts per forecaster for the next 7 days as well as today? I have already got this in my graph, I just haven’t put it here, but I can.
I'm wondering rather than averages across the resorts as there is large vartions in the base data, that you just report Perisher only as you have that visual measuring stick? We use weighted averages in our software usage reporting, however when we can actually see the highs and the lows rather than only averages, that gives the real story.
 

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
I'm guessing the 100% accuracy hot start for Jane has been due to a couple of zeros. i.e. forecast nothing, get nothing, job done.

I think the 'average' would be more interesting if you discard such data (here in Aus is probably the bulk of the days). We then see the forecast performances in and around weather events, i.e. when (a) it snows but by a different amount to predicted (b) it snowed when no snow predicted (c) it didn't snow when predicted, and disregard (d) no snow when no snow predicted.

Yeah I have data on that, and also on how much snow they’ve predicted this season so far vs how much has fallen. It shows for example, that j2ski, while being 75% accurate, is -245% off the overall amount this season. So I have been considering doing that, but the reason I don’t cut the 0 days is that there are forecasters who say that there’s going to be no snow on days that there is snow, so I can’t really cut those days without sacrificing accuracy. My weighting for the average takes into account the accuracy of forecasters during snow cycles, by combing their overall accuracy with their season snow snowfall accuracy.
 
  • Like
Reactions: zzaaarp

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
I'm wondering rather than averages across the resorts as there is large vartions in the base data, that you just report Perisher only as you have that visual measuring stick? We use weighted averages in our software usage reporting, however when we can actually see the highs and the lows rather than only averages, that gives the real story.

I can do just one resort if people want me to.
 

Rat trap bindings

One of Us
Ski Pass
Jul 17, 2017
449
1,036
263
I can do just one resort if people want me to.
I dont think it’s the issue of which resort just the one that the 7 day prediction is easlier to have the actual? Perisher does have that stake for all to see so could be the Bench Mark (pun intended) only need to know when they turn the table.
we much appreciate your effects tho!
 

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
I dont think it’s the issue of which resort just the one that the 7 day prediction is easlier to have the actual? Perisher does have that stake for all to see so could be the Bench Mark (pun intended) only need to know when they turn the table.
we much appreciate your effects tho!

Yeah, I’ll move to using Perisher from tomorrow on then. Unless people ask not to, but it’s certainly easier for me.
 
  • Like
Reactions: notty and JindySuz
Remove ads with a
Ski Pass
Remove ads with a
Ski Pass

fenrir

One of Us
Ski Pass
Jul 23, 2010
4,189
7,418
363
Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

5C3D2481-62E7-41C1-86FB-A39379F0780E.jpeg
Yeah that's my main gripe with a site claiming to be the most accurate forecaster. Constant revisions up until the last minute. It seems less like forecasting and more like observation.
You want a forecast you can plan around.
 

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
Yeah that's my main gripe with a site claiming to be the most accurate forecaster. Constant revisions up until the last minute. It seems less like forecasting and more like observation.
You want a forecast you can plan around.
Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

5C3D2481-62E7-41C1-86FB-A39379F0780E.jpeg
Also a general question with regards to the 7 day per day forecasts, i could publish them in two ways;
1) each day has a graph like the current "7 day outlook" graph on the current graphic
2) each day has a column of boxes like the "Todays outlook" section on the current graphic.
which one do you want?
Edit: Each day will also have a weighted average
 

fenrir

One of Us
Ski Pass
Jul 23, 2010
4,189
7,418
363
Also a general question with regards to the 7 day per day forecasts, i could publish them in two ways;
1) each day has a graph like the current "7 day outlook" graph on the current graphic
2) each day has a column of boxes like the "Todays outlook" section on the current graphic.
which one do you want?
Edit: Each day will also have a weighted average
It depends what you are showing. If you are comparing the forecast from 7 days ago with the current day to calculate accuracy then it would be in boxes like "today's outlook".
If you are showing the projection of the next 7 days from today then a graph is easier to read.
 
  • Like
Reactions: Star_Hawk

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
It depends what you are showing. If you are comparing the forecast from 7 days ago with the current day to calculate accuracy then it would be in boxes like "today's outlook".
If you are showing the projection of the next 7 days from today then a graph is easier to read.
I'm doing per day projections for the next 7 days, so ill go with the graphs then, with the weighted averages above each graph.
 

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
This is awesome @Star_Hawk thanks so much.

Have to admit I'm not coping with the differing scales on each of the 4-day bar charts. It's messing with my head.
I’ll have a look at fixing this for tomorrow’s data.
OK, I've moved the next 4 days data into one graph and colour coded the days, to hopefully make it easier to read. The weighted averages will be on the side.
 
  • Like
Reactions: Lifes2good

Snowmaker7

One of Us
Ski Pass
Jul 25, 2013
582
1,743
263
Bowral, NSW
I remember you mentioned that you'd be adding BOM and Yr.no (EC) (probably the 2 most prominent forecasting sites posted on these forums for a quick idea of what to expect). Are you still going to add them to your list?
 
  • Like
Reactions: Star_Hawk

catzizme

Hard Yards
Apr 5, 2020
90
154
83
Just another website to add to your aresenal, the weather network is Canada's official weather forecast website and has it's own model (canadian government funded, not sure of acronym) like BOM (ACCESS). I have found this websites automatic function to more accurately forecast the precipitation (snow) this year than any other, in different locations, although I would take its wind forecast with a grain of salt and still use ECMWF.

 
  • Like
Reactions: Star_Hawk

Lifes2good

One of Us
Ski Pass
Jan 21, 2012
866
1,429
263
52
Sydney, Aust
Apols if you have already explained this but how do you measure accuracy? If a forecaster says 5cm and it ends up being 7cm or 3cm how does that get included in your calcs?
 
  • Like
Reactions: Star_Hawk

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
Apols if you have already explained this but how do you measure accuracy? If a forecaster says 5cm and it ends up being 7cm or 3cm how does that get included in your calcs?
I usually give a margin of error of 2cm variance between forecasted total and actual total, anything more than that and they're just marked as wrong. However, at the end of this season ill look at the data and i plan to rework that a bit, but i just don't have the time now so I'm sticking with my current system.
 

Richard

Maintenance Dept
Administrator
Mar 14, 1995
16,543
25,976
813
Clarence Town
Janes weather calling 39cm on Monday! Mountain watch moves up 2 places in the accuracy rankings. Forecasts for the 27th July. Location: Perisher
Ranks (14).png
The tightening of the accuracy spread is interesting, I reckon, given enough time and rolls of the dice, the spread will shrink to around 5%, reflecting the crap shoot that long term (7+ days) forecasting actually is.

Suspect you will need to rework the margin of error (+/- 2cm) between success and failure and switch from Boolean accuracy result to a weighted accuracy result for 2023.
 

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
The tightening of the accuracy spread is interesting, I reckon, given enough time and rolls of the dice, the spread will shrink to around 5%, reflecting the crap shoot that long term (7+ days) forecasting actually is.

Suspect you will need to rework the margin of error (+/- 2cm) between success and failure and switch from Boolean accuracy result to a weighted accuracy result for 2023.

Yep, I plan on doing this, just don’t have the time at the moment. What I want to do is completely re do my system after the season for next year, but this works for now. My weighted average has been pretty accurate so far though so I’m happy with that.
 
  • Like
Reactions: Richard

Richard

Maintenance Dept
Administrator
Mar 14, 1995
16,543
25,976
813
Clarence Town
Yep, I plan on doing this, just don’t have the time at the moment. What I want to do is completely re do my system after the season for next year, but this works for now. My weighted average has been pretty accurate so far though so I’m happy with that.
Don’t get me wrong, I’m enjoying this exercise as much as everyone else.

To do is to learn what piece needs iteration. I doth my hat.

Also, Jane will always inevitably win regardless imho. A real meto, running her eyes over the data personally and forecasting for a weather condition (snow) for which she holds a real passion, for local regions she has studied her entire career…. has to have an edge right?!
 
  • Like
Reactions: Star_Hawk

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
Don’t get me wrong, I’m enjoying this exercise as much as everyone else.

To do is to learn what piece needs iteration. I doth my hat.

Also, Jane will always inevitably win regardless imho. A real meto, running her eyes over the data personally and forecasting for a weather condition (snow) for which she holds a real passion, for local regions she has studied her entire career…. has to have an edge right?!

Exactly why I’m working on implementing yr.no and bom. Hopefully that will make the top end of the accuracy table more interesting.
 
  • Like
Reactions: RustyJ and Richard

lewis

One of Us
Jan 25, 2018
472
760
263
Exactly why I’m working on implementing yr.no and bom. Hopefully that will make the top end of the accuracy table more interesting.
I'm pretty sure you've said this before, but just to confirm, is accuracy based on 1 day out? Or something else?
 
  • Like
Reactions: Star_Hawk

lewis

One of Us
Jan 25, 2018
472
760
263
Yeah it’s based on the snowfall from each day after it occurs. I put the snowfall in from the previous day each morning for accuracy.
I see, those accuracies are probably too high if it was anything else I would think.

The current accuracies in your posts do help to give a general picture of the reliability of the model (maybe more-so after a couple of seasons).
However, have you considered doing accuracy for each forecasting site/model 4 or 7 days out? Personally, I'd find those very useful, because, depending on the accuracy of a certain model, you could know how much faith to have in it at a specified range.
 

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
I see, those accuracies are probably too high if it was anything else I would think.

The current accuracies in your posts do help to give a general picture of the reliability of the model (maybe more-so after a couple of seasons).
However, have you considered doing accuracy for each forecasting site/model 4 or 7 days out? Personally, I'd find those very useful, because, depending on the accuracy of a certain model, you could know how much faith to have in it at a specified range.

Yeah, at the end of the season I’ll take suggestions on what to add for next season. I’m going to re work it over the summer, so suggestions will be helpful. I plan on doing this every season now since people seem to like it.
 

fenrir

One of Us
Ski Pass
Jul 23, 2010
4,189
7,418
363
Yeah, at the end of the season I’ll take suggestions on what to add for next season. I’m going to re work it over the summer, so suggestions will be helpful. I plan on doing this every season now since people seem to like it.
If you retain the data for forecasts at +4 and +7 days you should be able to perform the analysis at leisure later on
 
  • Like
Reactions: Star_Hawk

Star_Hawk

Obsessed with Stats
Ski Pass
Jun 23, 2022
205
495
113
Eastern Suburbs, Sydney
If you retain the data for forecasts at +4 and +7 days you should be able to perform the analysis at leisure later on
Just to give you an idea of my extremely messy data collection, here's some pictures of my excel sheet which has all my data prior to inserting it into the graphic each morning.

This is for the next 7 days
7days.PNG
This is for each individual day (ignore the "most accurate" column, that's just for my own interests)
Daily.PNG

And this was when i was working out what weightings worked best for the weighted average
working.PNG

And this is why i don't publish my excel sheet directly here lol
 
Remove ads with a
Ski Pass
Remove ads with a
Ski Pass

Log in

or Log in using