Snow forecasters comparison daily

[Rat trap bindings]

Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

 

[Star_Hawk]

Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

So would you like me to post the daily forecasts per forecaster for the next 7 days as well as today? I have already got this in my graph, I just haven’t put it here, but I can.

 

[Star_Hawk]

Forecasts for the 14th july, very big jump in the 7 day weighted average today. Janes weather maintains 100% accuracy. Currently working on adding 7 day per/day forecasts.

 

[zzaaarp]

Forecasts for the 14th july, very big jump in the 7 day weighted average today. Janes weather maintains 100% accuracy. Currently working on adding 7 day per/day forecasts.

I'm guessing the 100% accuracy hot start for Jane has been due to a couple of zeros. i.e. forecast nothing, get nothing, job done.

I think the 'average' would be more interesting if you discard such data (here in Aus is probably the bulk of the days). We then see the forecast performances in and around weather events, i.e. when (a) it snows but by a different amount to predicted (b) it snowed when no snow predicted (c) it didn't snow when predicted, and disregard (d) no snow when no snow predicted.

 

[Rat trap bindings]

So would you like me to post the daily forecasts per forecaster for the next 7 days as well as today? I have already got this in my graph, I just haven’t put it here, but I can.

I'm wondering rather than averages across the resorts as there is large vartions in the base data, that you just report Perisher only as you have that visual measuring stick? We use weighted averages in our software usage reporting, however when we can actually see the highs and the lows rather than only averages, that gives the real story.

 

[Star_Hawk]

I'm guessing the 100% accuracy hot start for Jane has been due to a couple of zeros. i.e. forecast nothing, get nothing, job done.

I think the 'average' would be more interesting if you discard such data (here in Aus is probably the bulk of the days). We then see the forecast performances in and around weather events, i.e. when (a) it snows but by a different amount to predicted (b) it snowed when no snow predicted (c) it didn't snow when predicted, and disregard (d) no snow when no snow predicted.

Yeah I have data on that, and also on how much snow they’ve predicted this season so far vs how much has fallen. It shows for example, that j2ski, while being 75% accurate, is -245% off the overall amount this season. So I have been considering doing that, but the reason I don’t cut the 0 days is that there are forecasters who say that there’s going to be no snow on days that there is snow, so I can’t really cut those days without sacrificing accuracy. My weighting for the average takes into account the accuracy of forecasters during snow cycles, by combing their overall accuracy with their season snow snowfall accuracy.

 

[Star_Hawk]

I'm wondering rather than averages across the resorts as there is large vartions in the base data, that you just report Perisher only as you have that visual measuring stick? We use weighted averages in our software usage reporting, however when we can actually see the highs and the lows rather than only averages, that gives the real story.

I can do just one resort if people want me to.

 

[Rat trap bindings]

I can do just one resort if people want me to.

I dont think it’s the issue of which resort just the one that the 7 day prediction is easlier to have the actual? Perisher does have that stake for all to see so could be the Bench Mark (pun intended) only need to know when they turn the table.
we much appreciate your effects tho!

 

[Star_Hawk]

I dont think it’s the issue of which resort just the one that the 7 day prediction is easlier to have the actual? Perisher does have that stake for all to see so could be the Bench Mark (pun intended) only need to know when they turn the table.
we much appreciate your effects tho!

Yeah, I’ll move to using Perisher from tomorrow on then. Unless people ask not to, but it’s certainly easier for me.

 

[Star_Hawk]

Forecasts and accuracy's for the 15th of july. Forecast location has been changed to perisher.

 

[Statusdenied]

Hey @Star_Hawk, thank you!

 

[JindySuz]

These next few days might see some changes in these rankings.

 

[Star_Hawk]

These next few days might see some changes in these rankings.

that's what I'm expecting. My prediction is that jane will defiantly not be at 100% after this storm, and I expect j2ski to also fall, as they aren't really that accurate during storms.

 

[Nidecker]

that's what I'm expecting. My prediction is that jane will defiantly not be at 100% after this storm, and I expect j2ski to also fall, as they aren't really that accurate during storms.

This is kinda fun especially with you doing the work...... I like it

 

[Star_Hawk]

Forecasts for the 16th July. Forecast area: Perisher. Storm begins today.

 

[Star_Hawk]

Forecasts for the 17th July. Janes weather moves to 80% accuracy. Forecast location: Perisher.

 

[Star_Hawk]

Forecasts for the 18th of July. Janes weather was spot on with the snow amount yesterday. Forecast location: Perisher.

 

[Star_Hawk]

Forecast's for the 19th of July. Not much on the cards for the next week here. Forecast location: Perisher.

 

[Star_Hawk]

Forecasts for the 20th of July. Forecast location: Perisher.

 

[Star_Hawk]

Forecast's for the 21st of July. Forecast location: Perisher.

 

[Star_Hawk]

Forecast's for the 22nd of july. Forecast location: Perisher.

 

[fenrir]

Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

Yeah that's my main gripe with a site claiming to be the most accurate forecaster. Constant revisions up until the last minute. It seems less like forecasting and more like observation.
You want a forecast you can plan around.

 

[Star_Hawk]

Yeah that's my main gripe with a site claiming to be the most accurate forecaster. Constant revisions up until the last minute. It seems less like forecasting and more like observation.
You want a forecast you can plan around.

Will implement 7 day per day forecasts this weekend.

 

[Star_Hawk]

Yeah that's my main gripe with a site claiming to be the most accurate forecaster. Constant revisions up until the last minute. It seems less like forecasting and more like observation.
You want a forecast you can plan around.

Whilst it is interesting to see what the rankings are for todays forecast, what would be a great interest IMO its to see how a week ahead stacks up. And even more as Snow-Forecast commits to even hourly 7 days or so out, for Perisher for eg. Lets see just with Perisher as you have the snow stake as the judge to test the forecasters out in a true predictions model rather than today as I think I could say what todays fall will be.?

Also a general question with regards to the 7 day per day forecasts, i could publish them in two ways;
1) each day has a graph like the current "7 day outlook" graph on the current graphic
2) each day has a column of boxes like the "Todays outlook" section on the current graphic.
which one do you want?
Edit: Each day will also have a weighted average

 

[fenrir]

Also a general question with regards to the 7 day per day forecasts, i could publish them in two ways;
1) each day has a graph like the current "7 day outlook" graph on the current graphic
2) each day has a column of boxes like the "Todays outlook" section on the current graphic.
which one do you want?
Edit: Each day will also have a weighted average

It depends what you are showing. If you are comparing the forecast from 7 days ago with the current day to calculate accuracy then it would be in boxes like "today's outlook".
If you are showing the projection of the next 7 days from today then a graph is easier to read.

 

[Star_Hawk]

It depends what you are showing. If you are comparing the forecast from 7 days ago with the current day to calculate accuracy then it would be in boxes like "today's outlook".
If you are showing the projection of the next 7 days from today then a graph is easier to read.

I'm doing per day projections for the next 7 days, so ill go with the graphs then, with the weighted averages above each graph.

 

[Star_Hawk]

Little late on the data today, working on next 4 days per day projections.

 

[Star_Hawk]

Forecasts for the 23rd july. Forecast location: Perisher. 4 day per day forecasts have been added.

 

[Star_Hawk]

Forecasts for the 24th of July. Forecast Location: Perisher. Fair bit happening over the next few days by the looks of things.

 

[Lifes2good]

This is awesome @Star_Hawk thanks so much.

Have to admit I'm not coping with the differing scales on each of the 4-day bar charts. It's messing with my head.

 

[Star_Hawk]

This is awesome @Star_Hawk thanks so much.

Have to admit I'm not coping with the differing scales on each of the 4-day bar charts. It's messing with my head.

I’ll have a look at fixing this for tomorrow’s data.

 

[Star_Hawk]

This is awesome @Star_Hawk thanks so much.

Have to admit I'm not coping with the differing scales on each of the 4-day bar charts. It's messing with my head.

I’ll have a look at fixing this for tomorrow’s data.

OK, I've moved the next 4 days data into one graph and colour coded the days, to hopefully make it easier to read. The weighted averages will be on the side.

 

[Star_Hawk]

Forecasts for the 25th July. Forecast location: Perisher. Have implemented new 5 day graph, hopefully its easier to read now.

 

[Snowmaker7]

I remember you mentioned that you'd be adding BOM and Yr.no (EC) (probably the 2 most prominent forecasting sites posted on these forums for a quick idea of what to expect). Are you still going to add them to your list?

 

[Star_Hawk]

I remember you mentioned that you'd be adding BOM and Yr.no (EC) (probably the 2 most prominent forecasting sites posted on these forums for a quick idea of what to expect). Are you still going to add them to your list?

Yeah, working on adding them.

 

[catzizme]

Just another website to add to your aresenal, the weather network is Canada's official weather forecast website and has it's own model (canadian government funded, not sure of acronym) like BOM (ACCESS). I have found this websites automatic function to more accurately forecast the precipitation (snow) this year than any other, in different locations, although I would take its wind forecast with a grain of salt and still use ECMWF.

https://www.theweathernetwork.com/au/weather/new-south-wales/thredbo

 

[Star_Hawk]

Forecast's for the 26th of July. Location: Perisher.

 

[Lifes2good]

Apols if you have already explained this but how do you measure accuracy? If a forecaster says 5cm and it ends up being 7cm or 3cm how does that get included in your calcs?

 

[Star_Hawk]

Apols if you have already explained this but how do you measure accuracy? If a forecaster says 5cm and it ends up being 7cm or 3cm how does that get included in your calcs?

I usually give a margin of error of 2cm variance between forecasted total and actual total, anything more than that and they're just marked as wrong. However, at the end of this season ill look at the data and i plan to rework that a bit, but i just don't have the time now so I'm sticking with my current system.

 

[Star_Hawk]

Janes weather calling 39cm on Monday! Mountain watch moves up 2 places in the accuracy rankings. Forecasts for the 27th July. Location: Perisher

 

[Richard]

Janes weather calling 39cm on Monday! Mountain watch moves up 2 places in the accuracy rankings. Forecasts for the 27th July. Location: Perisher

The tightening of the accuracy spread is interesting, I reckon, given enough time and rolls of the dice, the spread will shrink to around 5%, reflecting the crap shoot that long term (7+ days) forecasting actually is.

Suspect you will need to rework the margin of error (+/- 2cm) between success and failure and switch from Boolean accuracy result to a weighted accuracy result for 2023.

 

[Star_Hawk]

The tightening of the accuracy spread is interesting, I reckon, given enough time and rolls of the dice, the spread will shrink to around 5%, reflecting the crap shoot that long term (7+ days) forecasting actually is.

Suspect you will need to rework the margin of error (+/- 2cm) between success and failure and switch from Boolean accuracy result to a weighted accuracy result for 2023.

Yep, I plan on doing this, just don’t have the time at the moment. What I want to do is completely re do my system after the season for next year, but this works for now. My weighted average has been pretty accurate so far though so I’m happy with that.

 

[Richard]

Yep, I plan on doing this, just don’t have the time at the moment. What I want to do is completely re do my system after the season for next year, but this works for now. My weighted average has been pretty accurate so far though so I’m happy with that.

Don’t get me wrong, I’m enjoying this exercise as much as everyone else.

To do is to learn what piece needs iteration. I doth my hat.

Also, Jane will always inevitably win regardless imho. A real meto, running her eyes over the data personally and forecasting for a weather condition (snow) for which she holds a real passion, for local regions she has studied her entire career…. has to have an edge right?!

 

[Star_Hawk]

Don’t get me wrong, I’m enjoying this exercise as much as everyone else.

To do is to learn what piece needs iteration. I doth my hat.

Also, Jane will always inevitably win regardless imho. A real meto, running her eyes over the data personally and forecasting for a weather condition (snow) for which she holds a real passion, for local regions she has studied her entire career…. has to have an edge right?!

Exactly why I’m working on implementing yr.no and bom. Hopefully that will make the top end of the accuracy table more interesting.

 

[lewis]

Exactly why I’m working on implementing yr.no and bom. Hopefully that will make the top end of the accuracy table more interesting.

I'm pretty sure you've said this before, but just to confirm, is accuracy based on 1 day out? Or something else?

 

[Star_Hawk]

I'm pretty sure you've said this before, but just to confirm, is accuracy based on 1 day out? Or something else?

Yeah it’s based on the snowfall from each day after it occurs. I put the snowfall in from the previous day each morning for accuracy.

 

[lewis]

Yeah it’s based on the snowfall from each day after it occurs. I put the snowfall in from the previous day each morning for accuracy.

I see, those accuracies are probably too high if it was anything else I would think.

The current accuracies in your posts do help to give a general picture of the reliability of the model (maybe more-so after a couple of seasons).
However, have you considered doing accuracy for each forecasting site/model 4 or 7 days out? Personally, I'd find those very useful, because, depending on the accuracy of a certain model, you could know how much faith to have in it at a specified range.

 

[Star_Hawk]

I see, those accuracies are probably too high if it was anything else I would think.

The current accuracies in your posts do help to give a general picture of the reliability of the model (maybe more-so after a couple of seasons).
However, have you considered doing accuracy for each forecasting site/model 4 or 7 days out? Personally, I'd find those very useful, because, depending on the accuracy of a certain model, you could know how much faith to have in it at a specified range.

Yeah, at the end of the season I’ll take suggestions on what to add for next season. I’m going to re work it over the summer, so suggestions will be helpful. I plan on doing this every season now since people seem to like it.

 

[fenrir]

Yeah, at the end of the season I’ll take suggestions on what to add for next season. I’m going to re work it over the summer, so suggestions will be helpful. I plan on doing this every season now since people seem to like it.

If you retain the data for forecasts at +4 and +7 days you should be able to perform the analysis at leisure later on

 

[Star_Hawk]

If you retain the data for forecasts at +4 and +7 days you should be able to perform the analysis at leisure later on

Just to give you an idea of my extremely messy data collection, here's some pictures of my excel sheet which has all my data prior to inserting it into the graphic each morning.

This is for the next 7 days

This is for each individual day (ignore the "most accurate" column, that's just for my own interests)

And this was when i was working out what weightings worked best for the weighted average

And this is why i don't publish my excel sheet directly here lol