1. SPECIAL NOTE TO NEW WEATHERZONE FOLK

    If you recently registered and have not received a confirmation email - please check your 'Spam or Junk' folders. Especially if your email is Hotmail. More help with confirmation issues

    NOTE: This notice may be closed.

    Dismiss Notice

Resource Snow prediction accuracy thread

Discussion in 'Alpine & Southern' started by Jellybeans, Aug 4, 2017.

  1. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    [​IMG]
    Latest table from the last system completed.
     
  2. The Plowking

    The Plowking Part of the Furniture Ski Pass: Gold

    Joined:
    Aug 15, 2000
    Messages:
    17,150
    Likes Received:
    9,527
    Location:
    Bonbeach, Vic
    Top work.
     
    Jellybeans likes this.
  3. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
  4. Seafm

    Seafm Too far from the snow Ski Pass: Gold

    Joined:
    Jun 5, 2014
    Messages:
    4,126
    Likes Received:
    3,473
    Location:
    Cairns, Queensland
    Looks like the usual suspect is smoking weed again.
     
  5. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    That would be correct.
     
  6. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    [​IMG]
    Current system.

    I stopped doing the other one, because of my sickness last week.
    This will be the final table for the season.
     
    #56 Jellybeans, Sep 25, 2017
    Last edited: Sep 25, 2017
    The Plowking and Undies like this.
  7. The Plowking

    The Plowking Part of the Furniture Ski Pass: Gold

    Joined:
    Aug 15, 2000
    Messages:
    17,150
    Likes Received:
    9,527
    Location:
    Bonbeach, Vic
    *fixed
    Love your work.
     
    Jellybeans likes this.
  8. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Fixed the post :thumbs:
     
  9. DiscoStuAU

    DiscoStuAU One of Us Ski Pass: Gold

    Joined:
    May 30, 2015
    Messages:
    1,403
    Likes Received:
    3,397
    Location:
    Melbourne
    This may be the wrong spot to ask it- but do we have an idea or record of the amount of snow that has fallen this season on the hills? Not what we record at Spencer's creek but actually what has fallen each and every week then added up for a cumulative amount at the end of season...

    So we can say that "this year we had xxx amount of snow fall" etc...
     
  10. POW_hungry

    POW_hungry Part of the Furniture Ski Pass: Gold

    Joined:
    May 28, 2000
    Messages:
    16,956
    Likes Received:
    18,966
    Location:
    Eastern Burbs of Sydney
    Thredbo has a published cumulative season total (for what it's worth) of 473cm... from memory that mustn't be far off the total for the 2000 season.
    I vaguely recall it around 480-490cms.
     
    DiscoStuAU likes this.
  11. DiscoStuAU

    DiscoStuAU One of Us Ski Pass: Gold

    Joined:
    May 30, 2015
    Messages:
    1,403
    Likes Received:
    3,397
    Location:
    Melbourne
    Thanks Buddy. :)
     
    POW_hungry likes this.
  12. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Hotham publishes one too. 466cm.
     
    DiscoStuAU likes this.
  13. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    [​IMG]
    Here is the final table completed (sorry it took so long).
    As stated, finished for this season.
    Data is available for anybody to use.
    I am going to make some sort of table, with the average accuracy of the forecasters at some point. But feel free to do what you want with it.
     
    SMSkier, nfip and Majikthise like this.
  14. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Nearly finished the rigorous* analysis of the data.



    * not rigorous at all
     
    DidSurfNowSki and robbo mcs like this.
  15. Nidecker

    Nidecker One of Us Ski Pass: Gold

    Joined:
    Aug 5, 2009
    Messages:
    2,545
    Likes Received:
    1,517
    Location:
    Alpine the dark side of snowboarding
    Looking forward to this, is it my imagination, or has/was WZ quite on the money most of the time?
     
    W0nkey D0nkey likes this.
  16. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Ok.......
    Before I start this, I should mention the data errors.
    For starters, some of my estimations may be incorrect. I also haven't got the predictions from the exact same time for each system, due to the fluidness of weather forecasts.
    Errors on part of the forecasters featured include 'worded predictions', which I have had to estimate or not include. Grasshopper what the hell does 'the snow keeps coming down' mean??????

    Okay so this is the table based on 4 and 7 day predictions. I based my point system off how far the prediction was from the actual storm total at Perisher. For predictions that were a range (eg 10-20cm), I based my calculation on the median figure. I then created a points system, based on the bracketed figures.
    [​IMG]
    If you want to declare winners, for the 7 day category the winners are Jane and EC. And for the 4 day category, the winners are the BOM and EC.
    The point system isn't great, as I am comparing dusting systems and massive dumps. But this is the best I have. Cheers :thumbs:

    Edit: Also thanks @The Plowking for the support throughout this process.
     
    robbo mcs, MarkV, Bloke and 7 others like this.
  17. Majikthise

    Majikthise Sage Moderator Ski Pass: Gold

    Joined:
    Jan 1, 1970
    Messages:
    29,740
    Likes Received:
    12,612
    Location:
    Blue Mts
    ok so we watch for Jane or yr.no to get excited and then wait for the BoM 4 day rule to apply ;)
    love your work!
     
  18. The Plowking

    The Plowking Part of the Furniture Ski Pass: Gold

    Joined:
    Aug 15, 2000
    Messages:
    17,150
    Likes Received:
    9,527
    Location:
    Bonbeach, Vic
    Yeah....half arsed support at that...
    Next year im going to post synoptics to match dates for you......assuming you have time to do it again.
     
  19. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Yeah that would be appreciated.
     
    The Plowking likes this.
  20. Claude Cat

    Claude Cat On my bike Moderator Ski Pass: Platinum

    Joined:
    Jul 6, 2001
    Messages:
    111,095
    Likes Received:
    47,993
    Location:
    Canterbury, Vic
    Nice work.
     
    Jellybeans likes this.
  21. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    According to the stats, no.
     
    Claude Cat likes this.
  22. The Plowking

    The Plowking Part of the Furniture Ski Pass: Gold

    Joined:
    Aug 15, 2000
    Messages:
    17,150
    Likes Received:
    9,527
    Location:
    Bonbeach, Vic
    !
     
    Claude Cat likes this.
  23. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    It's back. The snow prediction accuracy thread: Round 2! Starting for the first June system. Working out the details, will probably keep the same format as last year. Can Jane, the BOM and EC keep their spots? tune in to find out. suggestions if there are any please.

    You assume correctly, I might not be doing every system though (because of other commitments, hopefully chiefly skiing, but also my overseas Wx project). Wonder how Synoptics could work out?
     
    D_Dog likes this.
  24. The Plowking

    The Plowking Part of the Furniture Ski Pass: Gold

    Joined:
    Aug 15, 2000
    Messages:
    17,150
    Likes Received:
    9,527
    Location:
    Bonbeach, Vic
    hmm.
    Will think on it.
     
  25. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Get back to me sometime before the end of next month in that case.
     
  26. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    It would have been interesting to do the big storm, but laziness and business got the best of me.

    Here is the first one of 2018, looking like a dusting ATM.


    This season I am basing the forecast on Perisher (as I did last year).

    edit: two dustings in fact.
     
    Chaeron and Vertikar like this.
  27. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Finally the update for the above system... that was not to be.

    And the current one
     
  28. Hermannator

    Hermannator One of Us Ski Pass: Silver

    Joined:
    Jul 9, 2010
    Messages:
    1,377
    Likes Received:
    751
    Location:
    Melburn
    Good initiative...

    I assume mountainwatch gets their info from Grasshopper

    Does anyone know where/how Snow-forecast.com gets their snowfall predictions (esp given they're worldwide)?
     
  29. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Yes.
    And snow-forecast.com gets their predictions from GFS (already in the table).
     
  30. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.

    This was for the last system. Was that figure correct (getting conflicting numbers), got that from a snow depth number checking app?
     
    cornice11 likes this.
  31. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Frog overstated a dump signal which flopped into a 10cm topup. What's new?
     
  32. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Obs added.


    Forecasts: about to experience.
     
    Swiss likes this.
  33. Lady Penelope

    Lady Penelope One of some lot ... Ski Pass: Gold

    Joined:
    Sep 7, 2014
    Messages:
    2,638
    Likes Received:
    6,197
    Location:
    Canberra
    Great work, @Jellybeans1000.
     
    Jellybeans likes this.
  34. mick chopps

    mick chopps Pool Room Ski Pass: Gold

    Joined:
    Jun 22, 2006
    Messages:
    68,975
    Likes Received:
    48,841
    Location:
    The barber's chair
    Looks like Jane nailed it, even if it was via a very broad range.
     
    Jellybeans likes this.
  35. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Another year, another comparative predictions table....

    Firstly the disclaimer from last year.
    This year I examined 8 systems (6 for the 4 day prediction range):
    • 23-26 June
    • 6-9 July
    • 16-18 July
    • 19-20 July
    • 5-8 August
    • 11-13 August
    • 29 Aug-2 Sept
    • 6-9 September
    They are roughly based around a 3-4 day and 6-7 day range. These are all based upon how far away the various predictions are from the storm total measured at Perisher. The median figure was used for ranges of figures. Points are based upon the bracketed numbers, with totals on the right. Here they are....



    Based on those figures, 7 day category goes to EC/yr.no, runners up is it's loyal sidekick GFS.
    4 day category goes to ACCESS-G (somewhat surprisedly), with joint runners up to GFS, EC and Jane.

    This wouldn't be complete if we didn't combine them with last year's and see what we get! Gaps in data are not noted.



    7 day category still has EC being consistently good, followed up by Jane. 4 day category has joint winners in EC and ACCESS-G, followed up again by Jane.

    There you have it, another great season wrapping up. Thanks again to all :thumbs:
     
    Swiss, D_Dog, Vertikar and 5 others like this.
  36. Claude Cat

    Claude Cat On my bike Moderator Ski Pass: Platinum

    Joined:
    Jul 6, 2001
    Messages:
    111,095
    Likes Received:
    47,993
    Location:
    Canterbury, Vic
    Nice analysis, thanks for taking the trouble :thumbs:
     
    Jellybeans likes this.
  37. Lady Penelope

    Lady Penelope One of some lot ... Ski Pass: Gold

    Joined:
    Sep 7, 2014
    Messages:
    2,638
    Likes Received:
    6,197
    Location:
    Canberra
    Love a statistical analysis! Just wondering if you could explain the raw data in the cells a little more, @Jellybeans1000 . For instance, does the figure in the cell represent the number of times a forecaster was (eg) 0-10cm off the total?
     
    Jellybeans likes this.
  38. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Yep that would be it. And then the amounts are put to the value in the brackets at the top, and added up to make a total. Higher values equal higher accuracy.

    You see a lot of 0-10cms, which are mixes of good guesses of dumps and well forecasted dustings. And then there are a few 50cm+ ones, which represent the dumps that fizzed or the sudden sprung up system, with the former more likely. Then the in between ranges represent all the other scenarios with plenty 10-20cm off, and then decreasing as the distance away from the actual total increases, which I think is a good result IMO. And of course you see lower ranges in the 4 day, compared to the 7 day, so you'd think there would be higher scores there, but as mentioned, there is a lower amount of data in that category.

    Does this help at all? More than happy to take any other questions :)
     
    Lady Penelope likes this.
  39. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    G'day all,
    This project has ran for the past two years, so it's time to re-evaluate.
    So do people think that it is useful?
    Any suggestions? Constructive criticism?
    @The Plowking?

    Thanks in advance.
     
    notty and The Plowking like this.
  40. Lady Penelope

    Lady Penelope One of some lot ... Ski Pass: Gold

    Joined:
    Sep 7, 2014
    Messages:
    2,638
    Likes Received:
    6,197
    Location:
    Canberra
    It is useful. Thanks for your work here, @Jellybeans
     
    Jellybeans likes this.
  41. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    Cheers
    Hope you are doing better, after all that happened to you :thumbs:
     
    Lady Penelope likes this.
  42. The Plowking

    The Plowking Part of the Furniture Ski Pass: Gold

    Joined:
    Aug 15, 2000
    Messages:
    17,150
    Likes Received:
    9,527
    Location:
    Bonbeach, Vic
    I reckon it's great, but have near zero spare time to assist :(
     
    Jellybeans likes this.
  43. nfip

    nfip Part of the Furniture Ski Pass: Gold

    Joined:
    Jul 24, 2006
    Messages:
    18,046
    Likes Received:
    17,124
    Location:
    South Co.
    I'm a fan .
    Appreciate your efforts.
    sorry I didnt give any feedback ,
    got lost in the black hole ;)
    will have a good look later when I get a chance, but yeah good to see it confirms what we pretty much know.
     
    Jellybeans likes this.
  44. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    That’s all good, no worries.
     
  45. MarzNC

    MarzNC One of Us Ski Pass: Silver

    Joined:
    Jun 8, 2017
    Messages:
    452
    Likes Received:
    282
    Location:
    Raleigh, NC, USA
    First time seeing this thread. I like it! My favorite course as a statistics grad student was called Descriptive Statistics. This is a great example of where applying common sense to numerical data can yield conclusions that are relevant.

    Seeing the trend over time, meaning multiple years, is always more useful than just one data point.
     
    Jellybeans likes this.
  46. Lifes2good

    Lifes2good One of Us

    Joined:
    Jan 21, 2012
    Messages:
    434
    Likes Received:
    436
    Location:
    Sydney, Aust
    I have a question - if a forecaster gives a 40cm range in their prediction how do you determine whether they have been more accurate than a forecaster who gives a precise number or a smaller range? (Apologies if this question has already been asked).

    ETA - to clarify, I see you use a median number, but do you apply weightings of any kind to indicate accuracy in the range?
     
  47. Jellybeans

    Jellybeans Walking on a dream Ski Pass: Gold

    Joined:
    Aug 24, 2015
    Messages:
    7,798
    Likes Received:
    8,429
    Location:
    Box Hill, Vic.
    The way I run it is to purposefully not discriminate against the use of ranges. and single numbers. I as you have stated, use a median number for the ranges. I don't weight these, because the idea is that they are the most confident in the region around the median for the final term.

    It's not an exact science, as opposed to the ranges by the Bureau which are 75% chance to 25% chance.
    If you have any ideas to improve, feel free to suggest.
     
  48. Lifes2good

    Lifes2good One of Us

    Joined:
    Jan 21, 2012
    Messages:
    434
    Likes Received:
    436
    Location:
    Sydney, Aust
    I'll have a think.

    It appears from the two years of data that yr.no is hands down the most accurate and they quote a single number which I find more useful in a practical sense compared with a 40cm range.
     
  49. Lifes2good

    Lifes2good One of Us

    Joined:
    Jan 21, 2012
    Messages:
    434
    Likes Received:
    436
    Location:
    Sydney, Aust
    @Jellybeans - would you mind PMing me the rest of the data tables from last season? I can only see the first four in the thread. I'm having a play with adding a 'discernment' factor based on the size of the range forecast.
     
    Jellybeans likes this.
  50. gmanpadre

    gmanpadre Hard Yards Ski Pass: Gold

    Joined:
    Jul 26, 2015
    Messages:
    52
    Likes Received:
    53
    The Plowking likes this.