Embed Size (px)
Citation preview
Follow Up. Evaluate & Optimize.Alexander Nordling, Application Consultant
Why Follow Up?
• To:• Find Errors
• Improve settings
• Decide investment/deinvestment
• Because:• Improve forecast quality
• Optimize costs
• Condition:• Fast feedback
• Easy data gathering
• Zooming possibility
Follow up
Forecasting
Follow Up in Aiolos• Designed to be flexible
+ Can do more or less anything- Settings to configure (one-time installation)
• Once configured, clear aim to be a 1-click-system
Day-ahead vs Intraday Manual vs Automatic Users specific
Value of working weekends & nights?Different weather suppliers & Weather stations Alternative forecasts / calculation models
Evaluate in different ways
• Analyze forecasts in diagram, table or with statistical key figures• Toggle between different resolution (original, daily, weekly, etc)• Toggle between multiple levels
Demo
Follow up basics• Diagram, Statistics
• Docking windows
• Follow up of one model
• Zoom down/Zoom in
• Multiple weather
Evaluate Forecast Providers
ME (Mean Error)
ME% (Relative Mean Error)
MAE (Mean Absolute Error)
RMSE (Root of Mean Squared Errors)
Max AE
Counts
𝑀𝑎𝑥 𝐴𝐸 = 𝑀𝑎𝑥(1
𝑛
𝑖=1
𝑛
𝑒𝑖
𝐶𝑜𝑢𝑛𝑡𝑠 =𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑓𝑜𝑟𝑒𝑐𝑎𝑠𝑡 𝑣𝑎𝑙𝑢𝑒𝑠
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑣𝑎𝑙𝑢𝑒𝑠=𝑥
𝑛
𝑀𝐸 % =1
𝑛
𝑖=1
𝑛σ𝑖=1𝑛 𝑦𝑖 − 𝑓𝑖σ𝑖=1𝑛 𝑦𝑖 𝑖
=1
𝑛
𝑖=1
𝑛𝑒𝑖𝑦𝑖
y=measured values, f=forecast value, e=error, n=number of values, x= 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑓𝑜𝑟𝑒𝑐𝑎𝑠𝑡 𝑣𝑎𝑙𝑢𝑒𝑠 𝑤ℎ𝑒𝑛 𝑚𝑒𝑎𝑠𝑢𝑟𝑒𝑑 𝑣𝑎𝑙𝑢𝑒𝑠 𝑒𝑥𝑖𝑠𝑡
Simulate forecasts for analysis
Follow up can be based on • Actual made forecasts
• Simulations/Recreations
Forecasting
Follow up
Simulations
Why simulate?
• No waiting time for actual forecasts
• To make an investment decision
• Backtesting, e.g. test a new setting
Simulate without “cheating”
In a real life situation you are probably missing data for a period back in time (energy measurements and weather observations)
Possible to set up restrictions to make it as real-life like as possible
Missingdata
Forecast
Real-life
Forecast
Simulation
Set restriction
Demo
Follow up advanced• Extract settings
• Simulations/Recreated forecasts
• Temporal settings
• Mode/User settings
• Different time resolutions
• Different time periods
• Different models
• Multiple weather selections
Automatic Follow Up
• Let the system search for errors• Alerts you if any of the forecast’s quality is
exceeding the set boundaries• Run automatically every week/day/hour• Gives immediate feedback• Set optimal weights dynamically
Thank you!Alexander Nordling, Application Consultant
Automatic Follow Up
• Let the system search for errors• Alerts you if any of the forecast’s quality is exceeding the set boundaries• Run automated Follow up every week/day/hour• Gives immediate feedback
Forecast quality
• Monitor forecast quality• Automated reports
QUALITY SERIVICE
Evaluate different forecast horizons
• In Aiolos you can evaluate the quality of different time horizons.
• Example: • 45 minutes ahead,
• 6 hours ahead,
• Day ahead,
• Week ahead, etc
Tuesday Wednesday
0
00 - 12 12 - 24 00 - 2412 - 24
72 h forecasts
Load forecast exported Hours to
ignore
Extraction
Several Forecasts for the Same Period?
Tuesday Wednesday Thursday Friday
00 - 12 12 - 24 00 - 24 00 - 24 00 - 2412 - 24
72 h forecast
Load forecast exported Hours to ignore Extraction
Start of extract
2 days ahead forecast: [email protected]
Morning forecast: [email protected]
11 o clock (spot) forecast: [email protected]
Afternoon forecast: [email protected]
Weather Weighting
W Forecast 1
Optimal Weighted E Forecast
W Forecast 2
W Forecast 3
W Forecast 4
W Forecast 5
E Forecast 1
E Forecast 2
E Forecast 3
E Forecast 4
E Forecast 5
Configuration of multiple weather forecast
Weather station 1
Forecast series
Station 2 Station 3 Station 4 Station 5
Forecast series Observation series
Climate series Climate series
F O F O F O F O
Cl Cl Cl Cl Cl Cl Cl Cl
Weighting multiple weather forecasts
Weather station 1
Load forecast model True load
0,30 ∗ 𝐿𝑜𝑎𝑑 𝑓𝑜𝑟𝑒𝑐𝑎𝑠𝑡 1 + 0,1 ∗ 𝐿𝑜𝑎𝑑 𝑓𝑜𝑟𝑒𝑐𝑎𝑠𝑡 2 + ⋯+ 0,3 ∗ (𝑙𝑜𝑎𝑑 𝑓𝑜𝑟𝑒𝑐𝑎𝑠𝑡 5) + 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
Automatic calculated coefficients
Weather station 2
Weather station 3
Weather station 4
Weather station 5
30 % 10 % 20 % 10 % 30 % Constant
Extract Settings
Tuesday Wednesday Thursday Friday
00 - 12 12 - 24 00 - 24 00 - 24 00 - 2412 - 24
72 h forecast
Load forecast exportedHours to
ignoreExtraction Continuing forecast…
Start of extract
Forecast made
Load forecast
exported
Hours to
ignore
Extractio
n
Forecast valid period
Continuing forecast…
Start of extract
2 days ahead forecast: [email protected]
Morning forecast: [email protected]
11 o clock (spot) forecast: [email protected]
Afternoon forecast: [email protected]
Extract Settings
Forecast made
Load forecast
exported
Hours to
ignore
Extractio
n
Forecast valid period
Continuing forecast…
Start of extract
• The starting point
Start of extract
Extract
• Length of forecast to evaluate
• Look for prog-file older than…
Look for prog-file created within this time period
Hours to ignore
Load forecast
exported
Extract Settings
24h Rest of 72h forecast
24h Rest of 72h forecast
24h Rest of 72h forecast
24h 24h 24h
12h6
12h6
12h6
Tuesday Wednesda
y
Thursday Friday
Several days’ extracts collected to a timeseries
Evaluating day ahead
Extract Settings
Evaluating 3 days ahead
Extract Settings
Evaluating weekend leave
Extract Settings
Evaluating weekly forecast
Follow up on Follow up Promises
Day-ahead vs IntradayManual vs Automatic
User specific results
Day-aheadExtract = 24Period to ignore = 12Search period = 6Latest
IntradayExtract = 1Period to ignore = 0:45Search period = 2Latest
Value of working weekends & nights?
WeekendExtract = 24-72Period to ignore = 12Search period = 6-54LatestManual/automatic
NightExtract = 1-24Period to ignore = 1-12Search period = 1-6LatestManual/automatic
Weight Tolerances
Max sum of abs weights: |w1| + … + |wn | > 1,25
Min sum of weights: w1 + … + wn < 0,75
Max constant: | c | > 0,05 × mean_load
Dynamic Fractions
• Dynamic fractions:
• Symbol: • Sums up with 𝛼𝑓 + 𝛽• Change values at hours ahead relative
forecast start date
• Evaluation of:
• Separate sub series• Models• Weather points• External
• Optimal subsets of series
Dynamic Fractions vs other tools
Multiple weather forecasts- Max 5 weather stations
+ Simple configuration
+ Support for missing weather
• Alternative models+ Switch models
+ Simple configuration
- Weighting not possible
Dynamic fractions+ Flexible
+ Unlimited number of sub series (weather, models)
+ Possibility to combine weighting of both weather
providers and weather sites
+ Weighting of multiple models+ Export each alternative+ Combine external forecasts- Expanded tree structure- More administrative work
Follow up on Follow up Promises
Alternative forecasts / calculation models
Different weather suppliers & Weather stations
