MUSE XII engagement

[alexdewar]

MUSE XII engagement

This page is for discussing the plans for the current MUSE2 engagement. For now, I'll just put some discussion points here for our meeting on Weds. If anyone has anything else to add, feel free to leave a comment.

The main thing to be decided is what features should be tackled next and in what order.

Summary of previous engagement

• Discussion page: MUSE XI engagement #512
• Previous proposal (private link)

Most of the planned tasks for this engagement were achieved, though some features planned for the second milestone were not. Other unplanned work was carried out (e.g. the revamp of the dispatch optimisation and many small features). Many of the features planned for the second milestone ended up being in the first release (v2.0.0) instead. See here for full list of issues closed for first release.

The two stretch goals for the second milestone were #509 and #523. There wasn't time to start on these, though these could potentially be worked on for this engagement. We will need guidance on how to implement them though.

Plans for current engagement

Now that we have a first release with a basic version of MUSE2 working, we should be making releases much more regularly going forward. I suggest we aim for a new release every one or two months while we're actively working on MUSE2, which should mean we get at least two more releases out during this engagement. I don't think we need to decide what goes in each release at this point though; let's just prioritise first.

@tsmbland and @martinstringer suggested a list of features we may want to consider next, so I'll include these here as I think it's a good starting point for discussion:

• Allow dependency loops between commodities #794 (I have concerns about how we'd actually do this -- let's discuss)
• Flexible inputs/outputs (== Add support for flexible assets #360?)
• Add commodity trade parameters #509
• "Non-LCOE objectives (capital costs, comfort etc.)" (NB: we have NPV, but it's broken: NPV is broken #716)
• Add support for "weighted" decision rule #544
• "Agent cost limits" (we already have agent_cost_limits.csv -- done?)
• Allow for adding process investment constraints #124
• "Storage representation (e.g. variable timeslice-level balancing)" (I guess this means batteries etc.?)

Some other big features to discuss:

• Allow agents to compete for demand #523
• Implementing "scopes" feature #795

We should also discuss features/fixes that are less exciting that need to be done anyway, e.g.:

• Fixes and better validation for process flows #931
• Refactor how process maps are represented and used #880
• Allow zero demand in the demand file #850

Input/validation:

• Allow for setting both upper and lower bounds for process availabilities #363
• Warn users about assets which will not be commissioned during the simulation #400
• For every commodity of "SVD" type, there must be at least one asset in the base year that can produce the commodity for every region where the SVD>0 for the base year. #169

Other tooling?

• Another repo for data analysis tools? Set up new repo for data analysis #752
• How best to help users add `muse2` to their `PATH`? #925
• Tooling to help users run different versions of MUSE2 #924

Other questions

• How much time do we actually have? We've provisionally allocated 60 days
• Is @tsmbland's availability likely to be the same as last time (~50% FTE)?
• We should schedule regular catch-ups again. Do we want to stick to a fortnightly schedule?

[ahawkes]

This is as far as I have got with a list. Note they are NOT in order of preference, and for several I do not yet have viable solutions..

1. Alternative price calculation method.
   Problem: At the moment we rely on shadow prices. But in complex systems these can be odd (e.g. the two_outputs example model, where there are zeros in strange places because the model is being forced to produce outputs that it doesn’t actually want via the process with two fixed outputs).
   Possible Solution: Use petgraph to order commodities to support direct marginal, scarcity and capex/fixed-costs inclusive commodity pricing options. Need to start at the top of the system and work down to service demands. Need to handle circularities (only recursive methods work?), and processes that have more than one SED/SVD output (can be done, but needs rules worked out).

2. Get NPV investment working.
   Problem: At the moment we are using marginal prices for commodities. This often is not enough to trigger investment. So often we find no profitable options among those available and the algorithm (correctly) fails with an error.
   Possible Solution: Change the way prices are calculated, to (a) include fixed and capital costs in some way (similar to MUSE1?), and/or (b) include a scarcity pricing algorithm that pushes price upwards at peak times. This could require price from 1 above, and even that may not be enough to trigger investment.

3. Circularities in the graph.
   Problem: It is common to find circular dependencies in energy systems. At present MUSE2 exits if it finds one of these.
   Possible Solution: Enable investment appraisal where there are circular dependencies in the graph. Note - case where circular commodity is not linked further down into the system (this is the case in the example I've provided), which means demand cannot be known (or rather, demand depends on investment and vice versa). Case where circular commodity is linked further down into the system (complete demand still cannot be known). Case of a big circularity that encompasses a large part of several supply chains (e.g. electrolyser H2 being used in a refinery).

4. Implement explicit input of capacity tranche sizes (if we want this?), mothball_years, and divisible/non-divisible assets.

5. Implement capacity addition limits (annual addition, annual growth/seed, total capacity limit by region/s).

6. Allow both upper and lower bounds on activity. And allow upper and lower bounds at different temporal resolutions (e.g. an annual upper bound applied over the whole year, and time slice level upper bounds too).

7. Implement commodity prod/cons/net constraints. Initially a dispatch issue, but also has implications for investment appraisal where investment signals may be positive for a candidate asset, but it cannot be used in dispatch due to a commodity limit. Vice versa, a candidate asset that is needed to achieve of commodity requirement may have negative investment signals.

8. Implement trade.

9. Implement scopes. Allow users to define scopes, perhaps have some pre-defined (e.g. all time slices, all regions, etc). Commodity and process investment constraints can then be associated with a scope, which defines how they are applied.

10. Implement flexible assets.

11. Implement investment appraisal decision rules: weighted, lexico.

12. Implement investment appraisal objectives: load-weighted or time-weighted marginal cost.

13. Implement CAPTIVE and COMPETE commodity types.
    Problem: At the moment demand is considered to be captive to an agent, but multiple process types can serve the demand as chosen by investment appraisal tools. But in reality often agents compete in a market without a pre-determined amount of demand to serve, or agents consider all processes at once when trying to serve a captive demand.
    Possible Solution: Introduce a distinction between commodity types – those where agents compete to serve demand and those where an agent is assigned a captive demand and must serve it. Current investment appraisal tools are just about (see LCOX point below) adequate for the CAPTIVE case, but the COMPETE case requires some changes – though a simple version might not be difficult (e.g. competition on LCOX, lowest one wins with capacity installed up to the agent’s capex limit or capacity addition limit or demand limit – whichever is smallest).

14. Change LCOX calculation?
    Problem: LCOX method currently maxes out dispatch (because the alternative is the VoLL). This may result in higher LCOX than necessary because in some cases an one can just not serve demand in expensive time slices (Assuming the a better technology would pick up that time slice later in the appraisal) or could consider multiple technologies to serve demand simultaneously. (issue linked to captive demand versus compete demand?).
    Possible Solution: Create an option for agents to consider multiple candidate assets (of different process types) to serve CAPTIVE demand. LCOX optimisation now includes all candidate assets with their capacities and dispatch as decision variables (much bigger optimisation problem – could be too slow).

[alexdewar]

Right, so I've had a go at prioritising @ahawkes's tasks above. I've thrown in a few small ones from the backlog, but there are plenty more small things that I haven't included for now in the interests of space. I'll make sure to include the important ones in the proposal.

Once we're happy with the priorities, we'll need to whittle it down to the tasks that we actually have time for when we write the proposal.

Are you broadly happy with the below @ahawkes?

Priority	Task	Issue	MoSCoW	Size	Need guidance?	Notes
1	Fixes and validation for process flows	#931	Must have	Small	No	Important to get this right. We don’t want bugs!
2	Reduced costs output		Must have	Small	No	Handy for work on circularities problem etc.
3	Captive/compete commodity types	#523	Must have	Medium	Yes
4	Circularities problem	#794	Must have	Large	Yes	May be easier to do captive/compete thing first. Might be required for trade feature to be useful.
5	Set up new repo for analysis	#752	Should have	Small	No
6	Trade	#509	Should have	Medium	Yes
7	Fix NPV	#716	Should have	Medium	Yes	@ahawkes has suggested solving this by changing the way we calculate prices
8	mothball_years and divisible assets	#911	Should have	Small	No
9	Allow upper and lower bounds on activity	#363	Should have	Small	No
10	Capacity addition limits		Could have	Small(?)	No	I’m not sure what this actually entails, so we’ll need an issue description
11	Implement commodity constraints	#143	Could have	Medium	Don’t know	I’m not sure if we know how to take this into account for investment
12	Scopes	#795	Could have	Medium	Yes	I think the most important thing here is figuring out use cases, so we can break it down into separate features. Need to consider the user interface for this carefully so it doesn’t make the input files more confusing than necessary.
13	Flexible assets	#360	Could have	Medium	Yes	I think we can figure out how to add the constraints, but not sure how it affects investment
14	Consider multiple assets for LCOX		Could have	Medium	Yes	Bit speculative. More info needed. Don’t think this is high priority.

[alexdewar]

Let me know if this seems sensible to you @ahawkes and I'll make a start on the proposal document.

We don't have to get it exactly right (and we will be working on things in parallel anyway), but as long as the things you really care about are near the top, I think we should be ok to go. Things at the bottom will probably not make the cut.

[ahawkes]

Looks good. Maybe just move flexible assets to be above trade?
Like I said in the meeting, I think the smaller things here could be tackled in the between-times while we're thinking how to deal with bigger problems.

[alexdewar]

Definitely. We'll be working on a range of things -- just wanted to check I'd captured the most important big features from your list.

Thanks!