Ordering of commodities in investment loop

[tsmbland]

The investment loop runs through the all commodities, performing investments to meet each of these commodities in turn.

SVD commodities are always run first, as we know the demands for these commodities from the input data. I don't think it matters which order these are run in. As we perform investments in processes to meet demands for SVD commodities, we will generate demands for non-SVD commodities (i.e. commodities that are input flows to SVD-producing processes). We can then perform investments to meet these commodity demands.

However, at this stage it's crucial that we get the ordering of commodities correct. E.g. by the time gas-producing technologies are invested in, we must have already performed all of the investments in gas-consuming technologies so we know what the gas demand is (and there shouldn't be any investment in gas-consuming technologies after this point).

In some models it might be possible to work out what order we should run the commodities in by setting up a graph of commodity flows between processes and doing some kind of topological sort. However, this assumes that there are no circularities in the commodity flows (i.e. a DAG).

If there are circularities, then I think the only solution is an iterative approach (i.e. running over all/some of the non-SVD commodities multiple times) until an equilibrium is reached. However, I worry about instabilities (investments spiraling out of control*)

So I think the options are:

• mandate non-circular commodity flows and see if a DAG approach works for ordering the commodities (I think circular commodity flows are a desired feature in MUSE2, so probably no point going down this route)
• try an iterative approach, see what happens, and come up with an approach to prevent instabilities if necessary

* I've seen this in MUSE1, which is why it's important not to have circular flows, although this isn't documented and there are no checks in place to prevent this

E.g.
The system needs to invest in electricity to meet consumer demands. It finds that the cheapest way to do this is the generate electricity from hydrogen. However, this hydrogen needs to be produced, and the cheapest way to do so is by using electricity. This then generates more electricity demand, which the system decides to meet using more hydrogen, which needs to be produced using electricity etc. etc.

[tsmbland]

I think for the MVP we prohibit the case of circular flows (we won't explicitly check this, just rely on the user to be sensible). Then the algorithm could be something like this:

1. Loop over the SVD commodities and perform investments to meet demands (I think this can be in any order)
2. We sum all inputs flows for the resulting assets to get demands for a set of non-SVD commodities. (Question: once we've performed investments, how do we know what the flows are? Can we use the "potential utilizations" calculated as part of the investment step, or do we need to do a mini dispatch optimization?)
3. Within this set of commodities, we loop over the SED type commodities, performing investments to meet demands
4. Again, sum input flows (just for these assets?) to get a set of demands for non-SVD commodities.
5. Again, within this set of commodities we loop over the SED type commodities. Some of these commodities may have previously appeared in step 3. In this case I'm not sure what we do. Do we scrap the assets that were invested in previously and perform new investments to meet the total demand, or do we keep them but invest in new assets to meet the additional demand?
6. Keep iterating until there are no new assets are invested in with SED inputs

Could well end up more complicated than this, but something to start with

[tsmbland]

For now though, lets just focus on SVD commodities

We'll have a loop looking like the following (very roughly), that just goes through the SVD commodities once

for commodity in svd_commodities:
	get commodity demand from the input data
	
	get assets serving this commodity
	calculate marginal cost of assets (note 1)
	
	get processes serving this commodity
	calculate marginal cost of processes (note 1)
	
	asset pool for the next year starts off empty	
	calculate unserved demand (all demand is unserved to start with)
	while unserved demand is positive
		calculate potential utilization of assets
		calculate potential utilization of processes
		
		calculate decision value of assets (note 2)
		calculate decision value of processes (note 2)
		
		add best asset/process to the asset pool for the next year
		calculate demand served by the asset pool (note 3)
		calculate unserved demand

Note 1: I'm wondering if the marginal cost of assets is equal to the marginal cost of the equivalent process? In this case, we just calculate the marginal cost for processes

Note 2: For now, the decision value is just equal to LCOX

Note 3: Do we assume the potential utilization calculated above?

Note 4: This will not necessarily work if there are assets/processes with multiple end-use commodities. We should restrict this case for now and come back to this later

Note 5: Some values are calculated at the timeslice level (marginal cost, potential utilization), others at the annual level (LCOX/decisions)

@alexdewar

[alexdewar]

This is v useful. Thanks!