Commodities with levies double-counted in reduced cost calculation

[alexdewar]

For commodities with levies, such as CO2 in the example models, these levies are included in the reduced cost calculation for assets twice: once via Asset::get_operating_cost() and once when we multiply flows by prices (for CO2 in the examples, the price will be the same as the levy).

This is wrong. We need to figure out a principled way to exclude these commodities from one or other of these calculation steps.

@ahawkes has suggested excluding OTH-type commodities from the flow-related calculation. This needs a bit of thought though, as there may be implications to doing this (for example, SED and SVD commodities can also have levies which can inform their prices too).

Marking as "on hold" until we can figure it out.

[ahawkes]

Also to exclude "primary output" commodities from the flow-related bit of the calculation. A

…

________________________________ From: Alex Dewar ***@***.***> Sent: 01 October 2025 17:08 To: EnergySystemsModellingLab/MUSE_2.0 ***@***.***> Cc: Hawkes, Adam D ***@***.***>; Mention ***@***.***> Subject: [EnergySystemsModellingLab/MUSE_2.0] Commodities with levies double-counted in reduced cost calculation (Issue #877) [https://avatars.githubusercontent.com/u/23149834?s=20&v=4]alexdewar created an issue (EnergySystemsModellingLab/MUSE2#877)<#877> For commodities with levies, such as CO2 in the example models, these levies are included in the reduced cost calculation for assets twice: once via Asset::get_operating_cost() and once when we multiply flows by prices (for CO2 in the examples, the price will be the same as the levy). This is wrong. We need to figure out a principled way to exclude these commodities from one or other of these calculation steps. @ahawkes<https://github.com/ahawkes> has suggested excluding oth-type commodities from the flow-related calculation. — Reply to this email directly, view it on GitHub<#877>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC37JLI7T7WSRFJWXGAQXFL3VP4ANAVCNFSM6AAAAACIAVXG26VHI2DSMVQWIX3LMV43ASLTON2WKOZTGQ3TIMRRHE2DINI>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[alexdewar]

Yep! I've opened a separate issue for that: #878

[alexdewar]

Overview

I'll expand on this issue description a bit, because I think we need to have a discussion before we start trying to fix this. If anyone else has thoughts too, that would be appreciated (@dalonsoa @tsmbland @Aurashk).

Note that I've opened some more issues re the reduced costs calculation (see #876). We'll tackle these one by one. The results will therefore be wrong until they're all fixed, but it does make working on the problem easier and it makes it easier to review.

The problem is that we are double-counting levies for commodities which have them -- such as CO2 in the example models -- when calculating reduced costs for existing assets (though going forward we will use this method to calculate them for candidates too: #879). There are two places they crop up in the calculation: once when we calculate operating cost and then again when we calculate what we call "revenues from flows" ($price \times flow$ -- for commodities with levies, the price is usually the same as the levy). I'll explain these in more detail below, but note that there are two places we could exclude CO2 from the calculation. The thing is that while we could do either with some justification and fix the results for our example models, I want to make sure that we come up with a solution that works for future models too. For example, it is totally possible to have a levy on an SED commodity, in which case the price may be the same as the levy, so a solution that just sets price to zero for OTH commodities, such as CO2 in the examples, won't work in this case.

If we have to just hack it for now to get the results, then we can do that for the sake of having something working for the MVP, but we should have an issue open to fix it in future. I'd rather not do this if possible, though. My concern is that if we pick a bad heuristic for e.g. excluding certain commodities from the flow revenues calculation then it will work for all the toy examples we use for the next year or so, but will silently break models further down the line and will give broken results without users noticing.

We could also constrain the system for now to prevent users from creating models with configurations which will pose problems for whatever hack we come up with. For example, we could prohibit adding levies to SED and SVD commodities for now, which would then mean we could set the prices of OTH commodities to zero for the calculation. Better to prohibit potentially problematic models than to give bad results that someone might publish! We could always lift these restrictions later if we come up with a better solution.

Reduced cost calculation

This is described in detail in the documentation, but we've written the code in a slightly different way to the equation in the docs (though they are supposed to be equivalent!). Note that there is an option to adjust prices for scarcity, which we are not doing by default, because it causes problems (#677), so we're using the non-adjusted versions of the equations.

I'll spell out what we're doing to make sure we're on the same page and that it sounds right to everyone.

We break it down into what we call "operating cost" and "revenues from flows".

• Operating cost: Made up of the process's variable operating cost + flow "costs"¹ + levies applied to flows ($flow \times levy$)
• Revenues from flows: $flow \times price$ for every process flow² (remember input flows are negative, so these are effectively subtracted)

Note that the method for calculating operating cost is also used when calculating cost coefficients for the dispatch optimisation, so if we decide we want to change this part of the code, we should make sure we're not breaking the cost coeff calculation at the same time.

Possible solutions

1. Set the price for OTH-type commodities to zero in the flow revenues calculation, as mentioned above. Suggested by @ahawkes. Has problems, as mentioned.
2. Same as 1, but also prohibit adding levies to SED and SVD commodities to prevent dubious results
3. Use a variation of the operating cost calculation that excludes levies when calculating reduced costs (presumably we'd still want to include them when calculating cost coeffs)
4. ...something else? There are probably lots of variations on these we could do

It's worth remembering that, in the current implementation, commodity prices are calculated as either commodity shadow prices or levies, but if there are both present, then the larger of the two will be used. So option 3 will give a different result for commodities that have both. That feels more right to me, because it actually incorporates both levies and shadow prices in some way (but my intuitions about economics are probably off 😉).

Any thoughts welcome!

Footnotes

1. == the cost column in process_flows.csv. I think they're zero for all the flows in the example models. ↩

2. Although we should exclude the primary output commodity for LCOX, but aren't doing so currently (The price for the commodity of interest should be zero when calculating reduced costs for assets with LCOX #878) ↩

[alexdewar]

PS -- I'm wondering now whether this is also a problem for the cost coeff calculation, which is similar to the reduced cost calculation, except that only input flows are considered for the "flow revenues" part. If there was an input commodity with an associated levy, I think it would be double-counted in that calculation too (once for the operating cost bit and once for the flow revenues).

[ahawkes]

Thanks Alex, Intuition says your solution (3) is the correct one. As the levy is includes via the flows calc, we don't need to add it in again in the operating cost calc (but we do still need to add variable operating cost). I think that the way we're picking up prices (commodity balance dual) should always factor in the levy (if the levy existed in the previous MSY - there is another case here where it didn't - but sticking to our principle of previous MSY prices this is at least explainable). Though there may be an issue with the sign of the price/flow. CO2 price should increase the activity cost, not decrease it - which I think is what the current formula does. Can we try this? I'm not sure what you mean re the cost coeff calc? Can you remind me which bit this relates to?

…

________________________________ From: Alex Dewar ***@***.***> Sent: 02 October 2025 11:13 To: EnergySystemsModellingLab/MUSE_2.0 ***@***.***> Cc: Hawkes, Adam D ***@***.***>; Mention ***@***.***> Subject: Re: [EnergySystemsModellingLab/MUSE_2.0] Commodities with levies double-counted in reduced cost calculation (Issue #877) [https://avatars.githubusercontent.com/u/23149834?s=20&v=4]alexdewar left a comment (EnergySystemsModellingLab/MUSE2#877)<#877 (comment)> PS -- I'm wondering now whether this is also a problem for the cost coeff calculation, which is similar to the reduced cost calculation, except that only input flows are considered for the "flow revenues" part. If there was an input commodity with an associated levy, I think it would be double-counted in that calculation too (once for the operating cost bit and once for the flow revenues). — Reply to this email directly, view it on GitHub<#877 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC37JLKPUZEX35YB2CWXZ733VT3FVAVCNFSM6AAAAACIAVXG26VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGNRQGMYTINZQHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[alexdewar]

Thanks Alex,
Intuition says your solution (3) is the correct one. As the levy is includes via the flows calc, we don't need to add it in again in the operating cost calc (but we do still need to add variable operating cost).
I think that the way we're picking up prices (commodity balance dual) should always factor in the levy (if the levy existed in the previous MSY - there is another case here where it didn't - but sticking to our principle of previous MSY prices this is at least explainable).
Though there may be an issue with the sign of the price/flow. CO2 price should increase the activity cost, not decrease it - which I think is what the current formula does.
Can we try this?

By activity cost, I guess you mean reduced cost? We're currently calculating it as $operating\_ cost - revenue\_ from\_ flows$. It sounds like you're saying we should be adding it instead (the signs might be different to in your formula)?

I'm not sure what you mean re the cost coeff calc? Can you remind me which bit this relates to?

The cost coeffs for the dispatch optimisation, as in the docs. We use the same operating cost routine here as for the reduced costs.

[alexdewar]

Sorry, I think I know what you meant about the sign of the price now... I think in general we're doing the right thing, but there might be something wrong with the way we're handling levies here.

In the case of CO2 for our examples:

• It is only ever an output flow, never an input flow¹
• It's a tax, so it should decrease the flow revenues
• However, the price is positive, because we just use the value of the levy as a price for commodities without shadow prices

There's something wrong here. Would it make sense to give CO2 a negative price instead? I'm wondering whether this should be the case for all levies that are applied to output flows, but not input flows, because a levy on an input flow should also reduce the flow revenue.

That might work, but it smells wrong to me for a couple of reasons:

1. Where does that leave us for levies that apply to both inputs and outputs (balance_type is net)? We don't want to have to conditionally multiply the price by -1 when calculating flow revenue! That's technically possible (by tracking prices' provenance), but a horrid design
2. Does it really make sense for the price of CO2 listed in the output file to be negative in this case, either in terms of economics or what users would expect?

In relation to point 1, it's never been entirely clear to me when it would actually make sense to have a balance_type of net. When would you want to tax a commodity flow by the same amount, regardless if it is being consumed or produced?

Given these problems, I'm wondering if it actually makes sense for us to give CO2 a "price" within MUSE2 at all. If it didn't have a price at all, then the double-counting problem would also go away (it'd just be skipped in the flow revenue calculation). If we went down this route, then we'd probably have to prohibit giving levies to SVD and SED commodities as having shadow prices and levies together would complicate things.

Footnotes

1. I've also noticed that the balance_type of CO2 is net for all the examples, but this doesn't seem right; as it's a tax on CO2 production, surely it should be prod? In practice it doesn't matter as CO2 isn't actually consumed by any processes in the examples, but if it were (e.g. with some kind of carbon capture gizmo) then presumably we wouldn't want this CO2 consumption to be taxed. ↩

[ahawkes]

Where is the CO2 price that appears in the commodity_prices output coming from at the moment? There is no commodity balance constraint on CO2 in this model...so I don't understand why this is there. I think I need to understand this before I can try to get my head around all the +ve and -ve values flying around here. But I think it might go something like this: if prices are generated by a commodity balance constraint, these should always be used, regardless of commodity type (SED, SVD, OTH). And where there is no commodity balance constraint, there should be no price in commodity_prices, and the levy should be applied to the input or output or both. We should define a positive levy as a cost, and a negative levy as an incentive. That means the equation to apply these in the LCOX case is flow*levy regardless of if its input or output (which is different to what's in the docs!).

…

________________________________ From: Alex Dewar ***@***.***> Sent: Thursday, October 02, 2025 15:52 To: EnergySystemsModellingLab/MUSE_2.0 ***@***.***> Cc: Hawkes, Adam D ***@***.***>; Mention ***@***.***> Subject: Re: [EnergySystemsModellingLab/MUSE_2.0] Commodities with levies double-counted in reduced cost calculation (Issue #877) [https://avatars.githubusercontent.com/u/23149834?s=20&v=4]alexdewar left a comment (EnergySystemsModellingLab/MUSE2#877)<#877 (comment)> Sorry, I think I know what you meant about the sign of the price now... I think in general we're doing the right thing, but there might be something wrong with the way we're handling levies here. In the case of CO2 for our examples: * It is only ever an output flow, never an input flow1 * It's a tax, so it should decrease the flow revenues * However, the price is positive, because we just use the value of the levy as a price for commodities without shadow prices There's something wrong here. Would it make sense to give CO2 a negative price instead? I'm wondering whether this should be the case for all levies that are applied to output flows, but not input flows, because a levy on an input flow should also reduce the flow revenue. That might work, but it smells wrong to me for a couple of reasons: 1. Where does that leave us for levies that apply to both inputs and outputs (balance_type is net)? We don't want to have to conditionally multiply the price by -1 when calculating flow revenue! That's technically possible (by tracking prices' provenance), but a horrid design 2. Does it really make sense for the price of CO2 listed in the output file to be negative in this case, either in terms of economics or what users would expect? In relation to point 1, it's never been entirely clear to me when it would actually make sense to have a balance_type of net. When would you want to tax a commodity flow by the same amount, regardless if it is being consumed or produced? Given these problems, I'm wondering if it actually makes sense for us to give CO2 a "price" within MUSE2 at all. If it didn't have a price at all, then the double-counting problem would also go away (it'd just be skipped in the flow revenue calculation). If we went down this route, then we'd probably have to prohibit giving levies to SVD and SED commodities as having shadow prices and levies together would complicate things. Footnotes 1. I've also noticed that the balance_type of CO2 is net for all the examples, but this doesn't seem right; as it's a tax on CO2 production, surely it should be prod? In practice it doesn't matter as CO2 isn't actually consumed by any processes in the examples, but if it were (e.g. with some kind of carbon capture gizmo) then presumably we wouldn't want this CO2 consumption to be taxed. ↩ — Reply to this email directly, view it on GitHub<#877 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC37JLPB5NMR6PRMHN2FXB33VU32PAVCNFSM6AAAAACIAVXG26VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGNRRGU4DEMZWGU>. You are receiving this because you were mentioned.Message ID: ***@***.***>