The unit US$2005/(tCO2/yr) breaks the magclass unit convention, which does not support nested parentheses. Specifically

library(remind)
library(quitte)

a <- convGDX2MIF(gdx = './tests/testgdxs/fulldata.gdx')

as.quitte(a)
Error in magclass::`getNames<-`(`*tmp*`, value = sub(" \\(([^\\()]*)\\)($|\\.)",  : 
  Inconsistent names! Number of dots per name has always to be the same as it is separating different data dimensions
In addition: There were 21 warnings (use warnings() to see them)

as.quitte(a[,,'(US$2005/(tCO2/yr))', pmatch = TRUE, invert = TRUE])
# A tibble: 371,241 x 7
   model  scenario region variable    unit               period value
   <fct>  <fct>    <fct>  <fct>       <fct>               <int> <dbl>
 1 REMIND default  CAZ    Consumption billion US$2005/yr   2005 1123.
 2 REMIND default  CHA    Consumption billion US$2005/yr   2005 1625.
 3 REMIND default  EUR    Consumption billion US$2005/yr   2005 8759.
 4 REMIND default  IND    Consumption billion US$2005/yr   2005  520.
 5 REMIND default  JPN    Consumption billion US$2005/yr   2005 3031.
 6 REMIND default  LAM    Consumption billion US$2005/yr   2005 1867.
 7 REMIND default  MEA    Consumption billion US$2005/yr   2005  985.
 8 REMIND default  NEU    Consumption billion US$2005/yr   2005  942.
 9 REMIND default  OAS    Consumption billion US$2005/yr   2005 1324.
10 REMIND default  REF    Consumption billion US$2005/yr   2005  597.
# … with 371,231 more rows

By leaving the paradigm of SE level pooling by state of matter (solid, liquid, gas) with the updates by Renato (fossil and bio distinction on SE level), a lot of FE level reporting does anymore correctly reflect what REMIND is doing.

The reason for this is, that FE distinction by PE carrier (e.g, FE|Transport|Liquids|Biomass) is based on SE level pools that do not exist anymore. An example from the reporting (reportCrossVariables.R):

tmp <- mbind(tmp,setNames(
                   output[r,,"FE|Transport|Liquids (EJ/yr)"] 
                 * output[r,,"SE|Liquids|Biomass (EJ/yr)"]
                 / output[r,,"SE|Liquids (EJ/yr)"],  "FE|Transport|Liquids|Biomass (EJ/yr)"))

If REMIND decides to supply all bioliquids to FE buildings or industry, which it can do now (since the link seliqbio to fehoi is explicit), this will not correctly be reflected in the reporting anymore.

The same issue shows up for all various combinations of carriers and sectors. Possible problems that come to my mind:

• bounds on shares that are defined on the FE level are not in line with the reporting variables (that's how I ran into this issue).
• FE price signals are more difficult to interpret from the reporting.
• Actual tech investments and capacities are not in line with produced amounts.l

One of the reasons why this might be difficult to fix is backward compatibility. In the end, if backward compatibility is to be kept, FE reporting by PE carrier has to be split into the old (seliq, sega, seso) variables and a new one which correctly calculates shares on the basis of the new pools, i.e., seliqfos, seliqbio, sesofos, sesobio, segafos, segabio.

Hi,
I put this in after discussions with Lavinia and Björn, so that automatic policy cost reporting only happens when the the gdp_scenarios are the same between policy and reference runs. This was done so that policy cost weren't reported for nonsensical policy-reference pairs.
The output.R script policyCosts.R was supposed to be used to create specific policy costs reportings.
Should this be changed? What should to be done instead? @fbenke-pik @LaviniaBaumstark @bs538 @strefler

Error in [email protected][i, j, , drop = FALSE] : subscript out of bounds

"Useful Energy" in the transport sector is not consistent with the common understanding of this term. Instead, UE in the REMIND transport sector is a combustion car equivalent, i.e., from one unit of fepet, apCarPeT produces one unit of UE, while apCarElT produces around 3 units of UE etc.

I tried to fix it once but it proved more difficult than expected (as almost always). Please contact me if you care to fix this. Maybe also a thing for remind2 @Renato-Rodrigues

reporting.R quits early with:

[1] "Executing reporting.R"
Error in setNames(pric_emu_base * bio_tax_factor * tdptwyr2dpgj, "Price|Biomass|Bioenergy tax (US$2005/GJ)") :
  dims [product 19] do not match the length of object [0]
> traceback()
6: setNames(pric_emu_base * bio_tax_factor * tdptwyr2dpgj, "Price|Biomass|Bioenergy tax (US$2005/GJ)")
5: setNames(pric_emu_base * bio_tax_factor * tdptwyr2dpgj, "Price|Biomass|Bioenergy tax (US$2005/GJ)")
4: mbind(tmp, setNames(pric_emu_base * bio_tax_factor * tdptwyr2dpgj, 
       "Price|Biomass|Bioenergy tax (US$2005/GJ)")) at reportPrices.R#312
3: reportPrices(gdx, output, regionSubsetList)
2: mbind(output, reportPrices(gdx, output, regionSubsetList)[, t, 
       ]) at convGDX2MIF.R#51
1: convGDX2MIF(infile, file = outfile)

on REMIND 779c113d using REMIND library version 36.147.1.
Therefore the MIF file produced only contains a couple of lines.

If output is NULL to begin with (i.e., the function is run alone), it will contain names(regionSubsetList) afterwards (e.g., EUR and NEU). If it is not NULL, however, (i.e., the function is run from convGDX2MIX()), it won't.
Shouldn't there be a consistent way to handle this?

(Traces back to https://redmine.pik-potsdam.de/projects/mo/repository/16/revisions/13975/diff/remind/R/reportEmi.R)

This

The weighted global average for efficiencies produces bad results if no capacities are given for a specific technology.

e.g., in the current validation round:

Variable Intensity|Final Energy|CO2 is reported twice, with differing units and content.

This will break quitte::calc_addVariable() when it is interacting with these variables, but more importantly is very unclear.

From EDGE-T there are levelized cost markups provided to REMIND to reflect the total costs of transportation. Although very aggregated (passenger long distance, short-to-medium distance and the same for freight), the information might be interesting for the reporting.

The yellow points shouldn't be shown - replace with NA?

From compareScenarios pdf - The gaps in the figure shouldn't be there, right?

Build summary:

── R CMD check results ──────────────────────────────────── remind 36.171.2 ────
Duration: 2m 10.7s

0 errors ✔ | 0 warnings ✔ | 0 notes ✔
Package check successful! Please choose an update type :
1: major revision (for major rewrite of the whole package)
2: minor revision (for new features or improvements)
3: patch (for bugfixes and corrections)
4: only for packages in development stage
0: no version increment (only to use if version is already incremented!)

Number: 0
Updated .zenodo.json file
Error: $ operator is invalid for atomic vectors

Trace:

> traceback()
5: citation(package = d$get("Package"))
4: format(citation(package = d$get("Package")), style = "text")
3: fillCite(d)
2: package2readme(".")
1: lucode2::buildLibrary()

with lucode2 0.4.3 on R 3.6.3

This is at least the case in the USA in 2100, for a budget 600 scenario (e.g. /p/projects/remind/runs/sspsdp_validation/remind-standalone/output/SSP2-Budg600_2019-09-16_01.43.03)
Not fully clear whether this is a reporting issue or a model issue.
This may also impact the emissions accounting.

Warning by lucode2::buildLibrary:

W  checking whether package ‘remind’ can be installed (19.7s)
   Found the following significant warnings:
     Warning: Package 'lucode' is deprecated.
   See ‘/tmp/Rtmp4Ejs2U/remind.Rcheck/00install.out’ for details.

At the moment, this inhibits updating DESCRIPTION. lucode2 version 0.1.0

When using years > 2005 for the y parameter in compareScenarios, the script fails without a readable traceback.

> compareScenarios(
+   def_mifs,
+   "~/git/REMIND/core/input/historical/historical.mif", y=seq(2015, 2055, 5), y_bar = c(2015, 2030, 2050),
+   fileName="compareScenarioDefault_shortTerm.pdf")
Error in if (magclass::getNames(data_new, fulldim = TRUE)[["scenario"]] %in%  (from compareScenarios.R#163) : 
  argument is of length zero
> traceback()
1: compareScenarios(def_mifs, "~/git/REMIND/core/input/historical/historical.mif", 
       y = seq(2015, 2055, 5), y_bar = c(2015, 2030, 2050), fileName = "compareScenarioDefault_shortTerm.pdf")

Testing the remind package with old.gdx results in

── Failure (test-convGDX2mif.R:48:3): Test if REMIND reporting is produced as it
`runParallelTests(my_gdxs)` threw an error.
Message: subscript out of bounds ("FE|Transport|Pass|Liquids|Biomass (EJ/yr)")

Offending line:

I receive the following error while running reportTechnology()
a <- reportTechnology("inst/extdata/old.gdx")
Error in .dimextract(x, k, 3, pmatch = pmatch, invert = invert) :
subscript out of bounds ("FE|Transport|Pass|Road|LDV|Liquids (EJ/yr)")
In addition: Warning message:
In readGDX(gdx, name = c("v32_storloss", "v_storloss"), field = "l", :
v32_storloss not found in GDX! v_storloss returned
Called from: .dimextract(x, k, 3, pmatch = pmatch, invert = invert)

This

Warning messages:
In if (v_investcost[, t, te] == 0) { ... :
  the condition has length > 1 and only the first element will be used

I suggest either explicitly testing only the first region, if that is intended

if (v_investcost[1, t, te] == 0) {

or if it is not, testing if all regions have 0 data

if (all(v_investcost[1, t, te] == 0)) {

or at least one region has 0 data

if (any(v_investcost[1, t, te] == 0)) {

depending on what makes sense for this comparison.

It seems several variables got lost in the reporting for the detailed industry version. Specifically, I am missing
"Emi|CO2|Industry|Direct"
"Emi|CO2|Carbon Capture and Storage|Biomass|Energy|Demand|Industry"
Emi|CO2|Energy|Supply|Gross

There might be more.
@0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q [Michaja]

The problem is the existence of 28 non-default packages in the Imports list. People who run R>4 can use the cluster to build the package, or have a colleague do it for them. As a permanent solution we can either remove dependencies, or allow a few notes. This has to be discussed and decided upon collectively.

Silvia discovered a discrepancy in the .mif files between FE|Buildings|Hydrogen + FE|CDR|Hydrogen + FE|Industry|Hydrogen + FE|Transport|Hydrogen and FE|Hydrogen. Turns out this is exactly 0.000315 EJ/yr or 1e-5 TWa.

# A tibble: 13 x 11
   scenario       period region Buildings     CDR Industry Transport    Total reported      diff
   <fct>           <int> <fct>      <dbl>   <dbl>    <dbl>     <dbl>    <dbl>    <dbl>     <dbl>
 1 SSP2-PkBudg900   2005 CAZ     0.000315 0        0.0112   0         0.0115   0.0112  -0.000315
 2 SSP2-PkBudg900   2005 CHA     0.000315 0        0.0551   0         0.0554   0.0551  -0.000315
 3 SSP2-PkBudg900   2005 EUR     0.000315 0        0.0561   0         0.0564   0.0561  -0.000315
 4 SSP2-PkBudg900   2005 IND     0.000315 0        0.0154   0         0.0157   0.0154  -0.000315
 5 SSP2-PkBudg900   2005 JPN     0.000315 0        0.0152   0         0.0155   0.0152  -0.000315
 6 SSP2-PkBudg900   2005 LAM     0.000315 0        0.0220   0         0.0223   0.0220  -0.000315
 7 SSP2-PkBudg900   2005 MEA     0.000315 0        0.0213   0         0.0216   0.0213  -0.000315
 8 SSP2-PkBudg900   2005 NEU     0.000315 0        0.00920  0         0.00951  0.00920 -0.000315
 9 SSP2-PkBudg900   2005 OAS     0.000315 0        0.0262   0         0.0265   0.0262  -0.000315
10 SSP2-PkBudg900   2005 REF     0.000315 0        0.0200   0         0.0203   0.0200  -0.000315
11 SSP2-PkBudg900   2005 SSA     0.000315 0        0.0131   0         0.0134   0.0131  -0.000315
12 SSP2-PkBudg900   2005 USA     0.000315 0        0.0678   0         0.0681   0.0678  -0.000315
13 SSP2-PkBudg900   2005 World   0.00378  0        0.333    0         0.336    0.333   -0.00378

Shouldn't

the reporting of

"Tech|Electricity|Storage|Battery|For Wind"
"Tech|Electricity|Storage|Battery|For PV"
"Tech|Electricity|Storage|Battery|For CSP
Tech|Transport|Pass|Road|LDV|ICE|Capital Costs
Tech|Transport|Pass|Road|LDV|EV|Capital Costs
Tech|Transport|Pass|Road|LDV|H2|Capital Costs
Tech|Transport|Pass|Road|LDV|ICEH2|Op Costs
Tech|Transport|Pass|Road|LDV|EV|Op Costs
Tech|Transport|Pass|Road|LDV|H2|Op Costs

in reportTech needs to be changed.

Currently they report the REMIND-internal values for mix technologies (for storspv/storwind) or a fraction of a vehicle (for LDV), so they should not be reported like "real values", and never uploaded in project reports.

Solution:

1. change the existing names to:
   "Tech|Electricity|Storage|StorWind"
   "Tech|Electricity|Storage|StorSPV"
   "Tech|Electricity|Storage|StorCSP"
   Tech|Transport|Pass|Road|LDV|0.65 of an ICE|Capital Costs
   Tech|Transport|Pass|Road|LDV|0.65 of an ICEEV|Capital Costs
   Tech|Transport|Pass|Road|LDV|0.65 of an ICEH2|Capital Costs
   Tech|Transport|Pass|Road|LDV|0.65 of an ICEH2|Op Costs
   Tech|Transport|Pass|Road|LDV|0.65 of an ICEEV|Op Costs
   Tech|Transport|Pass|Road|LDV|0.65 of an ICEH2|Op Costs

2. upscale the LDV costs by dividing by 0.65

3. Calculate actual battery costs from the

4. Implement seperate battery technology, H2 electrolysis technology and H2 turbine technology that are required in different parts from storspv, storwind and storcsp