Skip to contents

Dataset Generation Tutorial for gcamreport v6.0*

Source: vignettes/Dataset_Generation_Tutorial_v6-0.Rmd
Dataset_Generation_Tutorial_v6-0.Rmd

ATTENTION: this tutorial is compatible with gcamreport v6.0*, i.e, with gcamreport v6.0.0, gcamreport v6.0.1, and gcamreport v6.0.0-gas releases. For other releases, please check in the Tutorials drop-down menu in the main bar.

newline

Example 1: generate a dataset from a database

Assume you have a database called myDb in a folder named dbFolder and are willing to generate a standardized dataset for several scenarios: scen1, scen2, and scen3. With the run function, it will be generated and automatically saved in the same folder where myDb is.

  1. Load gcamreport library.
  1. Store the database path and name, the query path, the desired project name, and the considered scenarios in variables.
dbpath <- "/path/to/database"
dbname <- "gcamdb_name"
prjname <- "awesomeProj.dat"
scen <- c('scen1', 'scen2', 'scen3')

Notice that the extension is included in the project name. Accepted extensions: .dat & .proj.

Note: in case you followed the Docker installation, you should store your database inside the gcamreport folder and your root directory is called /app. Thus, your database path will be something like /app/path/to/database.

  1. Generate the dataset until the desired year. In this example, 2050. By default is 2100 and it should be at least 2025.
run(db_path = dbpath, query_path = querypath, db_name = dbname, prj_name = prjname, scenarios = scen, final_year = 2050, launch_ui = FALSE)

Note: project generation requires approximately 30 minutes.

Notice that the dataset will automatically be saved in .csv and .xlsx at /path/to/database/awesomeProj_iamc_report.csv and /path/to/database/awesomeProj_iamc_report.xlsx.

This procedure will generate a project file at /path/to/database/dbname_prjname.dat with all the necessary queries that you can use as indicated in Example 2.

The terminal will output the performed verifications and their final status.

newline

Example 2: generate a dataset from a project

Assume you have a project called myProj.dat and are willing to generate a standardized dataset out of it. With the run function, it will be generated and automatically saved in the same folder where myProj.dat is. Notice that myProj.dat should have all the required queries for the standardized dataset generation. If you are not sure you have all of them or you need to generate the project, see Example1.

  1. Load gcamreport library.
  1. Store the project path in a variable.
mypath <- "/path/to/project/myProj.dat"

Notice that the extension is included. Accepted extensions: .dat & .proj.

Note: in case you followed the Docker installation, you should store your database inside the gcamreport folder and your root directory is called /app. Thus, your database path will be something like /app/path/to/database.

  1. Generate the dataset until the desired year. In this example, 2050. By default is 2100 and it should be at least 2025.
run(project_path = mypath, final_year = 2050, launch_ui = FALSE)

Notice that the dataset will automatically be saved in .csv and .xlsx at /path/to/database/awesomeProj_iamc_report.csv and /path/to/database/awesomeProj_iamc_report.xlsx.

The terminal will output the performed verifications and their final status.

newline

Example 3: save or not the output and specify the file format or the directory

Assume you are in the situation of any of the previous examples but want to either not save the output, save it in .csv, .xlsx, or both extensions.

  1. Load gcamreport library.
  1. Use example1 database or example2 project description and add all the extra parameters that you consider in the run function. Specify the output saving options through the save_output parameter
## -- save the dataset in CSV and XLSX format
run(..., save_output = TRUE)    # this is the default option

## -- save the dataset only in CSV format
run(..., save_output = 'CSV')

## -- save the dataset only in XLSX format
run(..., save_output = 'XLSX')

## -- do not save the dataset
run(..., save_output = FALSE)
  1. Use example1 database or example2 project description and add all the extra parameters that you consider in the run function. Specify the output directory and output file name through file_name parameter. This will save the output in the indicated path as .csv and .xlsx. To modify this, see the above explanation.
## -- save the dataset in '/desired/directory' and in a file called 'awesomeOutput'
run(..., file_name = '/desired/directory/awesomeOutput')

newline

Troubleshooting for the run() function

A) Error on run("path/to/your/data/myData.dat")

In your R console, you might see this error:

> run("path/to/your/data/myData.dat")
[1] "Loading project..."
[1] "Loading data, performing checks, and saving output..."
[1] "ag_demand_clean"
Error in rgcam::getQuery(prj, "demand balances by crop commodity") :
  getQuery: Query demand balances by crop commodity is not in any scenarios in the data set.

This problem is due to a wrong path specification.

Possible solution: make sure that you specified correctly the path. In addition:

  • In case you are using gcamreport package following the R installation, try to copy the whole path to your data, for instance C:\Users\username\Documents\path\to\your\data\myData.dat if you are using a Windows distribution.

  • In case you are using gcamreport package following the Docker installation:

    1. make sure that your data is inside the gcamreport folder.

    2. make sure that you type correctly the path to your gcamreport folder when running the docker image (5th step in the Docker section)

    3. make sure that you are pointing correctly to your data. For example, if in the gcamreport folder you have a folder called amazingData with your dataset myData.dat, you should refer to it as

# option 1: full path
run("/app/amazingData/myData.dat")

# option 2: partial path
run("amazingData/myData.dat")

newline

B) Error related to system when using the Docker installation.

Once the R console is opened, you might see this message after introducing any command:

System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Warning message:
In system("timedatectl", intern = TRUE) :
   running command 'timedatectl' had status 1

Possible solution: simply type Ctrl+C and run your command again.