Error uploading data to the openENTRANCE platform

[ryan-oreilly]

Hello all and thank you for taking a look at this issue.

I am working on CS1 and attempted to upload our data validated by the nomenclature package to the platform and received the following error code after uploading:

2021-01-25 14:46:12,077 [MainThread ] [ERROR] 'datetime.datetime' object has no attribute 'split'

The format used in the subannual column to represent the first hour for all days in January (i.e. Representative hour ) is:
01-01 00:00+01:00

After reading through #45 & #46 I believe this might have something to do with the timestamp not being written as a Representative hour. Is this where this error code is coming from?

A subset of the data is provided in the zip file.

Thank you for the support

github.example.csv.zip

[danielhuppmann]

Thank you for reporting this issue, seems to be a bug - will look into it! cc @peterkolp

[danielhuppmann]

I ran the nomenclature.validate() on the data file you uploaded (not just the example attached above) and you have an entry 2000-12-01 00:00:00 in the region column. Therefore, the validation routine should return False (region not defined in the nomenclature). The "bug" (in the sense of an error instead of the validation log message) is caused by over-ambitious parsing by pyam, see the related issue raised opened in that repository.

cc @peterkolp @meksor

Closing this issue, because it is not a bug of the nomenclature but caused by incorrect data.

[ryan-oreilly]

Thank you for your assistance with this. After checking I noticed when I converted my csv to xlsx, the only file format that it permitted me to upload, Germany's region DEC00 got converted to Dec-00 or 2000-12-01.

Is there a way to upload a csv?

[danielhuppmann]

We don't support csv for file uploads to the IIASA Scenario Explorer because there are too many potential issues: column separation (, vs ;), various character sets, line ending characters, ...

I suggest that you use pyam or pandas to convert from csv to xlsx, that should resolve the issue - plus it is anyway preferable to have a scripted preparation of your files rather than a manual processing step.