question: how to start services for passwordReset or dataExtract ?

[traveller195]

Hello @matkoniecz @tomalrussell

sorry for the many questions here. But I guess you can help me :-)

Must the services for passwordReset or dataExtract started manually?

../app/src/api/services/passwordReset.ts
../app/src/api/services/dataExtract.ts

Compare server error: colouring-cities/colouring-dresden#40

current data extract page: http://colouring.dresden.ioer.de/data-extracts.html

[matkoniecz]

I would look into logs on server (or browser console) for passwordReset

data extract is scheduled via cron from what I remember

[traveller195]

hi @matkoniecz
thanks.
some additional questions:

how can I update my PM2 after changing some of the environment variables in environment.config.js ?
There are a lot of suggestions like "reload / restart" "--update env" etc. but it was not working for me.
I changed path for extracts (extracts_directory)...

second questions about data extracts:
It is not commited yet, but I changed a bit in extract_data.py, included new fields into COPY request in "export_attributes.sql" etc.
I also created a new directory /var/www/colouring-dresden/downloads, which a use also as extract_directory.
After running the python scipt manually the zipped file is saved to this directory and also the new extract was added to "bulk_extracts" db table as new row. In the frontend (currently only in staging env) the new extract is visible. But after the click on it, it will not be downloaded. Just "page not found". So I guess the app page routing was not successful. But I assume, that /downloads is also your taken path for extracts, so it should/ could be working?

perhaps it is because of missing extract_directory (currently it is not set, and I have those troubles in updating environment , see question one) to explain it: in the python file I coded the extract directory static without env variable to get the script started for the moment.

is it a problem of access rights to folder /downloads?

sorry for my long message here. Maybe you have a great idea which solves my issues

thanks in advance, all the best
Theodor

[traveller195]

@matkoniecz
okay, I could solve it now:

I should store the data extracts in the public folder:

/var/www/colouring-dresden/app/public/downloads

Should set permissions to 775
sudo chmod -R 775 downloads

And I changed extracts directory env variable manually in terminal.

So, everythin should work now.
Except my question, how to update environment variables from ecosystem.config.js with PM2 ...

best, Theodor

[traveller195]

@matkoniecz @tomalrussell
Hello again,
so, as I wrote in my message before, it is now working, but only partly.
The problem now is, that after each (currently still manually started python script) extraction of the data into app/public/downloads folder, I have to

1. change permissions to 775
2. stop pm2 , do new npm run-script build and start pm2 again
   only then the zipfile is downloadable.

How can I refresh it to provide e.g. weekly data extracts without stopping the frontend each time?

best, Theodor

[traveller195]

hello @matkoniecz and @tomalrussell

I am still working on this issue, how to finally start the data extract, so that it will be fine with the automated way weekly started from cron job.

My problem is still the same, see above.

I can run the proper python script

/var/www/colouring-dresden/maintenance/extract_data/extract_data.py

by execute: "python3 extract_data.py" on command line (later running it with cron.weekly)

I use currently the following directory to save the extracts:
/var/www/colouring-dresden/app/public/downloads

see here as well:
https://github.com/colouring-cities/colouring-dresden/blob/4e2713ac7357be347177a14eede1b01e16f741e4/maintenance/extract_data/extract_data.py#L27

so there is the new created extract:

also the database row was inserted, so it also appears in the list of available extracts on the download page.
But when I click to download this new file, there is only the message "page was not found"

But all older extracts can be downloaded successful.
When a stop pm2 and start "npm run-script build" ... and start pm2 again... now, also the new extracts is found and can be downloaded.

I did several searchs for "React and public folder" or concerning webpack or bundling.
But I guess it should be fine and I am not sure where the problem occures.

Questions:
1.) Which path do you use for saving data-etracts?
2.) Do you use the environment variables or another way?
3.) Should I change some permissions rules for the /public folder, perhaps?

maybe you can help me to find a solution without stopping and starting the app for each new data extract weekly again and again ;- )

thanks in advance

all the best,
Theodor

[traveller195]

to correct it: I want to update environments from ecosystem.config.js (not environment.config.js)

[polly64]

@traveller195 do loop @mdsimpson42 as well.