Studies not getting the EMIKAT ID assigned #20

DenoBeno · 2019-09-24T09:33:55Z

probably related to: #6, #5

We have just noticed that new studies aren't getting the EMIKAT ID assigned anymore. This has probably been the case for a while already. Please check & fix.

DenoBeno · 2019-09-24T09:53:13Z

See also #19

patrickkaleta · 2019-09-24T11:21:47Z

The problem has to be on Emikat side, since POSTs and PUTs currently don't work even when using the Emikat Swagger interface (so without any CSIS involvement). I already informed Heinrich and he is working on it.

BTW: I configured the CSIS helpers module to log some status messages when Studies are being sent to Emikat. You can find those message here: https://csis.myclimateservice.eu/admin/reports/dblog?type%5B%5D=EmikatHelperFunctions&hostname=

There you can see that it last worked last week on 18.9.2019. Then, until today, there were no other attempts to send Study updates to Emikat.

patrickkaleta · 2019-09-24T15:18:17Z

@humerh solved the problem. Our POST and PUT requests are being processed properly again by Emikat.

DenoBeno · 2019-09-24T16:49:23Z

Excellent. I hope it will stay this way.

…

On Tue, 24 Sep 2019, 17:26 Patrick Kaleta, ***@***.***> wrote: Closed #20 <#20>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20?email_source=notifications&email_token=AAWTC7QF7IE5L2ZKHT4DNRLQLIWTZA5CNFSM4IZ5GFV2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOTZ2URTI#event-2658486477>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWTC7RQL3C5FCPE2NY2RFTQLIWTZANCNFSM4IZ5GFVQ> .

p-a-s-c-a-l · 2019-09-25T09:11:02Z

@humerh It still doesn't seem to work for this study.

patrickkaleta · 2019-09-25T09:25:15Z

@humerh It still doesn't seem to work for this study.

This study doesn't yet have a data package and study area. That's why Emikat is not informed about this study and therefore there is also no emikatID available. (The csis helpers module checks for those relevant fields and informs Emikat only if they are available/have changed.)

A while ago I discussed with @humerh what would be considered relevant information for Emikat to be able to start the calculations. That is also mentioned in the readme.md of the Emikat repository.

I will add this information about the relevant fields in the readme of the csis helpers module as well.

patrickkaleta · 2019-09-25T09:27:55Z

Additionally, I can also create another Drupal log message indicating each time a request is not send to Emikat after a Study update. But that could spam our log messages, so I would only use this for a limited time.

p-a-s-c-a-l · 2019-09-25T11:56:22Z

Ah, ok, my fault. Now there is a study id but still no data from EMIKAT, e.g. Population Exposure: https://service.emikat.at/EmiKatTst/api/scenarios/3209/feature/tab.CLY_EL_POPULATION_INTERPOLATED.2016/table/data?rownum=1000

How long does that take @humerh ?

DenoBeno · 2019-09-25T13:15:19Z

In my opinion, you should show messages on the top of the page: emikat triggered, EMIKAT not triggered because... At least for admins and developers...

…

On Wed, 25 Sep 2019, 13:56 Pascal Dihé, ***@***.***> wrote: Ah, ok, my fault. Now there is a study id but still no data from EMIKAT, e.g. Population Exposure: https://service.emikat.at/EmiKatTst/api/scenarios/3209/feature/tab.CLY_EL_POPULATION_INTERPOLATED.2016/table/data?rownum=1000 How long does that take @humerh <https://github.com/humerh> ? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20?email_source=notifications&email_token=AAWTC7QIXG3WO36CLBFVQ7LQLNGWNA5CNFSM4IZ5GFV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7RT6BY#issuecomment-534986503>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWTC7XZQF76UPV2Y2ATFBDQLNGWNANCNFSM4IZ5GFVQ> .

p-a-s-c-a-l · 2019-09-25T14:24:17Z

Although the EMIKAT ID 3209 is now assigned to study 33 and EMIKAT ID 3189 to study 35, EMIKAT calculation was not triggered. But the logs show:

patrickkaleta · 2019-09-26T08:02:14Z

In my opinion, you should show messages on the top of the page: emikat triggered, EMIKAT not triggered because... At least for admins and developers...

Probably a good idea for developers and admins. I'll have a look at how to implement something like that.

patrickkaleta · 2019-09-26T09:59:30Z

Although the EMIKAT ID 3209 is now assigned to study 33 and EMIKAT ID 3189 to study 35, EMIKAT calculation was not triggered. But the logs show:

Emikat is indeed triggered and a batch job is created and queued (Denis and I checked this via an Emikat Client), but it seems that a previous batch job got stucked. Heinrich is not in the office today, but we already called to inform him of this issue. He'll get it fixed soon hopefully.

DenoBeno · 2019-09-26T15:32:06Z

I think that this is fixed now. In emikat client, all teh batch jobs have been finished...

Trying to trigger the https://csis.myclimateservice.eu/study/33/ again.... Yes, it's working.

I'm curious to see how long it takes to run it all now...

DenoBeno · 2019-09-26T15:47:09Z

aproximately 6 minutes.

For the future reference - if it isn't done in 10-15 minutes, then we have a problem.

DenoBeno · 2019-09-26T16:19:40Z

Ropening this.

While EMIKAT does finish the calculation batch job, some data is not available for the study #33. E.g., all of these are empty:

Unsurprisingly, nothing is shown on csis either.

Same for study 35. :-(

DenoBeno · 2019-09-26T16:44:42Z

Oh, come on. I tried to get a calculation working for linz and nothing happened. Logs say:

Nothing in EMIKAT batch-jobs.

The EMIKAT did start calculation when I changed the area again.

For linz, I do get the population, but no local effect data. :-(

For the reference: "result = 0" in emikat client means that no data was generated.

Batch took 5-6 minutes again.

DenoBeno · 2019-09-26T17:34:10Z

I tried for Napoli in the hope that there we would have all the data.. Result: a new type of error.

Napoli, second try, with different area.. First the server reconnected....

Then the UA fraction and population and all other layers built as they should:

Which doesn't mean that they are also shown on CSIS. Hm.... OK, this time EMIKAT is producing the data as required, so we also have an issue with the CSIS app...

p-a-s-c-a-l · 2019-09-27T05:35:22Z

Which doesn't mean that they are also shown on CSIS. Hm.... OK, this time EMIKAT is producing the data as required, so we also have an issue with the CSIS app...

The table and thus EMIKAT API do not receive the variables values we have agreed upon, see clarity-h2020/csis-helpers-module#14 (comment).

So probably that's an issue of the taxonomy / paragraph / whatever? We need value not meaning. I tried to fix it in csis-helpers-module, but it didn't work.

DenoBeno · 2019-09-27T07:24:59Z

Which doesn't mean that they are also shown on CSIS. Hm.... OK, this time EMIKAT is producing the data as required, so we also have an issue with the CSIS app...

The table and thus EMIKAT API do not receive the variables values we have agreed upon, see clarity-h2020/csis-helpers-module#14 (comment).

So probably that's an issue of the taxonomy / paragraph / whatever? We need value not meaning. I tried to fix it in csis-helpers-module, but it didn't work.

Ah, I wasn't aware of that one. OK, this shouldn't be that difficult to fix. If nothing else works, we'll replace the paragraph with node...

patrickkaleta · 2019-09-27T12:55:19Z

Which doesn't mean that they are also shown on CSIS. Hm.... OK, this time EMIKAT is producing the data as required, so we also have an issue with the CSIS app...

The table and thus EMIKAT API do not receive the variables values we have agreed upon, see clarity-h2020/csis-helpers-module#14 (comment).

So probably that's an issue of the taxonomy / paragraph / whatever? We need value not meaning. I tried to fix it in csis-helpers-module, but it didn't work.

The problem with the meaning and value field should be resolved for now.
@p-a-s-c-a-l: Can you confirm that your apps are getting correct values for emission_scenario, time_period,...?

...

Ah, I wasn't aware of that one. OK, this shouldn't be that difficult to fix. If nothing else works, we'll replace the paragraph with node...

@DenoBeno no, this should not be necessary. Extracting fields out of the paragraphs turned out to be quite straightforward, so IMO the current structure is fine and doesn't need to be changed again.

patrickkaleta · 2019-09-27T13:02:38Z

I have now also improved the status notifications as proposed by Denis. In addition to the log messages in the BE, Admins and developers now see directly in the FE, if Emikat has been triggered or not, after updating a Study (as well as whether or not a Recalculation will be evoked).

Only exception: changing the Study area doesn't show those notifications, since the Study is updated inside the map-component via REST and no page-reload is triggered. In those cases there will be only the log messages in the BE available.

p-a-s-c-a-l · 2019-09-27T14:25:30Z

Can you confirm that your apps are getting correct values for emission_scenario, time_period,...?

yes, it is.

DenoBeno · 2019-09-30T11:45:59Z

So probably that's an issue of the taxonomy / paragraph / whatever? We need value not meaning. I tried to fix it in csis-helpers-module, but it didn't work.
...

As explained in clarity-h2020/csis-helpers-module#14 (comment), this is not really a bug, rather a design feature that we'll need in the future. For now, you can ignore it, I have configured the "DP variable meanings" entries in such a way that they deliver the same information as "DP variables".

That's OK as long as we only use the variables with EMIKAT. The moment we start implementing the variables support for other resources, there will be more than one "DP variables" entry possible for each "DP variable meanings" entry. E.g., the "historical" time period will map to "Baseline" for EMIKAT and to "historical" for the heat hazards, see clarity-h2020/data-package#47 (comment) .

DenoBeno · 2019-10-01T16:53:48Z

Guess we can close this issue?

patrickkaleta · 2019-10-02T07:19:24Z

Guess we can close this issue?

Yes, we can close this. Emikat IDs are assigned to studies once they have all the relevant fields filled out.

DenoBeno assigned patrickkaleta Sep 24, 2019

DenoBeno added bug Something isn't working SHOWSTOPPER Feature or bug, that, if not addressed, renders the CSIS essentially useless labels Sep 24, 2019

p-a-s-c-a-l added this to the D1.4 CLARITY CSIS v2 milestone Sep 24, 2019

patrickkaleta mentioned this issue Sep 24, 2019

EMIKAT calculation not triggered by CSIS #19

Closed

patrickkaleta closed this as completed Sep 24, 2019

p-a-s-c-a-l reopened this Sep 25, 2019

p-a-s-c-a-l assigned humerh Sep 25, 2019

p-a-s-c-a-l mentioned this issue Sep 25, 2019

How to choose which "variable combinations" to use in a study? clarity-h2020/csis#101

Closed

p-a-s-c-a-l mentioned this issue Sep 25, 2019

Support for Exposure Data [EE API] #10

Closed

p-a-s-c-a-l mentioned this issue Sep 26, 2019

CSIS Demo for European Week of Regions and Cities clarity-h2020/csis#105

Closed

32 tasks

DenoBeno closed this as completed Sep 26, 2019

DenoBeno reopened this Sep 26, 2019

p-a-s-c-a-l assigned DenoBeno Sep 27, 2019

patrickkaleta closed this as completed Oct 2, 2019

p-a-s-c-a-l added the BB: Catalogue of ER & AO Catalogue of Elements at Risk and Adaptation Options Building Block label Oct 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Studies not getting the EMIKAT ID assigned #20

Studies not getting the EMIKAT ID assigned #20

DenoBeno commented Sep 24, 2019

DenoBeno commented Sep 24, 2019

patrickkaleta commented Sep 24, 2019 •

edited

Loading

patrickkaleta commented Sep 24, 2019

DenoBeno commented Sep 24, 2019 via email

p-a-s-c-a-l commented Sep 25, 2019

patrickkaleta commented Sep 25, 2019

patrickkaleta commented Sep 25, 2019

p-a-s-c-a-l commented Sep 25, 2019

DenoBeno commented Sep 25, 2019 via email

p-a-s-c-a-l commented Sep 25, 2019 •

edited

Loading

patrickkaleta commented Sep 26, 2019

patrickkaleta commented Sep 26, 2019

DenoBeno commented Sep 26, 2019

DenoBeno commented Sep 26, 2019

DenoBeno commented Sep 26, 2019 •

edited

Loading

DenoBeno commented Sep 26, 2019 •

edited

Loading

DenoBeno commented Sep 26, 2019

p-a-s-c-a-l commented Sep 27, 2019

DenoBeno commented Sep 27, 2019

patrickkaleta commented Sep 27, 2019

patrickkaleta commented Sep 27, 2019

p-a-s-c-a-l commented Sep 27, 2019

DenoBeno commented Sep 30, 2019

DenoBeno commented Oct 1, 2019

patrickkaleta commented Oct 2, 2019

Studies not getting the EMIKAT ID assigned #20

Studies not getting the EMIKAT ID assigned #20

Comments

DenoBeno commented Sep 24, 2019

DenoBeno commented Sep 24, 2019

patrickkaleta commented Sep 24, 2019 • edited Loading

patrickkaleta commented Sep 24, 2019

DenoBeno commented Sep 24, 2019 via email

p-a-s-c-a-l commented Sep 25, 2019

patrickkaleta commented Sep 25, 2019

patrickkaleta commented Sep 25, 2019

p-a-s-c-a-l commented Sep 25, 2019

DenoBeno commented Sep 25, 2019 via email

p-a-s-c-a-l commented Sep 25, 2019 • edited Loading

patrickkaleta commented Sep 26, 2019

patrickkaleta commented Sep 26, 2019

DenoBeno commented Sep 26, 2019

DenoBeno commented Sep 26, 2019

DenoBeno commented Sep 26, 2019 • edited Loading

DenoBeno commented Sep 26, 2019 • edited Loading

DenoBeno commented Sep 26, 2019

p-a-s-c-a-l commented Sep 27, 2019

DenoBeno commented Sep 27, 2019

patrickkaleta commented Sep 27, 2019

patrickkaleta commented Sep 27, 2019

p-a-s-c-a-l commented Sep 27, 2019

DenoBeno commented Sep 30, 2019

DenoBeno commented Oct 1, 2019

patrickkaleta commented Oct 2, 2019

patrickkaleta commented Sep 24, 2019 •

edited

Loading

p-a-s-c-a-l commented Sep 25, 2019 •

edited

Loading

DenoBeno commented Sep 26, 2019 •

edited

Loading

DenoBeno commented Sep 26, 2019 •

edited

Loading