1

I have a bunch of historical data (csv) which I want to make accessible through sth-comet. The data is the history of water levels from multiple revers. The data is not provided live, but more or less on a daily basis and contains all the historic records for multiple days.

What I did so far was:

  1. Convert the data into NGSIv2 format data model with dateObserved: DateTime and waterlevel : number field
  2. Update/append the data into Fiware orion
  3. Create a subscription for sth-comet for the entity type
  4. Access the historical data in sth-comet (wrong time)

With this I now have the problem that the "rcvTime" is of course the time when sth-cometreceived the data. Is there a way that I can "overwrite" that attribute or is there a better solution? I also looked at cygnus on inserting data but I think the underlying problem is the same. I could not find any hint in the avaiable documentation.

cisk
  • 589
  • 3
  • 13
Christoph S
  • 697
  • 1
  • 6
  • 29

1 Answers1

1

In the case of using Cygnus NGSIMongoSink and NGSISthSink you can use TimeInstant metadata in attributes to override received time with the time given in the metadata value.

Have a look to NGSIMongoSink documentation

By default, NGSIMongoSink stores the notification reception timestamp. Nevertheless, if (and only if) working in row mode and a metadata named TimeInstant is notified, then such metadata value is used instead of the reception timestamp. This is useful when wanting to persist a measure generation time (which is thus notified as a TimeInstant metadata) instead of the reception time.

or this similar fragment in NGSISTHSink documentation:

By default, NGSISTHSink stores the notification reception timestamp. Nevertheless, if a metadata named TimeInstant is notified, then such metadata value is used instead of the reception timestamp. This is useful when wanting to persist a measure generation time (which is thus notified as a TimeInstant metadata) instead of the reception time.

fgalan
  • 11,732
  • 9
  • 46
  • 89
  • Thanks for the hint about the `TimeInstant ` metadata, figured out that it also works directly with sth-comet, if the attribute metadata is present in the data I insert into orion. – Christoph S Aug 29 '19 at 07:13
  • Not fully sure... Looking into STH repository (https://github.com/telefonicaid/fiware-sth-comet/search?q=TimeInstant&unscoped_q=TimeInstant) it seems documentation doesn't mention this possibility. However, some parts of the code processes TimeInstant. Could you test it and tell if works (in which case I'll update STH documentation to include that), pls? Thanks! – fgalan Aug 29 '19 at 08:54
  • Yes, it works well. The rcvTime is correclty overwritten with the TimeInstant ```"waterLevel": { "type": "Number", "value": 555, "metadata": { "TimeInstant":{ "type": "DateTime", "value" : "2019-08-21T03:00:00.000Z" } } } ``` – Christoph S Aug 29 '19 at 10:49
  • Thanks for the feedback! Documentation modifcation done (among other things) in PR https://github.com/telefonicaid/fiware-sth-comet/pull/517 – fgalan Aug 29 '19 at 12:44