Skip to content

REST data collection frequency #8142

Description

@calestyo

Hey.

If I recall correctly, then Tigran said during my talk in Amsterdam, that the REST would internally query all data, when it is itself queried (via a HTTP GET) - like every time fresh data.

I just tried a bit and it seems that at least the /pools/{name}/usage data is however collected by dCache periodically once every 60s and afterwards, it returns always the same data.

E.g. the mode=… output of psu ls pool -l updates basically instantaneously after I change a pool's mode, while the corresponding poolMode in the REST JSON takes well up to 60s.

I've also seen that there's:

# Timeout for transfer info collection
frontend.service.transfers.timeout=1
(one-of?MILLISECONDS|SECONDS|MINUTES|HOURS|DAYS)frontend.service.transfers.timeout.unit=MINUTES
frontend.service.transfers.maxCacheSize=1000

# Timeout for cell info collection
frontend.service.cell-info.timeout=1
(one-of?MILLISECONDS|SECONDS|MINUTES|HOURS|DAYS)frontend.service.cell-info.timeout.unit=MINUTES

# Timeout for restore info collection
frontend.service.restores.timeout=1
(one-of?MILLISECONDS|SECONDS|MINUTES|HOURS|DAYS)frontend.service.restores.timeout.unit=MINUTES
frontend.service.restores.maxCacheSize=1000

# Timeout for alarms info collection
frontend.service.alarms.timeout=1
(one-of?MILLISECONDS|SECONDS|MINUTES|HOURS|DAYS)frontend.service.alarms.timeout.unit=MINUTES

# Timeout for pool info collection
frontend.service.pool-info.timeout=1
(one-of?MILLISECONDS|SECONDS|MINUTES|HOURS|DAYS)frontend.service.pool-info.timeout.unit=MINUTES

# Timeout for pool history info collection
frontend.service.pool-history.timeout=1
(one-of?MILLISECONDS|SECONDS|MINUTES|HOURS|DAYS)frontend.service.pool-history.timeout.unit=MINUTES

And indeed, changing e.g. frontend.service.pool-info.timeout gives me faster updates. 🥳️

Now:

  1. Can I somehow find out which of the properties correspond to which REST paths?
    I guess:
    • /cells is controlled by frontend.service.cell-info.timeout only
    • /domains by frontend.service.domain-info.timeout only
    • /pools/{name}/usage I'm not sure... is it frontend.service.pool-info.timeout only (including for the cellData field in it, which is like the entries from /cells) or is it also frontend.service.pool-history.timeout?
    • not sure about /space/tokens and /space/linkgroups do they update every second? or on every request?
  2. Could frontend easily support a true collect-live-on-GET (however often that is)? Perhaps in a way so that one needs to enable this mode, and only do so on an internal only instance of frontend?
  3. Are there any REST paths which are truly always updated already on every request, respectively which don't fall under the above settings?

Thanks,
Chris.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions