Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3151

Data Discovery/Server Discovery Service Implementation

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • SRCnet
    • Hide

      As expressed in SP-3000, there is currently no clear way forward to prevent latency of the visualisation of big data cubes within the SRC Net data lake. As these files could be in different repositories, a preliminary data cube transfer (or stream) where the parser server is located (usually, tools use a client-server architecture to parse big files) could produce a high-latency starting phase to start the visualisation and, also, provide a possible resources problem in the cache storage area and a network use

      A discovery service could prevent this movement, allowing remote operations where the data is located.

      The real implementation of the service allows a dynamic identification of tool servers (if a new server is deployed, it will be offered for connections) and, also, it will allow future extensions to cover high-availability and load-balance features (offering the best parser server at every time)

      Show
      As expressed in SP-3000 , there is currently no clear way forward to prevent latency of the visualisation of big data cubes within the SRC Net data lake. As these files could be in different repositories, a preliminary data cube transfer (or stream) where the parser server is located (usually, tools use a client-server architecture to parse big files) could produce a high-latency starting phase to start the visualisation and, also, provide a possible resources problem in the cache storage area and a network use A discovery service could prevent this movement, allowing remote operations where the data is located. The real implementation of the service allows a dynamic identification of tool servers (if a new server is deployed, it will be offered for connections) and, also, it will allow future extensions to cover high-availability and load-balance features (offering the best parser server at every time)
    • Hide

      AC: Adaptation of an existing library and implementation of a service in line with the SP-3000 provided specification

      Show
      AC: Adaptation of an existing library and implementation of a service in line with the SP-3000 provided specification
    • 24.2
    • PI24 - UNCOVERED

    • PI18-PB SRC-DM SRC-MiniNode SRC-Vis

    Description

      This is the continuation of https://jira.skatelescope.org/browse/SP-3000

      The exact definition of the protocol will be provided by SP-3000. This feature implies the real implementation of the service on top of a TAP or similar library used by the metadata management system.

      This has two parts:

      • Compilation of the metadata needed about the status of the Tool parser servers (e.g. VisIVO servers available in the SRC Net in line with the proximity to the data storages where a particular file is located)
      • Implementation of an extension of ObsCore TAP (or similar) with the new metadata needed by tools to make a client-server connection to their available parser servers

      Attachments

        Issue Links

          Structure

            Activity

              People

                r.bolton Bolton, Rosie
                Jesus.Salgado Salgado, Jesus
                Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (0%)

                  Feature Estimate: 0.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete00.0
                  Total00.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel