Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-3751

Advanced data discovery with the Rucio-datalink demonstrator using HiPS

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Feature
    • Could have
    • PI21
    • None
    • SRCnet
    • Hide

      Accessing data by using SODA cutouts in very huge cubes may be difficult due to lack of knowledge of regions of interest. Moment maps may help to discover features. But even those may be huge (with a typical peta byte to tera byte reduction). HiPS data organization (IVOA recommendation) is the only STANDARD way allowing to browse and access HUGE datasets by changing rapidly and on-line access scales. By storing HiPS of the original data  inside the Rucio DDM we will demonstrate how to manage efficiently huge sets of image data. Current R&D may extend this efficiency to 3D data. If this attempt is successfull we could try to  integrate HIPS computation within SRC nodes. This requires to integrate hipsgen or hipsgen-like softwares in the SRCnet platform

      Show
      Accessing data by using SODA cutouts in very huge cubes may be difficult due to lack of knowledge of regions of interest. Moment maps may help to discover features. But even those may be huge (with a typical peta byte to tera byte reduction). HiPS data organization (IVOA recommendation) is the only STANDARD way allowing to browse and access HUGE datasets by changing rapidly and on-line access scales. By storing HiPS of the original data  inside the Rucio DDM we will demonstrate how to manage efficiently huge sets of image data. Current R&D may extend this efficiency to 3D data. If this attempt is successfull we could try to  integrate HIPS computation within SRC nodes. This requires to integrate hipsgen or hipsgen-like softwares in the SRCnet platform
    • Hide

       

      • AC1 : Rucio allows to "load" HiPS built outside the SRC node. HiPS root URL are provided in DataLink by the rucio-datalink demonstrator  service
      Show
        AC1 : Rucio allows to "load" HiPS built outside the SRC node. HiPS root URL are provided in DataLink by the rucio-datalink demonstrator  service
    • 1.5
    • 1.5
    • 0
    • Team_ORANGE
    • Sprint 5
    • Hide

      Outcomes uploaded on Confluence
      https://confluence.skatelescope.org/x/FoOKDw

      Show
      Outcomes uploaded on Confluence https://confluence.skatelescope.org/x/FoOKDw
    • PI23 - UNCOVERED

    • PI20-PB

    Description

      Add "moment maps 2D-HiPS" access in  the nodes for Huge Cubes.

      The Inclusion  of moment maps HiPS in the recursive DataLink service ( described in feature SP-3750) is already done within the scope of SP-3750 (see 23rd of november demo. They are stored externally (at CDS) and demonstrate some benefit of the fonctionality.

       

      When this feature is achieved generation of HIPS inside the SRCnet can be the next step (PI 22 ?)

      Attachments

        Issue Links

          Structure

            Activity

              People

                r.bolton Bolton, Rosie
                F.Bonnarel Bonnarel, Francois
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (100.00%)

                  Feature Estimate: 1.5

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete417.0
                  Total417.0

                  Dates

                    Created:
                    Updated:

                    Structure Helper Panel