Uploaded image for project: 'SAFe Program'
  1. SAFe Program
  2. SP-1174

Design of data models and schemas for in-memory processing component interfaces

Change Owns to Parent OfsSet start and due date...
    XporterXMLWordPrintable

Details

    • Spike
    • Not Assigned
    • None
    • COM SDP SW
    • None
    • Data Processing
    • Hide

      Shared memory data models are an important enabler for work down the line. We need broad consensus on this to move forward.

      Show
      Shared memory data models are an important enabler for work down the line. We need broad consensus on this to move forward.
    • Hide
      • Document meta-data model. Arrow-style would be tensors plus tables, with associated typing and metadata information. Have a look at measurement set compatibility.
      • First data models appropriate for capturing meta-data required for SP-1158, both as documentation and expressed as Arrow schemas.
      Show
      Document meta-data model. Arrow-style would be tensors plus tables, with associated typing and metadata information. Have a look at measurement set compatibility. First data models appropriate for capturing meta-data required for SP-1158 , both as documentation and expressed as Arrow schemas.
    • 1
    • 33
    • 9.3

    Description

      In order to progress the prototyping of real-time processing using Apache Plasma as a shared memory data store, we need to agree on and document (a set of) memory data model schemas for data exchanged using (presumably) Apache Arrow. Specifically:

      • Visibilities (tensor?)
      • Per-observation / per-scan / per-visibility metadata

      Limited to what is needed for implementation of current user scenarios (see SP-1158).

      Attachments

        Issue Links

          is required by

          Enabler - An Enabler supports the activities needed to extend the Architectural Runway to provide future business functionality. These include exploration, architecture, infrastructure, and compliance. SP-841 Develop library code that implements data models and schemas following SP-1174

          • Not Assigned - Priority is not assigned yet.
          • Discarded

          Enabler - An Enabler supports the activities needed to extend the Architectural Runway to provide future business functionality. These include exploration, architecture, infrastructure, and compliance. SP-1175 Demonstrate Signal Displays using in-memory visibility streaming

          • Not Assigned - Priority is not assigned yet.
          • Discarded
          split to

          Architectural Decision - Software Architectural Decision ADR-21 Visibility memory data model

          • Analyzing
          mentioned in

          Page Loading...

          Page Loading...

          Structure

            Activity

              People

                p.wortmann Wortmann, Peter
                b.mort Mort, Ben
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Feature Progress

                  Story Point Burn-up: (0%)

                  Feature Estimate: 1.0

                  IssuesStory Points
                  To Do00.0
                  In Progress   00.0
                  Complete00.0
                  Total00.0

                  Dates

                    Created:
                    Updated:
                    Resolved:

                    Structure Helper Panel