Details
-
Feature
-
Must have
-
SRCnet
-
-
-
2
-
2
-
0
-
SP-4565 v0.1 Test Campaign preparation: Ingest image data into Rucio
-
Team_SRCARCH, Team_SRCPT
-
-
-
-
PI24-PB SRC23-PB SRCNet0.1 data-lifecycle tests-compilation
-
SPO-3478
Description
These will be one category of tests driving the test Campaigns for SRCNetv0.1
These will be end-to-end data lifecycle tests:
- Starting from data ingestion
- Data replication to multiple sites
- Data processing e.g. via Jupyter notebooks (or data visualisation)
- Uploading of new data products
Data lifecycles to be identified that focus on capturing the operational interaction of multiple components like ingestion, DDM and compute across v0.1 sites (as opposed to prioritised user science workflows)
These data lifecycle tests should be documented (e.g. confluence) and software elements should be provided in a portable format (mainly notebooks with the proper documented environment or containers if this implies the use of a certain batch processing library or similar)
Result expected:
Documented set of ‘live’ tests conducted during the identified high-availability window describing the data lifecycle being demonstrated, sites participating, data used (type/volume), workflow chosen, and description of any new data ingested. The Operations team will likely be consolidating this information in preparation for the test campaigns. Where applicable demos of the tests conducted, and dashboard views showing data ingestion, and replication.
Attachments
Issue Links
- Child Of
-
SP-4870 Science Enabling v0.1 - Roadmap
- Implementing
- is informed by
-
SP-4565 v0.1 Test Campaign preparation: Ingest image data into Rucio
- Implementing
- relates to
-
SP-4251 Enable use of WLCG DC inject tool for v0.1 data moving campaign scenarios
- Implementing
-
SP-4257 Develop the tests and monitoring for v0.1 Data Challenge
- Implementing
-
SP-4229 Establish a simulated-test-data generation pipeline for v0.1 (using Karabo)
- Done