SODA accessing data over HTTPS

Currently the SODA workflow that is implemented in the Data Management (DM) API looks something like:

1. Client executes /data/soda/.. route via DM API with requested data identifier and SODA service parameterised
2. DM API does prerequisite permissions checking based on bearer token presented
3. DM API contacts another API (site-capabilities API) to get the base storage path in order to formulate the physical path to the file on storage for the requested data identifier
4. If permissions check is passed, DM API issues redirect to client to SODA service with the local filesystem path to the requested data identifier as a query parameter
5. SODA service acts on local storage using this physical file path and responds to client

This works, but entirely circumvents the authorisation provided by the storage resource manager (SRM) that is controlling the storage. One consequence of this is that in production the SODA service would have to be firewalled off from "public" access as the authorisation occurs before the SODA service is called.

There is an alternative to the above which may be simpler. Permissions checking (that is, mapping the group membership provided by the token to a policy defining whether a user can do something at a particular API) is done at the Data Management API layer by a call to the Permissions API. This provides a singular source of truth for permissions checking which is preferable as we cannot ensure that all downstream services will implement calls to the Permissions API in a consistent manner.

The workflow would therefore be changed such that, at the Data Management API layer, the user is passed an exchanged token including the necessary (WLCG JWT profile) scope storage.read:/path/on/storage to allow read access to /path/on/storage; {}the specific authorisation at the SODA layer that this token is passed to is then actually quite simple and not dependent on checking anything against the Permissions API. The only things that the SODA service (or any other subsequent service) would need to do before processing is to validate the token and check that the location of the file that is being read matches the path (after the colon) in this scope. This is essentially implementing WLCG path based authZ in the service itself.