Scope
- YADE should support file transfer operations with Azure Blob Storage.
- Technically speaking Azure file transfer operations make use of the HTTPS protocol and a number of query parameters and headers. The implementation therefore is fairly straightforward.
- The sticking point is about what authentication methods should be supported and how they would be operated:
- If blob containers are made publicly available and require no authentication then the YADE supports this out-of-the-box.
- If Azure authentication methods are applied then different implementation strategies are to be considered. Azure supports a range of authentication methods - we picked two of them for this proposal:
- Use of Shared Key authentication, see Authorize with Shared Key
- Use of Shared Access Signature authentication (SAS), see Create an Account SAS
Authentication Methods
Find a preliminary comparison of authentication methods:
Capability | Shared Key | Shared Access Signature |
---|---|---|
Scope |
|
|
Access Duration |
|
|
Permissions |
|
|
- From the above comparison Shared Access Signatures are superior concerning more fine-grained access to resources.
- At the same time Shared Key authentication suggests ease of use as a single Shared Key can be used to authenticate any file transfer operations for an unlimited duration.
- Technically a single Shared Access Signature can be applied in a similar way allowing any file transfer operation on any blobs in a given container for a longer period, e.g. the next 100 years.
Operation
- Considering use of authentication methods with file transfer jobs there is a clear statement from SOS: implementation of any authentication method is out of scope of the JobScheduler product.
- Implementation of authentication methods is within the responsibility of the user. Reasons for this include that
- users should not trust any 3rd party implementation of authentication methods,
- a number of authentication methods are offered by Azure that allow an individual choice according to security requirements of an organization,
- there is ongoing development with Azure about improvement of authentication methods.
- Implementation of file transfer operations for Azure blob storage is within the scope of the YADE.
- Implementation of authentication methods is within the responsibility of the user. Reasons for this include that
- Technically this offers the following options:
- An organization can create a single Shared Key or Shared Access Signature that represents a constant value that is made available to all YADE file transfer jobs.
- An organization can create individual Shared Keys or Shared Access Signatures that are used with groups of YADE file transfer jobs or with individual jobs.
Implementation
- Find samples for use of authentication methods and respective file transfer operations from the following articles:
- When using a single Shared Key or Shared Access Signature for all YADE file transfer jobs then
- this could be added as a constant value to the JobScheduler configuration, e.g. with a parameter to the
./config/scheduler.xml
file. - this could be added to a Credential Store that would be accessed by any YADE file transfer jobs.
- this could be added as a constant value to the JobScheduler configuration, e.g. with a parameter to the
- When using individual Shared Keys or Shared Access Signatures per YADE file transfer job then a Monitor should be implemented that is assigned on a per job basis and that can be parameterized to use the key or signature specified by a job parameter or order parameter.
References
- YADE-561Getting issue details... STATUS
Resources
Pages
Navigation