...

• Considering use of authentication methods with file transfer jobs there is a clear statement from SOS: implementation of any authentication method is out of scope of the JobScheduler product. 
  • Implementation of authentication methods is within the responsibility of the user. Reasons for this include that
    • users should not trust any 3rd party implementation of authentication methods,
    • a number of authentication methods are offered by Azure that allow an individual choice according to security requirements of an organization,
    • there is ongoing development with Azure about improvement of authentication methods.
  • Implementation of file transfer operations for Azure blob storage is within the scope of the YADE.
• Technically this offers the following options:
  • An organization can create a single Shared Key or Shared Access Signature that represents a constant value that is made available to all YADE file transfer jobs.
  • An organization can create individual Shared Keys or Shared Access Signatures that are used with groups of YADE file transfer jobs or with individual jobs.

Implementation

User Tasks: Manage Shared Keys and Shared Access Signatures

Samples for creating Shared Keys and Shared Access Signatures

• Find examples for creating and using Azure Find samples for use of authentication methods and respective file transfer operations from the following articles:
  • Feature Proposal - Azure Blob Storage - Shared Key Authentication
  • Feature Proposal - Azure Blob Storage - Shared Access Signatures
• The samples are far from being perfect, however, they might be useful as a basis for a user's implementation.

SOS Tasks: YADE Integration

Implementation Strategies

• When using a single Shared Key or Shared Access Signature for all YADE file transfer jobs then
  • this could be added as a constant value to the JobScheduler configuration, e.g. with a parameter to the ./config/scheduler.xml file.
  • this could be added to a Credential Store that would be accessed by any YADE file transfer jobs.
• When using individual Shared Keys or Shared Access Signatures per YADE file transfer job then a Monitor should be implemented that is assigned on a per job basis and that can be parameterized to use the key or signature specified by a job parameter or order parameter.
• Should Shared Keys or Shared Access Signatures have to be renewed on a regular basis then this can be executed by a job, e.g. running daily, that is implemented by a user of JobScheduler.

YADE Changes

• Accept for both Shared Keys and Shared Access Signatures the respective configuration items:
  • Both authentication methods
    • Add HTTPS headers:
      • x-ms-blob-type : BlockBlob
      • x-ms-date : <date of request>
      • x-ms-version : <version as specified by user>
  • Shared Key
    • Add HTTPS authorization header:
      • Authorization : SharedKey <requester account specified by user>:<signature specified by user>`
  • Shared Access Signatures
    • Add SAS token to URL:
      • https://<blob storage owner account specified by user>.blob.core.windows.net/<container specified by user>/<blob specified by user>?<SAS token specified by user>
• Delimitation
  • The implementation is about integrating the above configuration items into HTTPS headers and query parameters.
  • The implementation is not about developing a new data provider that is subject to a separate feature request.

References

...