Product Knowledge Base

Space shortcuts

Page tree

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

Propositions for next features

Large File Handling

Problem

  • Concerns the transfer of large files (> 1GB) 
  • Performance is too slow

Proposed Solution

  • Currently one file is being processed in one channel.
  • Using multiple channels would increase the overall speed of transfer
    • Use of multiple TCP streams to transfer data in parallel
  • See also

Resume File Transfer

Problem

  • If an error occurs during transfer of a file then this file has to re-transferred completely

Proposed Solution

  • Automatically detect and resume the transfer after interruptions (e.g. connection is broken)
    • Detect 
      • JADE knows when a transfer was interrupted and would resume operation with the same parameters
      • Detection is limited for a configurable duration after interrupted file transfers
    • Resume
      • JADE knows the chunks that have been transferred and is capable to continue writing chunks
    • Fail-safe operation
      • JADE detects problems of modified files and inconsistencies and can fallback to re-transfer a file 

Priority of files to be transferred

Problem

  • Individual files of a file group should be prioritized within one transfer session.

Solution

  • This feature will be based on the use of a list of files as input parameter for file transfer. 
    • By pre-processing (external command or internal rules) this list would be sorted before transfer.
    • Multiple file lists could be specified which would result in the fact that the files of each list would be completely transferred before processing the next list.
    • Use of channels as specified in the Large File Handling features should be focussed on prioritized files.
  • Workaround: JADE supports to use a list of files as input parameter. Transfers are then performed in the sequence of that list. During pre-processing that list could be created and sorted by priority.

JMS Interface   

Problem

  • JADE does not dispose of a JMS interface 

Proposed Solution

  • Implementation with Apache Message Queue
  • Specification required

NFS Interface

Problem 

  • NFS is not available for JADE

Proposed Solution

HTTP, HTTPS as Data Provider    

Problem

  • This feature is implemented with limits. Missing functionalities are:
    • proxy support
    • support for directory listings
  • This feature is broken in the current JADE release 
    • Due to the recently introduced mutli-threading this feature is does not work

Proposed Solution

  • Implementation based on Apache Commons VFS
    • support for directory listing, authentication
    • no support for write operations        

Integration with LDAP

Problem

  • No authentication or authorization is available for JADE 

Proposed Solution

  • JADE Background Service History GUI supports LDAP authentication.
    • This component will be available with release 1.8
  • JOC should support LDAP
    • This could be part of a major re-write of JOC that will take some months
    • tbd: das problem ist nicht so schwerwiegend, da  hier keine Enduser im Spiel sind 

Publish-Subscribe Model

Problem

Solution

Centralized Configuration and Management    

JADE GUI for    

Routing rules

Monitoring Interfaces

Problem

  • JADE supports notifications to System Monitors, e.g. Nagios, op5, Zabbix, if used as a JITL Job in JobScheduler
  • Additional System Monitors can easily be integrated should a command line interface be available.
  • Other monitoring solutions with more sophisticated interfaces should be supported.

Proposed Solution

  • Start from the existing JobScheduler Monitoring Interface
    • add monitoring capabilities for the file transfer history as reported by the JADE Background Service
  • Implement interfaces for popular Monitoring Systems
    • IBM Tivoli Monitoring: should this be feasible by a client command line tool then
    • HP OVO: tbd
    • IBM WS Message Broker (MQ)
  • SNMP Interface: tbd 
    • architecture

Further Propositions

Transfer speed restrictions

Problem

  • JADE uses all the available bandwith for a transfer and leaves it to the network administration to manage rules for bandwidth usage
  • JADE should be configurable for use of bandwidth, e.g. in slow networks

Proposed Solution

  • tbd

 

 

 

  • No labels