EUROVO FC

AVO
SWG
Management
Science
Standards
Technology
Reports&Minutes
Forum
Events
Contacts
Help

EGG

Main

Know

Test

TWiki

Ideas on integrating CEA Workflows into AVO Portal Content Introduction & Rationale Timeline Setting the Scope Use Cases ... Asychronous activity proposal Aladin and MySpace

Introduction & Rationale

Astrogrid is developing a framework called the Common Execution Architecture (CEA). The purpose is to wrap unix command line applications and to drive them. Input and output data can be stored on shared MySpace nodes (Astrogrid:Astrogrid/MySpace). Workflows defining the execution sequence of several web services can be created and stored. Now the question is how and to which extend we can and should access this functionality through the AVO portal. This page captures related ideas, expected requirements and possible solutions.

Timeline

The Astrogrid framework will become available at the end of iteration 6, 30-Aug-2004. Some pieces are already available in the iteration 5 release (May 2004) such as MySpace support and an HTML based web interface for defining workflows.

AVO is targetting the 3rd year demo sometime Jan. 2005 and the selection of related science cases should happen in June 2004. ESO/ECF is planning to experiment with the Astrogrid framework in June.

After the selection and rough definition of science cases and an initial evaluation of the framework a decision on how to proceed is due in July. Obviously, the more CEA and MySpace nodes AVO and Astrogrid partners deploy the more opportunities there will be to build useful workflows.

Setting the Scope

An initial step is to set the scope of the interaction between CEA/MySpace and the AVO prototype portal.

There's a wide range of possible features that could but not necessarily need to be supported. In the simplest case it is a matter of loading existing workflows as configuration files and to trigger their execution. The other extreme is an integrated interface for defining workflows, some sort of asynchronous messaging system for user notification/monitoring and a MySpace browser. Alternatively, one can imagine an approach were some functionality like monitoring is done through an Astrogrid portal, whereas the AVO portal provides help to the astronomer to chose the correct parameters for the workflow.

Use Cases

• the 2nd year AVO demo (Sextractor, HyperZ, X-match)
• SWarp (Astrogrid test case)
• SED classifier service (with GAVO)
• ...

...

TBC

Asychronous activity proposal

There is also a proposal for creating and managing asynchronous activities on web services. This overlaps somewhat with CEA (CEA does aysynchronicity). It may be that the asynchronicity proposal influences the workflow interfaces and possibly causes some evolution of CEA.

Aladin and MySpace

Some way of using MySpace is needed, even if all workflows are pre-generated. Basically, there needs to be some way to consign data selections in Aladin to MySpace so that they are available to workflows and CEA. I can see four ways of doing this.

1. Aladin could talk directly to MySpace. It doesn't necessarily need the MySpace browser, but it needs to be able to allocate a logical file name (LFN) and to deliver the data there. Since MySpace is intended to be private, this means that Aladin has to work the access-control arrangements, which in turn means that it has to work with the AstroGrid community service, and that users have to log in to an account.
2. Aladin could push data to MySpace using GridFTP^?. Using the AstroGrid interface to MySpace in a web browser, the user would allocate the LFN in MySpace and have MySpace resolve it to a GridFTP^? URL, the latter being displayed on the screen. User would then cut and paste the URL into Aladin and Aladin would push the data.
3. MySpace could pull the data from Aladin. Aladin would display the URL for the data source (presumed public and uncontrolled). User would cut and past the URL into the AstroGrid UI. Myspace would pull the data.
4. Aladin could push the data using HTTP. User would allocate the LFN using the AstroGrid UI and have MySpace resolve it to a HTTP URL that includes a one-shot password generated by MySpace for this one push. User would cut and paste this into Aladin. Aladin would push the data.

Option 1 gives a better UI if we have time for it, but it's quite involved. Option 3 is the cheap one. Options 2 and 4 are better than option 3 if the data source cannot be expressed as a simple, HHTP URL; they allow Aladin to finesse the archive work. Options 2, 3 and 4 are equivalent from the user's point of view: they have the same number of control operations. Option 2 has higher performance than option 4 (GridFTP^? is more efficient and tuneable) but is harder to set up in Aladin.

Overall, I favour option 4. AstroGrid is already providing similar code for reading files from MySpace to a browser and this code is due in July. It would not be too hard to extend the code to cover a secured HTTP push. However, this would have to be approved as an AstroGrid feature.