|
|
|
1 Introduction
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). Nevertheless such a report is still usefull, because the collection of information from different more or less independant technological areas is in fact added value compared to the single pieces. This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
|
|
2.1 Server-centric and storage-centric IT-architectures
2.2 Storage networks
|
|
<
< |
Storage networks, as the name suggests are networks dedicated to storage. During the last couple of years a number of technologies have been established to implement the idea of a network of storage components available to computers connected to this network. Two acronyms are dominant in this area: NAS and SAN, Network Attached Storage and Storage Area Network, respectively. The first makes use of existing network connections (usually TCP/IP) and file sharing protocols like NFS, the latter builds on specialized network hardware. Especially for high throughput or transactional systems the performance requirements are pretty high and thus standard NFS type sharing mechanisms are not adequate. For NFS this is due to the limitations of the underlying TCP/IP based network and the NFS protocol itself. The advances in network speed (Gigabit ethernet) still make TCP/IP based storage networks and thus NAS solutions attractive at least for certain types of applications. In the high-end domain of storage networks the SAN concepts and mainly the fiber-channel technology is clearly dominant, but it is both high-priced and requires additional infrastructure in specialized cabeling and switching. More recently new technologies like iSCSI and * are gaining grounds and with infiniBand a whole new concept is taking shape which combines all network requirements on a single high-speed low latency network.
|
>
> |
Storage networks, as the name suggests are networks dedicated to storage. During the last couple of years a number of technologies have been established to implement the idea of a network of storage components available to computers connected to this network. Two acronyms are dominant in this area: NAS and SAN, Network Attached Storage and Storage Area Network, respectively. The first makes use of existing network connections (usually TCP/IP) and file sharing protocols like NFS, the latter builds on specialized network hardware. Especially for high throughput or transactional systems the performance requirements are pretty high and thus standard NFS type sharing mechanisms are not adequate. For NFS this is due to the limitations of the underlying TCP/IP based network and the NFS protocol itself. The advances in network speed (Gigabit ethernet) still make TCP/IP based storage networks and thus NAS solutions attractive at least for certain types of applications. In the high-end domain of storage networks the SAN concepts and mainly the fiber-channel technology is clearly dominant, but it is both high-priced and requires additional infrastructure in specialized cabeling and switching. More recently new technologies like iSCSI and * are gaining grounds and with infiniBand a whole new concept is taking shape which combines all network requirements on a single high-speed, low latency network.
|
|
2.2.1 Technologies for storage networks
2.2.2 I/O technologies for storage networks
|
|
<
< |
2.3 File systems
While storage networks are based on the idea to extend the availability of block oriented storage devices through networks, there is also the other concept of extending the file systems to cover more than one computer. Such 'global' file systems usually require direct support from the underlying OS to be fast and transparent to the applications which just want to access files.
|
>
> |
2.3 File systems and file services
While storage networks are based on the idea to extend the availability of block oriented storage devices through networks, there is also the other concept of extending the file systems to cover more than one computer. Such 'global' file systems usually require direct support by the underlying OS to be fast and transparent to the applications which just want to access files. Such file systems do not change anything about the (successful) concept of files and folders. However folders in particular, but also file names are usually context dependant and the folder implied structuring of files might be confusing if not wrong if seen in another context. This problem can be avoided by using virtual folders and virtual file names (virtual file system) which are mapped (linked) with the actual files. The mapping has to be part of the definition of the virtual file system, but the technology used to access the actual file can then be left over to the physical file system which might even be a global file system. Since the physical file system in this case had to care only about the physical placement of files and not about the context dependant logical placement it can much better be optimized
|
|
3 Projects
The Next Generation Archive System (NGAS) has been developed at ESO starting in February 2001. NGAS has evolved quite a bit from the first prototype to the operational system since then. It is installed and used operationally for various instruments including ESO's Wide Field Imager ( WFI@2.2 ), all VLTI instruments and HARPS. In the future it will also hold data from the UKIRT/WFCAM, VST/Omegacam and VISTA. ALMA has also adopted NGAS for the storage of bulk data. NGAS is an technological and operational concept for archiving and maintenance of massive amounts of data. In addition it allows processing of substantial parts or the whole data holding to be carried out in the archive while just moving the results to the requester. Although the main development started before the AVO project especially the processing capabilities have been developed to allow VO science projects to be carried out on wide field imaging data which is of particular interest in this area. A substantial part of the development went into the definition and implementation of operational concepts for large distributed astronomical archives.
|
|
1 Introduction
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). Nevertheless such a report is still usefull, because the collection of information from different more or less independant technological areas is in fact added value compared to the single pieces. This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
|
|
2.1 Server-centric and storage-centric IT-architectures
2.2 Storage networks
|
|
>
> |
Storage networks, as the name suggests are networks dedicated to storage. During the last couple of years a number of technologies have been established to implement the idea of a network of storage components available to computers connected to this network. Two acronyms are dominant in this area: NAS and SAN, Network Attached Storage and Storage Area Network, respectively. The first makes use of existing network connections (usually TCP/IP) and file sharing protocols like NFS, the latter builds on specialized network hardware. Especially for high throughput or transactional systems the performance requirements are pretty high and thus standard NFS type sharing mechanisms are not adequate. For NFS this is due to the limitations of the underlying TCP/IP based network and the NFS protocol itself. The advances in network speed (Gigabit ethernet) still make TCP/IP based storage networks and thus NAS solutions attractive at least for certain types of applications. In the high-end domain of storage networks the SAN concepts and mainly the fiber-channel technology is clearly dominant, but it is both high-priced and requires additional infrastructure in specialized cabeling and switching. More recently new technologies like iSCSI and * are gaining grounds and with infiniBand a whole new concept is taking shape which combines all network requirements on a single high-speed low latency network.
|
|
2.2.1 Technologies for storage networks
2.2.2 I/O technologies for storage networks
2.3 File systems
|
|
>
> |
While storage networks are based on the idea to extend the availability of block oriented storage devices through networks, there is also the other concept of extending the file systems to cover more than one computer. Such 'global' file systems usually require direct support from the underlying OS to be fast and transparent to the applications which just want to access files.
|
|
3 Projects
The Next Generation Archive System (NGAS) has been developed at ESO starting in February 2001. NGAS has evolved quite a bit from the first prototype to the operational system since then. It is installed and used operationally for various instruments including ESO's Wide Field Imager ( WFI@2.2 ), all VLTI instruments and HARPS. In the future it will also hold data from the UKIRT/WFCAM, VST/Omegacam and VISTA. ALMA has also adopted NGAS for the storage of bulk data. NGAS is an technological and operational concept for archiving and maintenance of massive amounts of data. In addition it allows processing of substantial parts or the whole data holding to be carried out in the archive while just moving the results to the requester. Although the main development started before the AVO project especially the processing capabilities have been developed to allow VO science projects to be carried out on wide field imaging data which is of particular interest in this area. A substantial part of the development went into the definition and implementation of operational concepts for large distributed astronomical archives.
Comparing NGAS to other technologies is not easy since it contains features of a number of existing technologies like NAS, global file systems, mirror RAID and file servers. The main features are listed below:
|
|
>
> |
- All operations are based on unique file IDs, but it is not the core NGAS system which takes care of assigning these file IDs, but rather the archiving plug-in, this makes it rather easy to preserve domain specific naming conventions without cluttering the NGAS core code.
|
|
- Transparent file archiving and retrieval through HTTP.
|
|
>
> |
- Support for file versioning: Any kind of file ID is just a name and does not make sure that two files carrying the same ID are really the same. For some applications it might even be desireable to use the same ID for two different versions of the same file. Thus NGAS by default does not overwrite any file, but rather creates a new version with the same ID.
|
|
- Automatic creation and maintenance of at least one secondary copy of every file.
- Automatic consistency checking of all files using a configurable mechanism.
- The core NGAS does not know about specific file types or special handling of such files. Specific handling is controlled by mime-types defined as part of the http header during archiving. A mime-type can trigger a plug-in which takes over the special treatment required for that type.
- Cloning of media or files.
- Mirroring: A NGAS host 'B' can be configured to subscribe to another host 'A', host 'A' will then push every newly arrived file to host 'B' using a standard HTTP-Post. The two hosts may belong to two completely independant archives, in fact the client (host 'B' in this example) could even be any kind of software which is able to deal with a HTTP-Post message.
- NGAS can execute processing plug-ins upon request of files, i.e. before a file is delivered it is processed and the result of the processing is send to the client. The available processing plug-ins can be configured in the server and the client can then select one of those.
|
|
<
< |
- Transparent media handling: If a disk is moved from one host to another or a new disk is inserted a number of activities are carried out in order to make sure that
|
>
> |
- Transparent media handling: If a disk is moved from one host to another or a new disk is inserted, a number of activities are carried out in order to make sure that the contents of that disk is consistent and aligned with the information contained in the NGAS database.
|
|
|
|
<
< |
Although the NGAS software (called NGAMS) works on a variety of platforms and is very flexible, NGAS as such is a full system and at least half of the development went into the integration of the software with its underlying hardware and OS. There are some features which a mainly related to the scalability of NGAS, which is based on the concept of adding new hosts along with capacity and throughput requirements. In effect this means that an NGAS archive consists of a full cluster of hosts which know about each other. Due to the usage of HTTP as a communication protocol these hosts do not have to be co-located but may be placed even on different continents. Features which deal with this clustering and scalability requirements include:
|
>
> |
Although the NGAS software (called NGAMS) works on a variety of platforms and is very flexible, the operational NGAS is a full system and at least half of the development went into the integration of the software with its underlying hardware and OS. There are some features which are mainly related to the scalability of NGAS, which is based on the concept of adding new hosts along with capacity and throughput requirements. In effect this means that an NGAS archive consists of a full cluster of hosts which know about each other. Due to the usage of HTTP as a communication protocol these hosts do not have to be co-located but may be placed even on different continents. Features which deal with this clustering and scalability requirements include:
|
|
- Transparent retrieval of files: HTTP retrieval requests can be send to any publicly available host in the archive, the requested file will be returned from the host actually holding the file either by proxy or by redirection. Since at least two copies should be available of any single file in the archive the system will even try to figure out which of the copies is physically closer to the requester.
- Automatic power-safe mode: If there is no activity on a NGAS host it may be shut-down in order to safe power. If this is done the host can request to be woken up by another (master) host after a certain time-span in order to carry out the consistency checking. If a file is requested through any master in the archive which is located on a 'sleeping' host, this host will be woken up to serve the file. The mechanism used to do this is again a configurable plug-in. The currently available and used method is wake-on-lan.
|
|
<
< |
- In addition there are a number of features which deal with disk space monitoring, e-mail notification and logging
|
>
> |
- In addition there are a number of features which deal with disk space monitoring, e-mail notification and logging.
|
|
3.2 WFCAM/VISTA Science Archive
The WFCAM/VISTA archive development is been decribed in detail in several documents where the most relevant in the current context is the WSA Hardware/OS/DBMS design Document.
|
|
1 Introduction
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). Nevertheless such a report is still usefull, because the collection of information from different more or less independant technological areas is in fact added value compared to the single pieces. This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
|
|
2.3 File systems
3 Projects
|
|
<
< |
The Next Generation Archive System (NGAS) has been developed at ESO starting in February 2001. NGAS has evolved quite a bit from the first prototype to the operational system since then. It is installed and used operationally for various instruments including ESO's Wide Field Imager ( WFI@2.2 ), all VLTI instruments and HARPS. In the future it will also hold data from the UKIRT/WFCAM, VST/Omegacam and VISTA. ALMA has also adopted NGAS for the storage of bulk data. NGAS is an technological and operational concept for archiving and maintenance of massive amounts of data. In addition it allows processing of substantial parts or the whole data holding to be carried out in the archive with just moving the results to the requester. Although the main development started before the AVO project especially the processing capabilities have been developed to allow VO science projects to be carried out on wide field imaging data which is of particular interest in this area. A substantial part of the development went into the definition and implementation of operational concepts for large distributed astronomical archives.
|
>
> |
The Next Generation Archive System (NGAS) has been developed at ESO starting in February 2001. NGAS has evolved quite a bit from the first prototype to the operational system since then. It is installed and used operationally for various instruments including ESO's Wide Field Imager ( WFI@2.2 ), all VLTI instruments and HARPS. In the future it will also hold data from the UKIRT/WFCAM, VST/Omegacam and VISTA. ALMA has also adopted NGAS for the storage of bulk data. NGAS is an technological and operational concept for archiving and maintenance of massive amounts of data. In addition it allows processing of substantial parts or the whole data holding to be carried out in the archive while just moving the results to the requester. Although the main development started before the AVO project especially the processing capabilities have been developed to allow VO science projects to be carried out on wide field imaging data which is of particular interest in this area. A substantial part of the development went into the definition and implementation of operational concepts for large distributed astronomical archives.
Comparing NGAS to other technologies is not easy since it contains features of a number of existing technologies like NAS, global file systems, mirror RAID and file servers. The main features are listed below:
- Transparent file archiving and retrieval through HTTP.
- Automatic creation and maintenance of at least one secondary copy of every file.
- Automatic consistency checking of all files using a configurable mechanism.
- The core NGAS does not know about specific file types or special handling of such files. Specific handling is controlled by mime-types defined as part of the http header during archiving. A mime-type can trigger a plug-in which takes over the special treatment required for that type.
- Cloning of media or files.
- Mirroring: A NGAS host 'B' can be configured to subscribe to another host 'A', host 'A' will then push every newly arrived file to host 'B' using a standard HTTP-Post. The two hosts may belong to two completely independant archives, in fact the client (host 'B' in this example) could even be any kind of software which is able to deal with a HTTP-Post message.
- NGAS can execute processing plug-ins upon request of files, i.e. before a file is delivered it is processed and the result of the processing is send to the client. The available processing plug-ins can be configured in the server and the client can then select one of those.
- Transparent media handling: If a disk is moved from one host to another or a new disk is inserted a number of activities are carried out in order to make sure that
Although the NGAS software (called NGAMS) works on a variety of platforms and is very flexible, NGAS as such is a full system and at least half of the development went into the integration of the software with its underlying hardware and OS. There are some features which a mainly related to the scalability of NGAS, which is based on the concept of adding new hosts along with capacity and throughput requirements. In effect this means that an NGAS archive consists of a full cluster of hosts which know about each other. Due to the usage of HTTP as a communication protocol these hosts do not have to be co-located but may be placed even on different continents. Features which deal with this clustering and scalability requirements include:
- Transparent retrieval of files: HTTP retrieval requests can be send to any publicly available host in the archive, the requested file will be returned from the host actually holding the file either by proxy or by redirection. Since at least two copies should be available of any single file in the archive the system will even try to figure out which of the copies is physically closer to the requester.
- Automatic power-safe mode: If there is no activity on a NGAS host it may be shut-down in order to safe power. If this is done the host can request to be woken up by another (master) host after a certain time-span in order to carry out the consistency checking. If a file is requested through any master in the archive which is located on a 'sleeping' host, this host will be woken up to serve the file. The mechanism used to do this is again a configurable plug-in. The currently available and used method is wake-on-lan.
- In addition there are a number of features which deal with disk space monitoring, e-mail notification and logging
|
|
3.2 WFCAM/VISTA Science Archive
The WFCAM/VISTA archive development is been decribed in detail in several documents where the most relevant in the current context is the WSA Hardware/OS/DBMS design Document.
3.3 Cobra
|
|
1 Introduction
|
|
<
< |
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
|
>
> |
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). Nevertheless such a report is still usefull, because the collection of information from different more or less independant technological areas is in fact added value compared to the single pieces. This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
|
|
- Bulk data storage and long term archiving.
- Bulk data processing.
Although closely related the two create their own set of specific requirements which are referred to in the GRID world by the two acronyms DataGrid and ComputeGrid. Most processing in astronomy is using input data from some experiment or detector, the only exception are simulations which may produce enormous data volumes as well. For the 'normal' detector data processing the DataGrid and ComputeGrid can not be separated without the need of moving large volumes of data from the storage to the compute nodes and possibly back.
|
|
>
> |
A slightly different yet related problem comes with the growing usage of data base management systems (DBMS) for the archiving and analysis of astronomical data in particular astronomical catalogues. Such systems pose additional requirements on the two main points given above. Specific DBMS issues are covered by the report of WP3.3.
This report consists of two main sections:
- Section 2 gives an overview of available and some emerging technologies as far as they seemed to be relevant in the given context. Technologies which are currently just in the research state have been left out, since experience shows that even if a technology seems to be superior, the commercial market might still adopt another one. This report does give an overview of all relevant commercial systems regardless of the price, but it should be mentioned here that given the amount of data the astronomical community is dealing with, the economical factor tends to have a very high weight in the technological decision process.
- Section 3 gives an overview of existing projects in the astronomical community.
|
|
2 Technology
2.1 Server-centric and storage-centric IT-architectures
|
|
1 Introduction
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
|
|
2.3 File systems
3 Projects
|
|
<
< |
The Next Generation Archive System (NGAS) has been developed by ESO starting in February 2001.
NGAS has evolved quite a bit from the first prototype to the operational system since then. It is installed and used operationally for various instruments including ESO's Wide Field Imager (WFI@2.2), VLTI and HARPS.
|
>
> |
The Next Generation Archive System (NGAS) has been developed at ESO starting in February 2001. NGAS has evolved quite a bit from the first prototype to the operational system since then. It is installed and used operationally for various instruments including ESO's Wide Field Imager ( WFI@2.2 ), all VLTI instruments and HARPS. In the future it will also hold data from the UKIRT/WFCAM, VST/Omegacam and VISTA. ALMA has also adopted NGAS for the storage of bulk data. NGAS is an technological and operational concept for archiving and maintenance of massive amounts of data. In addition it allows processing of substantial parts or the whole data holding to be carried out in the archive with just moving the results to the requester. Although the main development started before the AVO project especially the processing capabilities have been developed to allow VO science projects to be carried out on wide field imaging data which is of particular interest in this area. A substantial part of the development went into the definition and implementation of operational concepts for large distributed astronomical archives.
|
|
3.2 WFCAM/VISTA Science Archive
The WFCAM/VISTA archive development is been decribed in detail in several documents where the most relevant in the current context is the WSA Hardware/OS/DBMS design Document.
3.3 Cobra
|
|
1 Introduction
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
|
|
2.2.2 I/O technologies for storage networks
2.3 File systems
3 Projects
|
|
<
< |
3.1 NGAS
3.2 SkyServer
3.3 WFCAM/VISTA Science Archive
3.4 Cobra
|
>
> |
The Next Generation Archive System (NGAS) has been developed by ESO starting in February 2001.
NGAS has evolved quite a bit from the first prototype to the operational system since then. It is installed and used operationally for various instruments including ESO's Wide Field Imager (WFI@2.2), VLTI and HARPS.
3.2 WFCAM/VISTA Science Archive
The WFCAM/VISTA archive development is been decribed in detail in several documents where the most relevant in the current context is the WSA Hardware/OS/DBMS design Document.
3.3 Cobra
3.4 SDSS/SkyServer
A system external to the AVO initiative which makes have use of data base and advanced storage technology is the Sloan Digital Sky Survey (SDSS). The most relevant document here is the Data Mining the SDSS SkyServer Database, Microsoft technical report MSR-TR-2002-01 (January 2002).
|
|
4 Conclusions
5 Future work
|
|
1 Introduction
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
- Bulk data storage and long term archiving.
- Bulk data processing.
|
|
<
< |
Although closely related the two create their own set of specific requirements which are referred to in the GRID world by the two acronyms DataGrid and ComputeGrid. Most processing in astronomy is using input data from some experiment or detector, the only exception are simulations which may produce enormous data volumes as well. For the 'normal' detector data processing the DataGrid and ComputeGrid can not be
|
>
> |
Although closely related the two create their own set of specific requirements which are referred to in the GRID world by the two acronyms DataGrid and ComputeGrid. Most processing in astronomy is using input data from some experiment or detector, the only exception are simulations which may produce enormous data volumes as well. For the 'normal' detector data processing the DataGrid and ComputeGrid can not be separated without the need of moving large volumes of data from the storage to the compute nodes and possibly back.
|
|
2 Technology
2.1 Server-centric and storage-centric IT-architectures
|
|
>
> |
1 Introduction
The Storage and Compute Technology WorkPackage (WP3.2) was created in order to produce guidelines and recommondations for hardware which should support storage and computing for VO applications. Given the extremely fast evolution of computer hardware such guidelines once written are already out-of-date or groups which need to procure hardware at any given time during the project have their own special requirements (additional hadrware for a certain experiment) or boundary conditions (IT department only supports certain platforms/vendors). This report thus gives an overview of available technologies and their application in certain existing astronomical projects which have similar requirements. Astronomical applications pose two main problems on the IT infrastructure:
- Bulk data storage and long term archiving.
- Bulk data processing.
Although closely related the two create their own set of specific requirements which are referred to in the GRID world by the two acronyms DataGrid and ComputeGrid. Most processing in astronomy is using input data from some experiment or detector, the only exception are simulations which may produce enormous data volumes as well. For the 'normal' detector data processing the DataGrid and ComputeGrid can not be
2 Technology
2.1 Server-centric and storage-centric IT-architectures
2.2 Storage networks
2.2.1 Technologies for storage networks
2.2.2 I/O technologies for storage networks
2.3 File systems
3 Projects
3.1 NGAS
3.2 SkyServer
3.3 WFCAM/VISTA Science Archive
3.4 Cobra
4 Conclusions
5 Future work
<!--
-->
|
|
|
