Q&A about the Virtual Observatory
Q: What is a Virtual Observatory (VO)?
A: A virtual observatory (VO) is a collection of interoperating data archives and software
tools that utilise the Internet to form a scientific research environment in which astronomical
research programs can be conducted. In much the same way as a real observatory consists of
telescopes, each with a collection of unique astronomical instruments, the VO consists of a
collection of data centres each with unique collections of astronomical data, software systems
and processing capabilities.
The need for the development of a VO is driven by two key factors. Firstly, there is an explosion
in the size of astronomical data sets delivered by new large facilities. The processing and storage
capabilities necessary for astronomers to analyse and explore the forthcoming data sets will greatly
exceed the capabilities of the types of desktop systems astronomers currently have available to them.
Secondly, a potential scientific gold mine remains unexplored and underexploited because large data
sets in astronomy are unconnected. If large surveys and catalogues could be joined into a uniform and
interoperating "digital universe", entire new areas of astronomical research would become feasible.
Q: Everybody seems to be talking about Virtual Observatories. What efforts to assemble a virtual observatory exist at the moment globally?
A: There are 17 funded VO projects around the globe(early 2006). Most of them began in the period 2001 - 2003. Each one is representing up to ~20 national or international institutions. This amounts to an enourmous total expertise across. On the other hand it poses a challenge regarding the coordination of 100s of specialists of which the large majority is working on VO issues only a small fraction of their time. Hence the impression that everybody is talking about it is quite true and encouraging.
Q: What were the precursors of the VOs?
A: While the VO programmes started at much the same time and with much the same motivation, the concept of the VO has been around for some time. The NASA ADS project and the ESA ESIS project were forerunners; both of which did not really meet the expectations of a real VO. The people at the Strasbourg astronomical Data Center in France (CDS) have been working on VO-like systems for more than 10 years and their pioneering work will be essential to a real VO. The ASTROVIRTEL project triggered serveral other European Commission supported projects such as AVO. ASTROVIRTEL was managed by the ST-ECF on behalf of ESA and ESO.
Q: How did the International Virtual Observatory Alliance come about?
A: Starting in December 2001, members of the executive committees of AVO, NVO and AstroGrid started a series of regular telephone conferences with the aim of promoting collaboration. It soon became apparent that there was a serious need to co-ordinate efforts if the concept of an International Virtual Observatory was to become a reality. While each project had its own technical and scientific interests and priorities, and while each project must be able to pursue its own individual goals, there is a critical need to agree on key standards to make the VO relevant and functional for the global astronomical community. A meeting called "Towards an International Virtual Observatory" (see http://www.eso.org/vo2002) took place in Munich in June 2002. This meeting was meant to be a natural follow-on from the Pasadena meeting in 2000 that really launched the NVO in the US. The meeting was jointly funded by ESO, ESA, NASA and NSF, which, to my mind, illustrates the importance of the VO to these organisations and the transatlantic (if not global) nature of the effort. One of the most important events at this meeting was the formation of the International Virtual Observatory Alliance (IVOA) [see http://www.ivoa.net]. IVOA is not another project but an alliance of funded projects from all over the world. The aims are spelled out on the web site and include international agreements on interoperability and data standards for astronomy. The IVOA also agreed on a public roadmap for development over the next three years. It was thought to be very important that the VO effort provide concrete examples of new functionality and new ways of doing astronomy to the international astronomical community given the scale of the investment and some natural scepticism of the VO effort from classical astronomy.
Q: What has IVOA achieved so far?
A: Major milestones on the roadmap have been reached such as
- Standard data access methods: The Simple Image Access (SIA) and Simple Spectrum Access (SSA) protocols are used by dozens of services already and many more are being implemented by major observatories; the same is true for the Astronomy Data Query Language (ADQL) and the concept of SkyNodes for publishing object catalogs and data base content
- VOTable, an XML data and meta data exchange format
- Unified Content Descriptors (UCD): A vocabulary for expressing meta data concepts
- Registry: a system acting as yellow pages about astronomy resources has been specified; an intial version has been implemented as distributed, interacting (XML) data base.
- VO-aware tools: A whole suite of tools appeared in the past years supporting the various formats and interfaces. Libraries and technical tutorials are freely available on-line from the home pages of the various projects.
But more importantly, the VO started to produce science in terms of refereed publications such as the discovery of 30 supermassive, obscured, black holes, quadroupling the number of previously known objects of this particular flavor.
Q: How expensive are these projects?
A: The total investment so far is in the order of ~$20 million.
Q: What are the next steps?
A: After concluding above mentioned pilot studies it is now time to build an actual Virtual Observatory for broad use by the scientific community and the interested laymen. By 2007 this will result in many more VO-aware tools, but also dedicated web portals offering VO compliant data and compute services powered by the various contributing data centres. EURO-VO is an umbrella for such efforts in Europe.
Q: Why is IVOA necessary?
A: The IVOA is essential to the success of all the VO projects since many of today's astronomical projects are international and involve multi-wavelength data from many countries and funding agencies.
Q: How will the VO make new science possible?
A: It is fair to say that there are no killer new science problems that are unique to the VO (unlike the situation
for science cases for new ground and space based telescopes/instruments). Rather, the VO will make the scaling-up
of existing projects possible and will also make 'large projects' doable by more people with smaller groups and
less resources.
Astronomy is an international science and becoming more so every day. All the new facilities on the drawing boards
(JWST, ALMA, SKA ...) involve collaborations with the US, Europe and the Asian Pacific countries. Project teams live
on different continents and need access to large amounts of data from different wavelength bands stored in archives
in a range of countries run by multiple organisations. This is not unique to astronomy and forms the basis of the
paradigm shift to GRID technologies for solving complex problems in the 21st century. The VO is a GRID application
for astronomy. It will use GRID technologies and educate the astronomical community in their use. However, it is not
just GRID, VO is also about data interoperability. Astronomy has unique interoperability issues, as do other
sciences.
Firstly the VO will enable complex problem solving in astronomy by using the GRID paradigm to allow astronomers to
access the kinds of data volumes they need to attack really interesting new problems in cosmology and stellar
astrophysics. Secondly, we need new algorithms to work on these large data sets. The GRID will provide a way of
applying these algorithms on remote resources (probably near to the data repositories). But the algorithms for
operating on large data sets will be a product of the VO. The algorithms will come from collaborative efforts,
funded by the VO, between astronomers and computer scientists, computational scientists and statisticians.
Thirdly, the VO will provide the data-level interoperability so that diverse datasets can be joined to solve
complex problems. We often need to connect radio and infrared data or X-ray and optical data. This is usually
very hard and time consuming due to the lack of interoperability standards and automatic systems. The VO will
provide these standards and systems to make multi-wavelength astronomy practical and routine.
Q: What are the main technological obstacles to the VO?
A: As mentioned VO will be a GRID application for astronomy and more. The data volume problem for astronomy is similar to particle physics and bioinformatics. We have facilities that can turn out ~ 0.5 terabyte of data each night of observation. Archives are currently in the 10-100 Tbyte range with doubling times of less than 12 months. New facilities such as the LSST will turn out multi terabyte datasets each night. Astronomy has just as much need for scalable storage and compute solutions as well as performant database technologies as do other physical sciences. In other words, we are part of the GRID driving force as well. At the moment we see a real future in Web Services.
Q: When will the VO become available?
A: The target in Europe is to have a real open access VO covering all European data centres by the end of 2007 with roughly similar timescale in the US and elsewhere. Again, this will be progressive and not a big-bang of new functionality.
co-funded project