Outline of the Artiste Proposal

Summary

The objective of the project is to develop and prove the value of an integrated art analysis and navigation environment aimed at supporting the work of professional users in the fine arts

The environment will exploit advanced image content analysis techniques, distributed hyperlink-based navigation methods, and object relational database technologies. It will build on existing metadata standards and indexing schemes

Outline of work:

research and develop advanced image content analysis techniques for digitised works of art;
develop techniques to automatically categorise art works using these algorithms;
develop metadata representations for image categorisation;
develop seamless distributed access to multiple collections;
develop distributed content-based navigation methods for art collections;
build a robust and scalable integrated environment that incorporates the above components using an object-relational database;
develop a report on the impact on standards detailing augmentations of Z39.50 with RDF
establish business models that give the content owners direct control over the distributed representation, access and exploitation of digital multimedia content and metadata;
identify how the technical systems developed in the project can be deployed and exploited in a number of sectors.

Project Justification and Scope

European museums and galleries are rich in cultural treasures but public access has not reached its full potential. Digital multimedia can address these issues and expand the accessible collections. However, there is a lack of systems and techniques to support both professional and citizen access to these collections.
The ARTISTE project will address professional users in the fine arts as the primary end-user base. These users provide services for the ultimate end-user, the citizen. The main target roles are listed below.
Art researchers, historians and museum curators have a requirement for dynamic search and retrieval of high-resolution art by image content. This will substantially increase their efficiency in tasks such as matching art fragments, detecting and verifying authorship and researching painting styles and methods. They would like to reduce the time and effort required in indexing and categorising art works by automating these processes.
Publishers and educational course providers also require access to high-resolution art, with the ability to search and retrieve art work quickly using a variety of search terms and the ability to support train-of-thought analysis.

Innovation

The areas of innovation in this project are as follows:

Using image content analysis to automatically extract metadata based on iconography, painting style etc;
Use of high quality images (with data from several spectral bands and shadow data) for image content analysis of art; Use of distributed metadata using RDF to build on existing standards;
Content-based navigation for art documents separating links from content and applying links according to context at presentation time;
Distributed linking and searching across multiple archives allowing ownership of data to be retained; and
Storage of art images using large (>1TeraByte) multimedia object relational databases.

Image Content Analysis

The project represents a radical departure in terms of indexing works of art. Attempts have been made to index the objects and subjects in paintings using ordered systems, but these rely on expert knowledge of the content and in-depth experience of the classification system. The ARTISTE approach will use the power of object-related databases and content-retrieval to enable indexing to be made dynamically, by non-experts.
The high quality of images in the system, particularly in terms of colour, will allow comparisons to be made that in the past were based on information of too low a quality to support the assertions made. Comparisons made using the shadow data will allow comparison of brush strokes, etc.
Not much research has been carried out worldwide on new algorithms for style-matching in art. This is probably not a major aim in Artiste but could be a spin-off if the algorithms made for specific author search requirements happen to provide data which can be combined with other data to help classify styles.
In terms of colour matching all previous work has been non-colorimetric, i.e. varied RGB colour comparisons. The ARTISTE approach will use the unique collection of CIE colour values from the Vasari/Marc projects to carry out accurate colour comparisons. Preliminary work at UOS MMRG in this area shows that one can even begin to match reconstructed spectra with pure pigment spectra. The ability to put a colour value from say a colour meter reading of a pure known pigment in as a query to obtain a search across a collection would to ARTISTE. With similarly accurate collections such as those in the Uffizi gallery it is also possible to allow authors to find comparisons between collections, which were just not possible with non-calibrated images. NGL's fading data can also be used to provide clues about matching to colours before 100 years of light fading for example. This could help in cases such as textile/fabric sample matching.
Based on experience from working with art historians and art gallery professionals MMRG is aware of requirements in retrieval of art, rather than classification. In other words ARTISTE would aim to give searchers tools which hint at links due to say colour or brush-stroke texture rather than saying "this is the automatically classified data". Also by concentrating on specific examples such as fabric finding, pigment finding, specific shapes etc., we are producing new research results rather than trying to satisfy a huge array of users, which has been shown to be the weakest approach in content-based retrieval.

Indexing and Metadata

The ARTISTE project will build on and exploit the indexing scheme proposed by the AQUARELLE consortia. The ARTISTE project solution will have a core component that is compatible with existing standards such as Z39.50. The solution will make use of emerging technical standards XML, RDF and X-Link to extend existing library standards to a more dynamic and flexible metadata system. The ARTISTE project will actively track and make use of existing terminology resources such as the Getty "Art and Architecture Thesaurus" (AAT) and the "Union List of Artist Names" (ULAN).

Integrated Art Collections

ARTISTE will integrate art collections while allowing the owners of each collection to maintain ownership and control of their data. This will be achieved by virtually integrating the collections using the concept of distributed linking. The distributed linking will add links to content (both text and images) at presentation time. This will enable a user to add links to content that they do not own or have write access to.
Distributed links will ease the management of links by separating them from the content. This means that new links can be applied to an existing resource without modification to that resource (for example when a new image content analysis algorithm has been performed on the data set). In addition, different sets of links can be applied depending on the user viewing the resource.

Object Relational Databases for Storage of Art Images

The proposed architecture will allow multiple distributed databases to be integrated, removing the need for centralised repositories. However, as volumes are expected to exceed a terabyte, ARTISTE will use scalable object-relational database technology to manage these large data volumes.