Cultivation of Knowledge Spaces

Wolfgang STRAUSS, Monika FLEISCHMANN

MARS-Exploratory Media Lab, Fraunhofer Institute for Media Communication,

Tel.: +49-(0)2241-14-3450, Fax: +49-(0)2241-14-2133

wolfgang.strauss@imk.fhg.de, monika.fleischmann@imk.fhg.de,

http://imk.fraunhofer.de/mars

http://netzspannung.org

Research into the opportunities offered by electronic media as regards finding and acquiring knowledge for the field of electronic art and culture is the focus of the work being carried out by us at the Department of Media Arts Research Studies, the MARS Exploratory Research Lab of Fraunhofer Institute for Media Communication. ¹

In the broadest sense, the aim is to visualize information and create networked ‘knowledge spaces’ which are accessible to users as new forms of playing and learning. Experimental methods, tools and interfaces that support communication between the digital and physical spaces and investigate in knowledge creating are being developed.

We will describe the elements of netzspannung.org, an online media laboratory as a Competence Center for digital art, culture and technology. This article will illustrate the requirements on electronic and digital media concepts in the context of knowledge in extended environments like mixed reality public spaces.

Digital Media, Interactivity, Interface Design, Information Architecture, Information Visualization, Web-based Knowledge Space, Mixed Reality

Digital media, information and network technologies have profound implications for the opportunities for communicating and building knowledge. The constantly growing network of networks creates a space in which people can work together to create and depict new forms of knowledge from a universal archive, regardless of the time or place they occupy. In this sense, the Internet is both, an ‘archive’ and a ‘cultural memory’. It is becoming a space for telematic presence and is fulfilling a ‘globalisation role’ that can create a feeling of cultural togetherness – regardless of geographical location.

Below, four interferential layers of knowledge spaces will be outlined which build on current research and development work for new forms of teaching and learning at the MARS Exploratory Media Lab and from which the concept of ‘knowledge spaces’ derives.

• The "information space" is primarily where the infrastructural network of a data archive is created and categorized information (metadata) creates a data-body.
• The "explorative space" offers knowledge tools like Semantic Map and Timeline, which support the intuitive discovery of data.
• The "participation space" is where tangible, immaterial and invisible interfaces provide experimental access to enterable knowledge spaces in the form of Mixed Reality Installations and Ambient Media Lounges.
• The "mediation space" ultimately focuses on the significance of the social context on the basis of on-line and on-site Workshops.

The Internet platform netzspannung.org provided the basis for the design and development of the technical and structural elements required for a database-backed knowledge platform which supports the homogenous networking of different types of reference sources derived from existing online archives. Of particular significance in this context is the description of data objects (metadata), i.e. the categorised information that makes it possible to locate an object in a database.

Where they are accessible online at all, the main body of existing media culture archives use highly-specialised and proprietary processing systems. To date, these archives have no uniform standard for indexing archive objects with metadata.

These specific requirements from media art ultimately lead to the model of a ‘data body’, which occupies the digital knowledge space. This data body links itself to all relevant data and maps it, associating itself also with the context of the author or project. In principle, media art - with its specialist needs - requires its own descriptive language, a so-called "media art markup language".

fig 1 : Organisation structure of a "data-body" , e.g. author's or project data-body

The multimedia online archive netzspannung.org locates media art at the intersection of art, science, technology and communication. Topics such as production, distribution, connectivity, collaboration, communication, theory formation, publication, education, archiving and public visibility form the electronic arts community’s basic requirements on an Internet platform for media art. These topics had been adressed as the 10 most important issues for the production of media art in the CAT - Communication, Art & Technology – feasibility study of 1998 through an email-poll. ² Individual, curated modules of the netzspannung.org platform form the basis for growing data resources and content.

The annual competition to promote new ideas, ‘digital sparks’, is one of the modules which generates high-quality content through a peer review process. Another module is the series of "tele-lectures" – hosted by universities, museums or media partners - uses broadband streaming and archiving of key lectures to create a temporarily extended auditorium and a sustained cultural memory.

The ongoing interdisciplinary lecture serie ‘Iconic Turn’, ³ which is organized since summer 2002 by Burda Academy and the Ludwig-Maximilians-University in Munich is transmitted live from netzspannung.org into other Universities and afterwards categorized and archived on the platform for public use. Other lecture series were given to us for netzspannung.org from national and international partners, e.g. there are series on the following topics (some in english, some german) :

1. "Museum as Medium - Media in Museum. Perspektives of Museologie" of the Institute for Art History, University Stuttgart,
2. „Frames of Viewing / Perception, Experience, Judgement" of the House of the Worlds Cultures, Berlin together with the Getty Research Institute, Los Angeles,
3. „Image War and Image Floods" of the ZKM - Centre for Art and Media Technology, Karlsruhe.

Conferences, ⁴ workshops ⁵ and (online) exhibitions ^6f are further sources of salient contributions and up-to-date information. In 2003 we plan conferences, workshops and exhibitions with partners like the Goethe Forum, the House of the World Cultures, the Fraunhofer Head Office for Information & Communication Technology in Berlin, the Artist in Residence House Balmoral Castle in Bad Ems, with V2 in Rotterdam or the Ars Electronica in Linz. All events will be documented on netzspannung.org.

For the community, the platform ultimately offers open publication channels, primarily for presentations and discussion. netzspannung.org therefore provides its users with an infrastructure for publishing their own work and also generates its own, curated content. It therefore represents a constantly-growing, high-quality pool of information belonging to one specific community and is aimed at an audience of professionals and university academics.

After its first year of existence (10/2001 – 10/2002), the bilingual platform contains around 300 projects in the form of media files, texts, images, videos and some 100 hours of recorded talks known as tele-lectures. Over the period of a year, a community has sprung up of around 1.700 registered users of the netzspannung.org data pool. Over the last three months, some 22.000 visits were recorded on the site.

The Three-Layer Model visualizes the architecture of the netzspannung.org Distributed Community Engine. It shows the open, documented interfaces that allow users to implement their projects. The architecture can be understood as a "network operating system". The base is an "internet hard-disk" that allows the storing of standard formats like xml but also self-defined data models. The base has an interface that connects to an "application layer". On top of the application layer is an "interface layer" for creating individual skins. The architecture supports various protocols like Corba, Soap and http, making it very flexible and offering different layers of complexity or simplicity. Thus, the architecture is attractive for beginners as well as for experienced users.

fig 2 : Sketch of netzspannung.org platform architecture

The aim of future developments is to network netzspannung.org with other archives. To achieve this goal from a technical point of view, interfaces will be defined as web services and a "dynamic archive adapter" will be developed. In this context, semantic web technologies can be of particular benefit, since they automatically establish a metastructure at semantic level between heterogeneous content and the systematics of different archives. Netzspannung.org then facilitates central access with an interface to a decentral network of various archives and becomes, in the true sense of the word, a portal.

Just as large telescopes help astronomers to see the stars, digital cultures need new instruments to be able to see, survey and evaluate the rapidly-growing volumes of data. The Semantic Map interface from netzspannung.org represents a ‘telescope for viewing and evaluating the data cosmos’, as astronomer Roger Malina put it at Artmedia VIII, held in Paris in October 2002.

"Knowledge discovery tools" (Strauss et al., 2002) - one of which is the Semantic Map - are special tools which permit a ‘bird’s eye’ view of large volumes of heterogeneous data and facilitate the visualization of data resources as contextualised information spaces. Because they also support active structuring, preparation and communication, they are also recognition tools._<0} According to the context in which they are used, three different interfaces have been implemented so far.

• The "Semantic Map" compiles content into clusters and facilitates an explorative navigation of interdisciplinary relationships based on semantic interrelations.
• The ‘Timeline Interface’ arranges content in parallel into various categories and time (x-, y-grid) in order to identify chronological relationships between different fields of content.
• The "Knowledge Explorer" is a more complex tool for communities of experts. Experts can use it to structure data pools, but also create personal knowledge maps and share them with other members of the community who can then tap into uncharted pools of information.

In the next stage of development, the tools will be personalised and made available for use online. The members of netzspannung.org can then use these to create personal knowledge maps based on their own content and an information pool they have selected themselves. (Novak at al., 2002)

Below, the Semantic Map will be described in more detail as an example of knowledge discovery tools. The Semantic Map is an interface, which evaluates and visualises semantic links between individual documents in the netzspannung.org database. Interactive visualisation provides an overview of the contexts and relationships between the data. It also gives access to individual documents which are then displayed in separate windows. For the conference entitled "cast01/living in mixed realities" ⁷ organised by the MARS research group, the Semantic Map serves as an overview of the conference contributions that can be used for research purposes. The following illustration depicts the map’s structure:

fig 3 : Semantic Map interface: layers of information and data processing

1. Each document is represented by a small blue square on the map.
2. Keywords provide information on the content of a cluster.
3. Documents with a greater number of content links are grouped in clusters.
4. The visualization process is based on the Kohonen Map procedure: It connects all the documents to each other and arranges them in a two-dimensional grid according to their content links.

The question of how online archives can be implemented on both metaphoric-virtual and physical-real levels as enterable and tangible knowledge spaces is answered by the mixed reality methods developed by the authors for penetrating and overlayering physical and electronic spaces. The notion of "Mixed Reality" was mentioned for the first time in 1994 by Paul Milgram and Fumio Kishino in their paper on Mixed Reality visual displays aiming to enhance the limited research field of Virtual Reality (Milgram and Kishino 1994). In contrast to classical, two-dimensional interfaces, users of mixed-reality installations are drawn into the centre of the proceedings.

The ‘electronic Multi-User Stage Environment’ (eMUSE) developed by the authors, enables for a mixed-reality space continuum in which an enterable audio archive - the Murmuring Fields (Fleischmann et al., 2000) - as an interactive sound space for multiple users, can create the impression of a concert room. The space appears as a room furnished with data (Strauss et al., 1999). The room stands for a physical interaction space but the furniture of data is virtual and stands for an information space. It is a spatially organised information architecture in which data is revealed through users ‘movement and action in the combined real-virtual space,and through interaction with other users. The mixed reality architecture described opens up windows to data-space; conversely data-space becomes present in real space.

fig 4 : Imagine a room furnished with data

Our concept reverses the idea of a mnemotic technique, which is normally used for rhetoric purposes, into a spatial mnemotic technique. It is based on mentally assigning parts of dialogues and sounds to specific parts of the space, and then delivering, scratching and acting that dialogue by walking, dancing, performing in the whole space. The virtual space is structured in four zones with speeches from scientists and philosophers in different languages like english, german, french, english. By walking around, the performers play with this sentences, words, syllables, phonemes in a poetic way and give new meaning to the words by playing with it. The words are taken from interviews with Vilem Flusser, Marvin Minsky, Joseph Weizenbaum and Paul Virilio like they were included in the "Home of the Brain" VR installation of 1992, ⁸ but the audiovisual construction changed for this performative mixed reality environment.

The Electronic Multi-User Stage Environment (eMUSE) is an experimental software platform for the fusion of virtual and real physical spaces. The audiovisual module and the network
module of the eMUSE structure support networked scenarios in real-time based on
Java and a VRML browser. eMUSE is built as a modular system providing independent levels of implementation for rendering and displaying the virtual environment. It supports multiple users in shared physical and virtual spaces and allows other interfaces to be integrated easily. The system consists of three main parts: the external user interface driver, the multi-user support driver and a VRML browser. eMUSE not only integrates several people live in a shared space and represents them in a virtual environment, it also allows the collaboration of spatially remote users via the Internet.

The Body Track system has been developed to map participants' physical movement inside physical spaces to a virtual environment. The system tracks the movement of several people simultaneously and currently provides the following functionalities: It monitors the shape, position, speed and direction of up to five people in real -time. The co-ordinates of the user are mapped onto the co-ordinates of the virtual space allowing navigation in the virtual environment. The tracking data provided by the system is used to control the position and the appearance of the user representation (alter ego or avatar), and to control individual viewpoints in the virtual space (VRML scene). Body Track provides important components for performance art and combines them with interactive networked scenarios: the possibility of free movement in space, the transmission of that movement in real time to a virtual environment, and the participation of several performers and their interaction with each other.

With the performance installation "Murmuring Fields", we developed an audio archive and a sound space for the mixed reality stage. Data spaces and action spaces are linked via an intuitive interface. In the interactive sound space "Murmuring Fields", performers experience the sound of the virtual space through movement. They ‘play’ the space with their bodies, as they would play an instrument.

fig 5 : Layers of a Mixed Reality-Stage Installation

Based on the eMUSE technology developed in this project, we are adapting knowledge tools for the physical space under the title "the enterable knowledge space". In the initial studies, the recipients move over a semantic map covering the floor, and this now becomes a ‘life-sized’, enterable orientation landscape by virtue of the on-screen display. The data space – now a three-dimensional archive – can be explored by moving through the space and making appropriate gestures. In an interactive, dynamic interchange, content (text, image, video, sound) is selected and depicted in the space.

fig 6 : Initial studies for "the enterable knowledge space": enterable "semantic map" and gesture based "timeline"

Mediation Space

The process of discovering and developing concepts for knowledge communication in digital cultures must extend beyond simply integrating new technical concepts, and must also investigate the social contexts of teaching and learning. At MARS Lab we initiate experimental workshop events, which examine classical teaching situations. The experiments discussed below using two examples contain cross-university, interdisciplinary and project/action-focused approaches in media culture education. In the classical education system, young people are still being educated using mostly verbally-fixed thought and analysis methods. Even though this efficient teaching and learning method continues to be of fundamental significance, current developments in education are showing how important a knowledge of individual skills is in graphic-associative or kinaesthetic thinking in order to be able to exist in the media reality of our society.

The MARS workshop entitled "I see what you hear" is an example of experimental, project-focused learning in heterogeneous workgroups of children with ‘suffer’ in school on their natural gifts, on phenomenons like dyslexia, synesthesia and some on hyper intelligence. The participants at the teaching event comprised young people aged between 10 and 15 years, plus art and art therapy students. Working together as equal partners, they developed concepts for kinetic objects after they were trained with experimental methods to use their senses in a playful and synesthetic way in order to be more creative. Using the technical principles of the theremin ⁹ a series of small, interactive exhibition pieces were developed in the workshop. The workshop leaders - three of our PhD students at MARS Lab - were chiefly there for moderation purposes. The first stage for the total of eighteen participants involved only the students, who were introduced to the topic over workshops lasting seven days. Then came working groups made up of one young person and one student in the practical part of the workshop. At the end, the results of their work were presented in an exceptional public exhibition. ¹⁰

The link between practical work and reflection led to a playful and sensory workshop that gave the participants the opportunity to play an active part in designing the workshop content and create funny and poetic semi-autonomous kinetic objects. The various methods of approach used by the groups, whose age and knowledge differed, and the interdisciplinary approach of integrating state-of-the-art information technology into an artistic and playful context gave rise to extremely productive potential for both sides. The workshop participants share the experience of independently creating the theremin circuit on a circuit board they have soldered themselves – along the lines "Think first, then solder": The bee flies over the flowery meadow which is humming with the sounds of summer and a mouse runs squeaking among the legs of the visitors. Most impressive is the water installation, which is spreading out and exploring the exhibition space as soon as visitors pass by – it looks like an hommage to Fischli & Weiss.

fig 7 : Workshop activities : I see what you hear

Tele-Lectures

Universities are increasingly being faced with tighter budget restrictions. This makes it more difficult to cover interesting topics and bring eminent lecturers on board. The concept of a decentral network of auditoria from different universities is one attempt to address this problem. With ‘tele-lectures’, the MARS ‘Mobile Unit’ streaming lab, is linking the auditoria of several media and art colleges and universities via broadband Internet streaming to form a single telematic space. A lecture being given in one location is transferred to the auditoria of the other participating universities and moderated there. Discussions then follow. All lectures are also recorded and archived on netzspannung.org. In contrast to conventional Internet TV broadcasters, the focus of the tele-lectures lies on the shared experience of live transmissions on the spot and the embedding of content in the context of university teaching.

The further development of the tele-lectures format focuses on the creation of a feedback channel which facilitates communication between the lecturer and the audience listening at the various locations. In future, all auditoria will need to have the basic equipment for incoming and outgoing live transmissions. This infrastructure should facilitate an audio-visual exchange between universities that requires as little maintenance as possible and is easy to use. The ‘Mobile Unit’ of the MARS Lab functions as a mobile multimedia streaming laboratory prototype for the mobile classroom. (Pfuhl and Peranovic, 2002)

In our current projects, at the MARS Exploratory Media Lab we are planning to link the layers described in this paper (information space, explorative space, participation space and mediation space) to form a single knowledge space for media culture education which bridges the digital and physical spaces.

The knowledge discovery tools, with their opportunities for visualizing information, the explorative investigation of large volumes of data and the creation of personal knowledge maps, represent a key navigation tool and method for shifting through and visualising the data space. The limits of two-dimensional user interfaces are increasingly emerging as inadequate. The space that surrounds us and the objects around us can however, when given digital capacities, become an extended space – a mixed reality environment. The authors believe that the transition to the surrounding space, which becomes an interface, represents a key step towards giving humans the most intuitive access possible to information technologies. Finally, reference is made to the significance of social contexts for communication concepts, without which learning and creating is barely possible.

Presently on netzspannung.org an online exhibition is being investigated on the topic of "cultural heritage". It will present applications to exemplify the actual digital cultural heritage. The aim is to connect netzspannung.org to other archives and build up a networked space of cultivation online and build up a public space with a Media Lounge in a Museum on site.

Fleischmann, M.; Strauss, W.; Novak, J. (2000). Murmuring Fields Rehearsals - Building up the Mixed Reality stage. Proceedings of KES, Brighton.

Fleischmann, M.; Strauss ,W.; Blome, G.; Novak, J.: Paal, Stefan (2002): netzspannung.org – A Collaborative Knowledge Space For Media Art And Technology. Ichim02 – International Cultural Heritage Informatics Meeting, Museums and the Web conference, Boston 2002

Gorbet, M., Orth, M. & Ishii, H. (1998). Triangles: Tangible interface for manipulation and exploration of digital information topography. Proceedings of conference on human factors in computing systems, Los Angeles, (CHI 1998). ACM Press, pp. 49-56.

Griswold, G. at al. (2002). ActiveCampus - Sustaining educational communities through mobile technology. UCSD CSE technical report, CS2002-0714.

Ishii, H. & Ullmer, B. (1997). Tangible bits: Towards seamless interfaces between people, bits and atoms. Proceedings of conference on human factors in computing systems, Atlanta, (CHI '97), ACM Press, pp. 234-241.

Kohonen, T. (2001). Self-organizing maps. New York: 3rd edition, Springer.

Milgram, P. & Kishino, F. (December, 1994). A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems, Vol E77-D. Miller, E.P. (1972). Interaction of vision and touch in conflict and nonconflict form perception tasks. Journal of Experimental Psychology, 96, 114-123.

Novak, J; Fleischmann, M.; Strauss, W.; Wurst, M.; Morik, K.; Ziegler, J.; Kunz, C. (2002);
Augmenting the knowledge bandwidth and connecting heterogeneous expert communities through uncovering tacit knowledge, in Proceedings of IEEE 4^th International Workshop on Knowledge Media Networking, IEEE Computer Society Press, CRL, Kyoto, Japan, July 2002.

Pfuhl, D.; Peranovic, P. (2002): Mobile Streaming Lab – leading to a modular Learning Environment, conference proceedings, VSMM 2002 Creative Digital Culture, Gyeongju, Korea, 25^th-27^th September 2002.

Strauss, Wolfgang; Fleischmann, Monika et al. (2002), Knowledge Discovery and Memory Space as asymmetric Information. Paper for IEEE proceedings CA 2002 Computer Animation, Geneva

Strauss, W., Fleischmann, M., Thomsen, MR, Novak J., Zlender, U., Kulessa, T., Pragasky, F. (1999), Staging the space of mixed reality – reconsidering the conceptof a multi-user environment. VRML99, Paderborn, Germany, February 23-26 1999.

Weiser, M. (1991). The computer for the 21st century. Scientific American, (11/1991), 94-104.