Dance Becoming Data Part Two: Conversation Between Anton Koch and Scott delaHunta

September 2017

In early 2014, the first funded phase of Motion Bank came to a close with the publication of the so called on-line scores of the guest choreographers Deborah Hay, Jonathan Burrows/ Matteo Fargion, Thomas Hauert and Bebe Miller. Planning immediately commenced to continue the project, but with a more visible focus on creative research and development of Motion Bank’s open software systems for recording and data management, annotation and on-line publication (PMa, PM2 and MoSys). Now hosted @ Hochschule Mainz University of Applied Sciences, Motion Bank is working with a small team of coding artists developing these systems. One of these, Anton Koch, is an artist, researcher and developer based in Germany.¹ PART ONE of Dance Becoming Data emphasises the role of dialogue between collaborating software and dance artist/ researchers on topics such as processes and contexts, critical and cultural histories. This exchange becomes even more essential today in the context of 21st century developments in digital and networked media; as the following conversation aims to illuminate.

Scott: At Motion Bank we are trying to establish that the research into the development of software systems for working with dance data is on an equal level with the research we do into embodied creative practice in dance. Motion Bank cannot do one without the other; they are on the same plane and absolutely integrated. So I’m curious about your approach, as an artist and researcher, to the development of software systems, in other words how do you see your coding work as artistic practice? [I like the section in Casey Reas’ book FORM + CODE which is titled ‘thinking in code’]. And I am specifically curious about how you see your artistic/ research role in the Motion Bank project (where we are developing systems for others to make use of, but reject the view that we are providing a service for users)?

Anton: Coding as an artistic practice is quite interesting to look at since it originated as an engineering discipline, something you use to solve problems that can be identified, abstracted and then dealt with through application of mathematical methods. In this mindset, it might seem alien to produce software that, on some level, carries or even entirely serves an artistic position and expression. I think with the dawn of what we call the creative coding practice there has been an accelerated shift in perspective, long ago preceded by people like Manfred Mohr and John Maeda, bringing Math into the Art Gallery and breaking away from the craft. Of course, coding still fundamentally serves a pragmatic purpose: you write functional code that executes and produces a result, but: this doesn’t necessarily follow a finite process employing generalisation and reduction to solve a larger problem, eventually arriving at a solution and then publishing this outcome. It rather produces more or less fragmented instances of programs that carry a statement, question or illustrate an agenda that can be messy and even might initially seem as if it does not serve any specific purpose. Creative coding, artistic practice and research converge in a constant oscillation between development, hacking, field testing and communication with partners across disciplines, while only following a very broad vision or intuition. For me at Motion Bank this currently means building on my experience in the craft of systems and product design, exploring the possibilities of digitally augmenting artistic practice within a choreographer’s native working process while letting this exchange feed back into my own practice of creative coding.

Scott: Bertha Bermudez is a researcher on the documentation projects with the choreographers Emio Greco|PC, and she has done quite a lot of work on what they call ‘naming movements’² that focuses on creating more stable relations “between words, movements, structures and possible modes of documentation and representation” (133). They developed a specific lexicon called ‘pre-choreographic elements’ referring to principles used in the creation of new movement material. This list currently consists of 17 main principles.³ I have seen this as an agreement on terms (even if temporary since the naming is fluid) and it seems to suggest a standard, although one that contains a lot of qualitative, descriptive richness and room for interpretation. How would something like this function in machine-readable terms? Or a more focused question for Motion Bank, how can data related/ management and annotation platforms (PMa, PM2/3, web standards, linked data, etc.) create a meaningful environment for this kind of complex information?

Anton: Obviously, for someone like me working rather analytically, the “Abcdaire” of Bertha Bermudez is incredibly fascinating and both having had a look at the spreadsheet containing the classifications and getting an introduction to the referenced embodied expressions at the last Choreographic Coding Lab⁴ by Suzan Tunca, who is a researcher at ICK Amsterdam,⁵ a dance platform directed by Emio Greco and Pieter C. Scholten, partnering with Motion Bank. I come to think it is a prime example of a nonreductive, yet formulaic or “encoded” approach to a local choreographic vocabulary. This absolutely connects to the way I see the role of information architecture in dance, providing a mutable semantic set that connects language with movement, all the while not claiming an absolute “truth” but rather a loose coupling with enough room for artistic interpretation.

And I guess we can agree, that the full spectrum of contained information in the expressions formed from this vocabulary can be infinitely complex, thus rejecting the prospect of retroactively “unlocking” dance through an automated “deciphering” or classification. But in order to approach a mapping of this complexity we don’t necessarily need complex tools. On the contrary, we need tools that are atomic in their functionality and are able to connect and describe large volumes of highly diverse datasets, while being accessible to humans that are not highly trained rocket scientists and come from various disciplines. For me, annotation currently is the single most important method to map and model artistic processes and bodies of work, because it allows people of all backgrounds to add to the data. This might very well be annotation of material through conscious movement by trained individuals, rather than strictly text-based expression, to name just one example. Organised through application of standards and technologies borrowed from the Semantic Web, specifically its distinct concept of Ontologies as standardized object descriptors, resulting data structures can still be machine readable, given the machines are equipped with appropriate algorithms, some of which are already commonplace, others yet to emerge in the future.⁶

There is one particular paradigm that currently excels in mapping and modeling contemporary works of art and their underlying creative processes and that is Linked Data,⁷ which in its earliest iteration already had quite some success as what we call the “World Wide Web”. It lays out the world as a graph of individually addressable resources, while not making assumptions on the nature and implications of the resources, nor the form of their connection, but merely stating that there is a reference and connection using shared and domain-specific Ontologies bringing together subjects, predicates and objects to describe semantics. In this way, both a machine and a human can traverse said graph and try to make sense of it by inferring additional connections and relations. Finally, in order to map resources to ontologies, their description must be standardized to some extent in order to be processed and eventually put into perspective. A motion capture of a dance performance does not claim to tell its story, but it rather provides a mechanical fragment of a larger structure that may or may not give clues about a deeper meaning. The W3C’s standard for web based annotation makes this idea very appealing when thinking aboutf modeling contemporary artworks and including process references.⁸

Scott: I have done quite a bit of work with cognitive scientists collecting data for their studies of dance cognition (thinking with the body, distributed creative cognition, long term memory, etc.). One of these colleagues is Kate Stevens, who wrote in 2005: “Much work remains to develop psychological theory that can explain the parallel, multidimensional, and ineffable processes at work in choreographic cognition” and “The challenge for this (dance) research is the development of testable theories that guide interpretation and explanation of interrelations between the affective, cognitive and physiological data.”⁹. Do you think our own attempts at dance datafication might benefit from any of these studies and their results?¹⁰ I’m also thinking about another colleague David Kirsh who gave the first Movement and Computing workshop keynote in 2014.¹¹ Kirsh videos hours of dance-making in the studio and uses annotation to help understand the distribution of creativity.¹²

Anton: In order to attempt an approximation of the inherent complexities of choreographic practice, the vast diversity of the processes surrounding it and mechanisms at work within it, I suppose that no singular scientific discipline is able to get much closer to a bigger picture. But in order to draw conclusions from cognitive research and dance practice, we are very much dependent on more choreographers disclosing their methods and not only letting us study their creative processes but to engage in a bidirectional process of expression and introspection. If we look at a local choreographic vocabulary as describing a non-verbal language and mapping its components to either written language or further non-verbal abstractions, then there are two fundamentally different approaches to working with the underlying and resulting data. There is a dominant aesthetic approach looking to find meaning through analysis of existing documented expressions, but there is also a poetic approach that creates or infers meaning by using existing vocabulary for new expression.

Scott: I remembered you said something about developing systems in ways that you don’t have to be a specialist to maintain and work in? Can you tell me something more about that? Do you consider Marc Downie’s FIELD (used with the CLA project of Wayne McGregor), the Gesture Follower from IRCAM a framework like ours or something more specialised? Or a multi-modal development platform like EyesWeb.¹³ As I understand EyesWeb is a closed system that cannot be easily combined with other tools and approaches. In other words, it is difficult to extract some features from EyesWeb and use them in another environment. I guess we aim for something different with our systems?

Anton: We owe a lot to the early efforts within this practical field with projects like EyesWeb, the Gesture Follower, etc. exploring possibilities of affordable and accessible augmentation of performance or research by algorithmic analysis and multi-modal data translation. But one crucial aspect was not that much on the radar during that time and that was a strict concept of interoperability, modularity and abstraction that would allow these projects to grow beyond the scope of their planned implementation by going from tool to process to mindset through abstract adoption across implementations. I remember around the early aughts, after the burst of the bubble, that voices like the Web Standards Project started getting traction and pushed for implementing standards such as those proposed by the W3C consortium.¹⁴ This effort to establish open standards for development was motivated by a simple, pragmatic desire to facilitate easier and more agile development. During that time we saw a widespread boost of free open source versus the dominant closed source strategies and proprietary techno-monopolies of the first internet bubble. The focus in the “Web 2.0” startup scene shifted from patents and technology to knowledge of methods and a virtuosity in “hacking” existing infrastructure and technology into, let’s stick to the valley dweller’s language here, “disruptive” new applications. This was supported by the web behemoths like Amazon and Google renting out their own infrastructure while essentially giving away products and services for next to nothing, boosting development while both cementing their status as the backbone of the web and, more importantly: gatekeeper for linked, user-generated data. This enormous and diverse volume of data, together with the possession of adequate computing power to mass process and infer unseen relations across its full spectrum, has now become a zombie form of Cybernetics, mainly visible as the holy grail of finance, the military-industrial complex and governments of all flavours. But this might be a good moment to get off the haunted history ride and turn to reflect on the principles fundamental to Linked Data, together with the range of available open development and distribution tools, seeing them as an unprecedented chance to create domain-specific networks aiming to map an approximation of relationships within the field of dance theory and practice built by the community itself on a peer to peer infrastructure combining decentralized ledgers, signatures and file storage, sharing computational load, optionally connecting to the large cloud providers. A dance company should not be forced to invest in server farms or a full-time admin to participate, so we need concepts that accommodate such a technologically heterogeneous network of contributors and consumers.

Of course everyone is free to develop their own frameworks and tools and it definitely makes sense to build libraries from code that you are constantly reusing, but the underlying formats and implemented ways to exchange the data should be as abstract, simple and open as possible. This is why Motion Bank rather curates from existing technologies and tries to create concepts at an intersection of creative platforms supporting a quite specific workflow like the ones you mentioned above, combined with distributed databases supplying high quality dance data alongside proposed methods that can be incorporated within the framework of your choice. It is mostly about maintaining an open and democratic access to the single most valuable and irreplaceable resource: High quality, open data sets, that are accompanied by pointers on how to make sense of them and that can be extended through interdisciplinary research, adding to intellectual value shared by partners that span from large research institutions to small dance companies or individuals. The software systems built around this, comprising of transparent and modular web-based technologies, should lower the bar for entry to enableg access by a diverse field of interested individuals, foster the adoption of annotation based practice and the collaborative mapping and modeling of complex cultural phenomena by a multitude of practitioners and theorists. Coding frameworks are vital to this ecosystem, but I think currently they can easily take a back seat to methods, data and relations.

Scott: What would be your vision for the future, once this bar for entry is lowered?

Anton: When imagining the possibilities described before and comparing them to our current situation, I feel we are still in an early phase of collecting basic data and process information. This is still very much a unidirectional process in a way that it does not yet constitute a living archive and only rarely provides a unique and meaningful extension to current or emerging practices. While this collected data might not yet feed back into choreographic culture in meaningful way, we need substance and a basis to begin laying the fundament for what i would describe as a dance-data-continuum, something that not only encapsulates fragments of heterogenous data, but is in a constant fluctuation due to processes invoking the data, interpreting it, inferring relations and adding to it. For dance to become data beyond a digital snapshot, data must become dance in a perpetual feedback-loop that integrates observers, be that human or machine, consuming the data, injecting layers of subjectivity and context. Otherwise the digital archive might just end up as a cryochamber for cultural artifacts that, while being vital for documentation, introspection, preservation and study of contemporary dance practice and history, ultimately seems to fall rather short of its possible scope of relevance in the development of future art forms and culture.

Scott: In my view, there is a need to conceptualise a new perspective that reframes the real-time computation that seems to be the initial point of attraction for many encounters between digital technologies and dance; and also resists the concept of tool-making and user-driven design which enforces a functional engineering kind of mindset (which you referred to above). I think this goes to the titling of PART ONE. Dance becoming data suggests a philosophical ontological question, although my question has more to do with issues of ontology arising from information science, some of which you have addressed already. Still it seems like we must integrate our research into dance digitisation with the pressing questions emerging from the fields of software studies and digital humanities ; on our attempts to consider the intersections and overlaps of dance and coding practices that take into account the digital networked milieu and our understanding of the affordances of dance data.

Anton: To arrive at a notion of “becoming” within our static graphs and classifications, I’d like to look at augmenting our object oriented perspective with elements of A. N. Whitehead’s notion of Process Philosophy, elegantly exemplified in Sha Xin Wei’s book “Poiesis and Enchantment in Topological Matter”,¹⁵ although with a very different agenda and I’m not sure he’d share my insistence on Annotation, Linked Data and Ontology as central to the choreographic-creative process. Yet, leaning on his ideas, my utopian dance archive adds a “primordial ooze”, the dynamic data-continuum I mentioned earlier, allowing for a bastardized form of digital Quantum-Mechanical-Metaphysics where objects come into being through cognitive and computational inference of proximities and observation of superpositions within the data flux. It in turn incorporates these results in a feedback loop that ultimately conflates human and machinic poiesis and aesthetics into entirely new perspectives.

Mind you, this is still wildly speculative and still only barely on the horizon, as even a broad adoption of native, data driven practices well as an organic growth into a critical mass of reusable data needs time. But instead of worrying about a perceived lack of variety and artificially inflating digitisation efforts or forcing them on artists, we should rather continue to lay the foundation for this dynamic continuum by starting to accept strict interoperability and non-interference with existing workflows as core axioms of our work, while critically examining our idea of “Data Aided Choreography” with its implications for theory, practice, perception and distribution within fundamentally intertwined practices of interdisciplinary collaboration and research, choreographic thinking and creative coding.

Notes