Succession

Succession (Whitelaw, 2014) addresses the industrial and pre-industrial heritage of Newcastle-Upon-Tyne and surrounds, reflecting my encounters with the city during an extended visit, and the meditations on energy, history, industrialisation and capital that the place provoked. Newcastle was a key Roman settlement, later one of the crucibles of the Industrial Revolution, now finding its way in a ‘de-industrial’ Britain (Chakrabortty, 2011). While the work is highly localised, it reads this place as a cypher with far wider implications. The work’s title draws a term from ecology, alluding to continuous and ongoing change and adaptation. Succession aims to mine the city’s industrial past in order to fuel consideration of our possible futures.

In practice, the work is a web application that draws on a corpus of some two thousand image records harvested from the Flickr Commons, and combines these elements into new composites (or fossils). Each fossil is composed of five randomly selected source images arranged, composited and potentially repeated. Sources are cited, so that while composites often radically obscure, transform or juxtapose their elements, the sources and their attendant contexts remain navigable and intact. Composites may be saved, acquiring a permalink to become a new citable online object. The generative process of composition is performed live, in the browser; each viewer will encounter a series of unique composites. The system allows for around 2.5 × 10¹⁵ combinations of elements (ignoring spatial and blending variations). At a rate of one combination per second, it would take around eight million years to show all permutations.

The saved composite in Figure 1 shows how the generated artefacts can operate as what Drucker and Nowviskie call ‘aesthetic provocations’ (2004: 431) while enlisting the contexts and referents of their source elements in speculative juxtapositions. This composite is dominated by a 1993 photograph of Wearmouth Colliery in its final week of operation, a poignant image of the last days of Newcastle coal. But a spectral water-bird (from Gould’s 1837 Birds of Europe) seems about to splash down in those desolate puddles. Faintly in the background is the bustling River Tyne itself, circa 1880; and on the left of the frame a carriage destined for Newcastle’s Metro system, under development in the mid-1970s. Almost imperceptible at bottom left is the HMS Opal, a torpedo destroyer, under construction at the Sunderland shipyards in 1915. So this composite encompasses not only 150 years of urban history, but a latent portrait of twentieth-century capital, the rise and (UK) fall of extractive industry, war, urbanism, and pre-Industrial naturalism and the non-human lives it records.

Figure 1

Succession saved composite 1413513552860 (Whitelaw, 2014a).

This example shows how formal and visual transformations inflect narrative or historical interpretations, and thus that these generative artefacts are not simply bundles of citations, but speculative visual propositions. Layering emphasises simultaneity and atemporal juxtaposition, rather than chronology: Gould’s duck, about to dive into the colliery puddles; or perhaps swimming on the 1880s Tyne? Faded traces evoke the presence of the lost; visual collisions prompt an interpretive search for coherence.

One byproduct of this generative approach is that it is impossible to characterise its results exhaustively. Instead, we can think of the system as defining a large space of potential outcomes. Artist Bill Seaman has developed a number of works using similar techniques: he outlines a ‘recombinant poetics’ (Seaman, 2001: 424) that seeks ‘emergent meaning’ (ibid.) within digital arrays of textual and audiovisual elements. Seaman (2001: 426) usefully describes his combinatoric generative system as an ‘authored electronic space’, emphasising that it is expansive, but not arbitrary. Where Seaman created each source element, Succession imports them; but nonetheless its generative space is highly authored — in part through this curation of its corpus. Sources were selected for content relevance, as well as visual potential. The content-base grew and was pruned around the conceptual focus in a slow process of subjective evaluation, exploratory search and tangential investigation. The Internet Archive’s Flickr stream was a key influence: this immense set of bulk-harvested book illustrations offers uncurated insights from a range of sources, including nineteenth-century trade journals and volumes of naturalist scholarship. The resulting corpus is diverse and heterogeneous, but retains enough commonality to generate meaningful connections; in Seaman’s words, this authorship seeks a ‘resonant unfixity’ (Spielmann, 2002).

The rules for compositing elements are also authored, tailored around the idiosyncracies of the sources and the poetic aims of the work. Layered composition is both formal machine and metaphor: to address a city built on coal it seemed necessary to combine and compress, to obscure while hopefully at the same time intensifying the energy latent in those sources. Thus, the image blending modes are biased to overlay dark elements; this treatment resonates with the engraved illustrations in the Internet Archive and British Library collections. Process and corpus (algorithm and data) are interdependent, but equally authored. Rather than a neutral, inert (Cartesian) space, this authored space is co-constituted by an active assemblage of media, concepts, subjective interests and computational processes.

Drifter

One of the unexpected outcomes of Succession was the folding in of documents of nonhuman lives, and the poignant juxtaposition of natural science illustrations with the machinery of industrialisation. This prompted an interest in how digital archives might be used to more intentionally address the troubled complexities of landscape. As well as drawing on Nowviskie’s conception of digital humanities in the Anthropocene (2015), this approach was informed by John Thackara’s (2015) advocacy of a ‘bioregional’ practice focusing on local ecological systems including food networks and water catchments.

Drifter (Whitelaw, 2016) was developed in early 2016 in response to an invitation from the Land Dialogues conference at Charles Sturt University, Wagga Wagga (Land Dialogues, 2016). Canberra, where I work and live, shares a river catchment with Wagga: Land Dialogues, a multidisciplinary meeting sited on the Murrumbidgee, offered an ideal venue for a project investigating multi-layered digital representations of our shared river system.

The Murrumbidgee River is a major inland river in south-east Australia. It stretches some 1600 kilometers from its headwaters in the mountains of southern New South Wales (NSW), passing close to Canberra before running west through inland rural centres including Wagga Wagga. Like many of Australia’s watercourses, the river has a dense and troubled history of European settlement, exploitation and intervention. It remains a site of significant conflict and controversy, focused on the competing demands of environmental conservation and agriculture. The river’s course includes multiple dams and weirs, constructed over the twentieth century to divert water for irrigation and electricity generation. Its catchment also sustains ecologically significant swamps and wetlands, including the Fivebough and Tuckerbil wetlands, recognised under the Ramsar convention. The river’s flow is regulated under the 2012 Murray Darling Basin Plan, which sought to reduce agricultural extraction and return enough water to ensure the river’s long-term sustainability. The plan has been greeted with open hostility from some communities along the river; in 2011 young men in Griffith, NSW burned copies of the draft plan in protest (Barlow & Doyle, 2011).

Drifter approaches this landscape as an expansive bundle of post-settlement concepts and perspectives on place and site including geography, history, ecology, agriculture, social discourses and cultural representations. These dimensions are formative forces, intervening in the land (building dams, diverting water for irrigation) as well as offering divergent epistemological and ontological frames through which the landscape is represented (as a resource, as habitat, as a figure in a narrative of settlement and colonisation). As the ongoing conflict around this landscape shows, these divergent frames are far from resolved. Drifter begins from the premise that traces of these diverse dimensions of the landscape persist in various forms, and are legible as data — digitised photographic records, administrative archives, newspaper reports, scientific observations, geospatial information. Its core question is how this data can be drawn together to address the tense, complex multiplicity that continues to characterise the river catchment. As in Succession, it aims to draw together and recombine digital collections for speculative ends, inviting unforeseen interpretations and unpredictable connections.

In practical terms, Drifter is a set of three web-based interfaces or displays. Each of these draws on specific datasets and collections and recombines them in specific visual and temporal structures. This approach acknowledges from the outset the impossibility of a total or comprehensive representation of this landscape or its data traces; these three displays are intended as partial and provisional views, drawing on Drucker’s notion of parallax in humanities displays — the revealing divergences between distinct but related representations (Drucker, 2011).

Map

Drifter’s Map offers a geospatial view of the river catchment, using familiar web map methods to create a navigable surface for exploration. However rather than use pre-existing cartographic ‘base layers’ (like the ubiquitous Google Maps) this map selects and overlays a limited set of features. In an effort to defamiliarise the landscape, and in particular to unwind the anthropocentric focus of traditional maps, it omits roads, rail lines, administrative boundaries and other conventional features such as elevation; it includes some towns and cities only as minimal indications. Instead, this map emphasises water in the catchment. It combines and layers geospatial data from a range of sources, including OpenStreetMap, the Digital Chart of the World, the US Geological Service HydroSHEDS dataset and the Australian Department of the Environment and Energy’s Australian National Aquatic Ecosystem classification framework, which provides detailed documentation of wetland areas within the catchment. Collectively these datasets map not only the Murrumbidgee River but its entire filigree network of creeks, streams and tributaries, as well as a multitude of other watery structures, intermittent and permanent: swamps, soaks and flood plains as well as dams and weirs.

Visualised, these structures generate a satisfying intensity of detail, but also prompt a foundational question: what is a river? The HydroSHEDS dataset prompted this question during the project development. This global hydrological database is based on land elevation data, and thus records watercourses at extreme spatial detail: a psychedelic overload of capillaries known as stream lines (Figure 2). This water-landscape is both recognisable and unfamiliar. When overlaid with water areas and wetlands in the Drifter map, the Murrumbidgee no longer seems a single, discrete thing, but a dappled field of ponds, lagoons, creeks and rivulets, fans and meanders; many complex things, not one simple blue line on a map (Figure 3). Visualised in this way, the data suggests that this river may not be what we imagine it to be.

Figure 2

HydroSHEDS stream line data, south-eastern Australia. Image by the author.

Figure 3

Drifter Map (from Whitelaw, 2016). Murrumbidgee catchment between Griffith and Hay, New South Wales.

Two other datasets are overlaid on the map’s base layer. One is a corpus of newspaper articles linked to 37 different locations within the river catchment: towns and other European settlements as well as lakes, dams and weirs. For each location, a set of up to 200 article references were harvested from the National Library of Australia’s Trove digitised newspapers service. Along with place names a set of query terms (‘river’, ‘creek’, ‘dam’, ‘irrigation’, etc) was used to extract articles referring to the river system. Article excerpts appear spontaneously on the map, anchored to their related location, showing snippets of the digitised text, along with links to the full article on Trove. In line with Trove’s overall coverage, articles are largely from the period 1880 to 1945, a period that coincides with many of the major post-settlement interventions in the river system, including the establishment of the Murrumbidgee Irrigation Area.

The second dataset is a corpus of scientific observations, drawn from the Atlas of Living Australia (ALA), an aggregator of biodiversity data. The ALA includes detailed taxonomic data structures and geospatial search functions, along with a well-documented API. Across the whole Murrumbidgee catchment, the Atlas lists millions of individual observations. Here a slice of that data is used, focusing on a single taxonomic family. Frogs (Amphibia) are tightly bound to watery habitats and heavily impacted by the degradation of the river. Several frog species endemic to this region are critically endangered, such as the Southern Bell Frog, the Yellow-spotted Tree Frog and the Southern Corroboree Frog. The ALA lists some 6700 individual geolocated observations of 37 frog species within the catchment. Observations date as far back as 1891, based on specimens held by the Australian Museum (Anon, 1891), but most observations date from after 1960. Like the newspaper articles, the frog observations appear on the map spontaneously, randomly selected based on the current field of view, and linked to the full ALA record. Audio of the frog’s call is linked with each observation wherever possible, so we both see and hear these data points. Their appearance is sequenced in order to overlap, so that the observations are artificially reconstituted into a chorus, an audible trace that is also a spatialised sonification of frog occurrences across the river catchment.

Through extracting, combining and layering datasets, the map interface forms a stage, a designed space constituted through juxtapositions. These juxtapositions are random and in part unpredictable, much like the image overlays of Succession. A key difference here is that the data sources are intentionally heterogeneous; orthogonal rather than parallel; and in this sense the collisions between layers are designed (or staged). The juxtaposition of frog observations and newspaper excerpts is a staged conversation whose moment-by-moment details are uncertain, but whose broad range of concerns are, to use Seaman’s term, resonant rather than disparate.

Compositor

The third and final view takes a contrasting approach; like Succession, it draws on visual records and constructs composites, though there they are simple crossfades between pairs of images drawn from different historical domains. Drifter was initially presented in a gallery exhibition context, as a non-interactive display, cycling between its three views. In this presentation, the compositor acted as an interstitial or transition mode, between the longer form, text-centric Sifter and Map visualisations. The collection data here consists of two harvested and curated image sets. The first is a set of some two hundred historic images linked to the river, drawn from collections including the National Library of Australia, State Archives of NSW and the Powerhouse Museum. These date largely from the late 19th century to the mid 20th: major interventions such as Burrinjuck Dam, the Irrigation Area and its newly-planned settlements are documented in maps, plans, diagrams and photographs. Scenic views, sketches and mementoes of river settlement also feature. The second set is a collection of 49 more recent photographs. Provided by freshwater ecologist Dr. Skye Wassens and the Institute for Land Water and Society, Charles Sturt University, these images document scientific fieldwork in the wetlands of the Lowbidgee and the mid-Murrumbidgee between 2008 and 2015. This set includes close-up specimen photographs recording specific frog, insect and fish species, as well as wider views of the wetlands, verdant lagoons and reedy marshes. Although gathered as documentation of ecological fieldwork, these images are often strikingly beautiful, implying a subjective point of view with a strong attachment to these places.

The compositor randomly selects a pair of images, one from each set, and stages a gradual cross-fade between them. As in Succession, blend modes are used to generate unpredictable interactions between the two layers such that one will selectively reveal or obscure the other; the intention is to generate and prolong a liminal state where both documents (and both domains) coexist. Here too a simple generative, combinatorial process multiplies two small datasets into an expansive set of possible outcomes — almost 10,000 different composites. At times these juxtapositions are relatively blunt (a frog, cupped in a human hand/Burrinjuck Dam under construction); at others they can be more subtle and ambivalent. In either case, much like Succession this view uses selective harvesting and curation combined with generative unpredictability to seek out resonant collisions between collection items (Figure 4).

Figure 4

Drifter compositor (from Whitelaw, 2016). In this composite are: (1) a 1907 image of a cable tram trailer, constructed for the Burrinjuck Dam tramway (NSW State Archives, 2010); and (2) a 2011 photograph of Avalon Swamp in the Lowbidgee Wetlands, provided by Dr. Skye Wassens, Charles Sturt University.

1. Data as Troublesome Trace

The foundational constituent here is data, for to be blunt these works draw data, not things, together. They harvest tens of thousands of records, taking the form of metadata, textual data and digital images. The value and scope of any mashup is inevitably conditioned by its source data, and here we have reason to be critical. As Drucker argues, the etymological underpinnings of data as ‘given’ is problematic for humanities thought, bringing with it a realist epistemology of an observer-independent world (Drucker, 2011). Drucker proposes as an alternative capta, which is ‘taken’ rather than given; that is, actively shaped through selection, observation, interpretation and structuring. The heritage records that these projects build on are indeed capta, typically constituted through multiple stages of ‘taking’, including the acquisition of a physical object (photograph, newspaper article or biological specimen), its registration within an institutional collection, according to specific norms and practices, and the eventual digitisation of metadata and/or object. At every stage along this chain of provenance, data is shaped and conditioned through a stack of social, cultural, organisational and technological processes. Computation plays an increasingly significant role. The Internet Archive’s Flickr stream, a key source in Succession, is the product of an automated computational process that harvests image content from the Archive’s digitised books (Miller, 2014), enriching the collection while also radically amplifying its scale. Optical character recognition, used to automatically transcribe Trove newspaper articles, is similarly invaluable in making article content digitally legible even though its output is riddled with errors. As Miriam Posner (2015) argues, humanities data is messy, unwieldy, and demands careful interpretation. In this sense data is troublesome.

This character prompts caution, but it does not negate data’s value or agency as a material in these works. These projects maintain faith in the connection, however mediated and conditioned, that data offers to diverse worlds and actors. Data here is a troublesome trace. It is undeniably constrained in its scope; it indexes a tiny portion of the complex histories and landscapes and systems at issue. In documenting the pasts of these places and landscapes it shows only what was legible, relevant or significant to the record keepers. Those partial, conditioned records are further mediated through the socio-technical processes of digitisation. These factors would trouble any summative, generalising conclusions — any move to reduce or collapse data into definitive findings. By instead using generative techniques that retain the fragmentary specificity of their data sources and privilege open-ended interpretation, these works seek to expose and build on the trace that remains, and to draw things (not only data) together.

2. Sampling

In working with large datasets, these projects face a question of representational scale. In previous work with digital collections, I have used data visualisation techniques to generate compact summative representations of whole collections (Whitelaw, 2015). These approaches rely on abstracting details of individual records, grouping and clustering them according to their metadata. Succession and Drifter demonstrate a complementary approach based on sampling. Rather than generating abstracted, summative representations, a sampling approach represents the collection through its individual constituents. This technique has several distinct dimensions. Firstly, it is a form of systematic statistical representation. Given a repeated, random selection method, a set of samples offers a compact but statistically representative subset of the whole corpus. In these works, samples often unfold over time, developing a cumulative impression of the dataset. This approach reflects the empirical use of random sampling in the sciences; a pragmatic approach to dealing with scale and complexity.

This dimension has nothing to say about the content of the sample, however; in fact it entails a statistical averaging-out that blurs specific details. The metonymic dimension of the sample begins to account for the way that specific content operates, as extracted fragments often signify through part-for-whole relations. A photograph from the Newcastle-Upon-Tyne shipyards is a documentary fragment that also signifies an overlapping set of larger contexts and concerns: shipbuilding and shipyards in general; Newcastle’s industrial heritage; Northern industry and its decline; the de-industrialisation of the UK. Similarly, an ecologist’s image of a frog from a Murrumbidgee swamp, cupped in a human hand, stands as part of a complex network including other frogs, river habitats and systems, other wildlife, scientists and institutions. Samples also act as metonyms in relation to their sources, and the disciplines and domains they represent. A diagram from the Transactions of the Institution of Mining Engineers showing the forces on a suspended cage in a mine shaft, speaks to a specific professional domain and its associated epistemology (Internet Archive Book Images, 1898). The metonymic function of these samples depends heavily on an interpretive understanding, on the knowledge that links these nested and associated contexts and concerns. As such, it too involves a degree of abstraction; the part-for-whole function takes the sample to represent something greater than, but also other than, itself.

The third and final dimension is the sample as an evocative fragment in and of itself. These samples are never self-sufficient, but shards of larger narratives, discourses, institutions and contexts. In both Succession and Drifter, links to the public source of each sample encourage the audience to pursue and parse these connections. On 14 November 1966, the Canberra Times reported: ‘Four bulldozers changed the course of the Cotter River in dense bush country south-west of Canberra on Saturday morning, in the first major step of the Corin Dam wall construction’ (Anon, 1966) (Figure 5). This fragment points to highly localised, specific events in time and space. That specificity is in part indexical — it cues the function of the data as an indexical trace. At the same time it is evocative, and in a sense incomplete: its terse description is a prompt for imagination, an invitation to fill in the details of the story.

Figure 5

Cotter River Diverted, Canberra Times, 14 November 1966.

In fact, this fragment can equally be read under all three dimensions of the sample. It is specific, but also metonymic: these bulldozers stand for all bulldozers, building dams, bridges, weirs, channels and other river interventions; and in turn for the whole modern paradigm that reshapes the landscape in this way. It is metonymic, but also a random sample; just one of over 5,000 harvested snippets on the Cotter River used in Drifter. Here these three dimensions of the sample operate concurrently (Figure 6).

Figure 6

Three concurrent dimensions of the sample. Image by the author.

3. Generative Recomposition

In Drifter and Succession, fragmentary samples are recomposed into complex, unstable composites. This recomposition makes use of simple computational techniques, but builds on the resonance of its content to generate expansive outcomes. The works show how simple generative processes can be developed to deal with digital heritage content, and begin to outline the space of similar applications.

In Succession, each fossil is composed of five unique, randomly selected images from a single corpus of around two thousand. The selection of elements here is independent of their content, so the diversity of the corpus, curated through the project’s development, is a key factor in the character of the results. A second layer of generative logic controls the appearance of each composite. A set of defined presentation modes position an image as either covering the whole frame, contained within it, or repeated in a vertical or horizontal strip; and a set of compositing rules control the visual layering of the images within the composite. In combination with the random selection of sources, these rules describe an immense range of variation, but at the same time they constrain that variation within an authored space of potential outcomes that is constituted through both the corpus and its recombinant logic.

Where Succession uses a single heterogeneous corpus, Drifter uses multiple corpora, and in this case generative techniques juxtapose elements between corpora as well as within them. The set of frog observations in the map view share both subject matter and institutional domain, forming a coherent data layer. The newspaper excerpts are more diverse in content, but remain bound by the shared features of their sources. As a result, the character of the generative composite is shaped more by the character of each corpus than random selections within it. Each sample functions metonymically to refer to its corpus, and its associated domain. Here generative recombination juxtaposes divergent perspectives and ontologies, as well as data-points; mashing up domains is one way that these techniques can draw things together, without erasing their differences. As Latour’s ‘compositionist manifesto’ states, ‘things have to be put together … while retaining their heterogeneity’ (Latour, 2010: 473–4).

Computational autonomy is a key attribute of these generative techniques. Automated selection and juxtaposition generate astronomical spaces of potential, and thus unforeseeable collisions between samples. As in other uses of generative techniques in design, the payoffs for this unpredictability are in part hedonistic: the constant variety of outcomes promises endless potential; sparks of surprise offer small pleasurable rewards. Like other mashups, these techniques also entail a sense of liveness: as fragments are continually sampled and recombined, we encounter something that is happening, rather than something that has happened. In working with digital heritage content this is particularly relevant. While the samples refer to mixed and heterogeneous pasts, their continual reassembly enacts a restless and ongoing renewal, a call for reconsideration.

Ross Gibson recognises something similar at work in his project with Kate Richards, Life After Wartime (LAW). LAW is a ‘story-engine’ that recomposes an archive of crime scene photographs together with sound and text fragments in the service of an affective, emergent understanding (Richards, 2006: 453). For Gibson (2006) the process of re-connecting and re-animating fragments is a response to the ‘aftermath cultures’ of Australia (and arguably other western nations), where landscapes, ecologies and knowledge are ‘ailing or out of balance’. In response, he proposes:

Our parlous states need imagination. We need to propose “what if” scenarios that help us account for what has happened in our habitat so that we can then better envisage what might happen. We need to apprehend the past. Otherwise … we won’t be able to divine the continuous tendencies that are making us as they persist out of the past into the present. (Gibson, 2006)

Gibson further links this aim to the restlessness of generative recomposition and its interpretation: ‘It’s the restlessness that’s crucial, the way the artwork … prompts the imagination by artful imbalances and implied possibilities for completion or patterning’ (Gibson, 2006: emphasis in original). For Gibson the computational juxtaposition of fragments is ultimately an engine for human interpretation. Incompleteness and implied pattern drive an active process, a stitching-together and imaginative reconstruction. The same processes are at play in Drifter and Succession. As Gibson argues, the computational scale of generative techniques is not an end in itself, but a means to invite complex understandings of a fragmented terrain.