Throbber: Executing Micro-temporal Streams

Loading webpages, waiting for social media feeds, streaming music, movies and other multimedia content are mundane activities in contemporary culture. Such mundane activity includes the involvement of network-connected devices from fixed desktop computers to portable tablets and smart watches, all involving data transmission and distribution across multiple sites—referred to as data. In these scenarios, data, according to David Berry, Wendy Chun and Matthew Fuller¹ is constantly perceived as a stream, indicating characteristics of vast volume, speed of update, continuous flow and delivery. The concept of streams² now characterises the Internet rather than web pages. Data streams indicate events that are regarded as the latest and instantaneous versions. In this way, the now that we experience through perceptible streams is entangled with computational logics.

Users usually encounter a distinctive spinning icon during the loading, waiting and streaming of data content. From a user perspective, this spinning icon represents an unstable streaming of the now. A graphical animation known as throbber indicates to users that something is loading and in-progress, but nothing more. A similar yet very different form of process indicator, such as a progress bar, expresses even more information than a throbber. In contrast to a progress bar, which is more linear in form, a throbber does not indicate any completed or finished status and progress. It does not explain any processual tasks in any specific detail. Previously such computer operations were commonly explained in conjunction with the use of a progress bar instead, as for example, with statements about the transferring and copying of specific files and directories, or illustrating installation procedures. A throbber usually appears in a graphical format, sometimes only with the additional word ‘loading.’ All that is presented is a spinning icon, perceived as repeatedly spinning under constant speed as well as hinting at invisible background activities for an indeterminate and unforeseeable timespan. If one looks up the dictionary definition of the verb ‘throb’³ it is defined as a strong and regular pulse rhythm that resonates with a throbber’s design and in regards to how it performs on the Internet today. But such design could be seen as an over-simplification of the micro-operations of networked technology, making one believe that the network is working with a certain regularity and that all data are queuing underway, thus rendering the network conditions of the now.

This article investigates data processing that takes place behind a throbber that may not be visible to users. Although a throbber is normally expressed in a visual format, the underlying processes are specifically technical, operational and invisible. Through speculating various design and execution of a throbber, as well as catching the network packets and close reading the specification of protocols, I have programmed a throbber with same usual graphical elements but each ellipse runs in a different tempo that is real-time responded to the network packets arrival, resulting in an unpredictable spinning speed and ellipses’ visibility (Figure 1 and 2). The project called The Spinning Wheel of Life ⁴ explores the computational and operational logics behind a running throbber. This article illustrates how a cultural and operative reading of an abstracted form of throbber allows an examination of data streams in contemporary computational culture.

A cultural reading of a throbber

With its distinct design characteristic of a spinning behaviour hinting at background and networked processing, the throbber icon acts as an interface between computational processes and visual communication. One of the earliest uses of the throbber can be found in the menu bar of a Mosaic web browser from the early 1990s which was developed by the National Center for Supercomputing Applications (NCSA), ⁵ with the browser interface designed by scientist Colleen Bushell. This throbber ⁶contains a letter ‘S’ and a globe that spins when loading a web page. This kind of spinning throbber, with the company’s graphical logo, can also be witnessed in subsequent software browsers, in which they are usually located on the top right corner of the browser’s menu bar, such as Netscape and Internet Explorer. While the throbber spins it visually indicates actions are in progress. These actions, from a user point of view, could be interpreted as the loading of web data or connecting to a website by a software browser. From a technical perspective it involves Internet data transmission and a browser that renders the code. The spinning behaviour stops when a webpage has finished loading. A web browser is software able to render and display requested content, making network calls and requests and storing data locally. In this respect, the spinning throbber concerns the complex entanglement of code under network conditions. A throbber, with its spinning characteristic, can therefore be said to be rooted in, and specific to, Internet culture.

More recently the throbber icon is no longer only attached to software browsers, appearing also on different web and mobile applications including social media and e-commerce platforms in particular. The contemporary throbber transforms from a spinning logo into a spinning wheel ⁷ that consists of lines or circles that are arranged in radial and circular form, moving in a clockwise direction. A throbber is mostly produced in the format of an animation, loading each frame one by one as most of the throbbers that we see have been preconfigured during the development of the software. The display logic of the throbber image relies on other technical and operative functions ⁸ that have been implemented in the software applications. In other words, the actual handling of data does not have a direct relation with how the throbber is animated visually. Therefore a throbber is animated and spun, or throbbed, at a constant rate, demonstrating a regular tempo. Each individual element of the wheel ⁹ sequentially fades in and out repeatedly to create a sense of animated motion (Figure 3). These spinning wheels appear after a user has triggered an action such as swiping a screen with feeds in order to request the updated information. They also appear after a user has confirmed an online payment or is waiting for a transaction to complete. Perhaps most commonly of all a throbber is seen when a user cannot watch a video clip loading smoothly over an Internet connection. As a result an animated throbber appears as a spinning wheel on a black colour background occupying the whole video screen while the video is buffering.

A throbber represents the speed of network traffic which is also tied to our affective states and perception of time. Emotionally it can be annoying and frustrating to encounter buffering as it involves unknown interruptions (Broida, 2010; Farman, 2017; Stelter, 2011). Things do not flow smoothly and users become impatient when waiting for an unknown period of time or for something yet to come. Taiwanese artist Lai Chih-Sheng exhibited his throbber animation, titled Instant(2013),¹⁰expressing the relationship between waiting and time. This waiting is considered unproductive in that it consumes time. As artist-researcher James Charlton describes it: “It is a gaze that goes beyond the screen to an event not yet here.” ¹¹ The loading time of the throbber appears wasted and unproductive as it is often associated with the perception of slowness of a network. However, waiting, according to Jason Farman, has meaning on its own, which is not only technical but he refers to cultural expectations (2017).

Allowing and implementing Internet fast lanes is against cultural expectations on freedom and equality. On September 10th in 2014 a campaign called “Internet Slowdown day” ¹² was launched as part of the ‘Battle for the Net’ promoting net neutrality. Customised loading icons similar to a throbber were put up on different websites symbolising the potential impact of controlled traffic that would be implemented by Internet Service Providers in the name of increasing profit. The campaign argues for Internet speed equality across all websites and that no unequal conditions, such as fast-lane traffic, should be given to any prioritised website. More than 10,000 corporations such as Etsy, Kickstarter, Netflix, tumblr and Vimeo, showed support by putting up self-designed throbber icons. As is evident in this context the throbber has a significant and symbolic meaning within cultural and political realms.

In contemporary art this cultural icon was remade by artist Aristarkh Chernyshev, who showed the spinning behaviour through customised LEDs in a physical installation. The LEDs formulated the word ‘LOADING,’ circulating in a motion directly reminiscent of a spinning throbber. Chernyshev’s artwork LOADING (2007) ¹³ aimed to present this icon and its data exchange process as a cultural phenomena. Throbber, ¹⁴ as a cultural icon, expresses various dimensions of time—from the loading time of a browser to the regular tempo of a spinning throbber, to the slowness and waiting of Internet network—in understanding data streams.

All the instances above suggest that the smoothness of network traffic is important as it relates to user experience, time perception and the productivity of waiting through daily instances. When watching data streams we may expect things would flow in a timely, smooth and continuous fashion that constitutes liveness. It might be similar to television liveness in which there is a constant flow of “discrete segments” ¹⁵ as well as a flow of connection. This flow of connection means a constant unfolding of reality (Feuer, 1983), or “alive view,” ¹⁶ of the world—a connection between the viewer and the outside world.

The concept of flow ¹⁷was theorised by Raymond Williams in 1974, in which the programming of content, one programme after another, implies continuity to hold their viewers. This concept of flow is fundamental to television studies, suggesting that the technology of television involves institutional plans of programming, the delivery of mediatised representation as well as the viewer experience thereof. As Williams puts it, “[the] phenomenon of planned flow, is then perhaps the defining characteristic of broadcasting, simultaneously as a technology and as a cultural form.¹⁸
Even though there is interruption, what Williams calls the “natural break” ¹⁹ of the advertisement in television, its arrangement is a planned flow with the number of breaks and the corresponding duration as part of the overall television production. Television technology in this respect consists of a relatively continuous and steady flow of programming, which constitutes the immediacy effect that is brought to viewers by the flow of content.

Media studies scholars Stephen Heath and Gillian Skirrow further discuss the temporal dimension of flow in the television context. They argue that a live television programme alludes to an equivalence between creation and transmission, viewing time and time of event. ²⁰ The organisation of different times maintained the immediacy of a television programme through the experience of flow.

However, in the context of digital streams in which the experience of immediacy is concerned, a data stream is organised through computational time with different micro-processes that exhibit highly unstable temporality instead. The interruption of a data stream, such as buffering process that manifests in the form of a throbber, cannot be planned (as with television) insofar as it is subjected to its technical conditions at any moment of time. Additionally, the immediacy of a stream’s narration cannot be simply understood as a planned sequential flow or as discrete segments of programmes. The discreteness of streams is not characterised by content but rather, as I propose in this article, by the very nature of digital and computational processing.

The throbber is widely seen and used in cultural practices but most people in everyday life do not want to see it shows on their screens as it represents slowness and interruption. Application providers present a near perfect connection in which data flows smoothly as streams, or a stream is “fully synchronized” ²¹ with ‘excellent’ performance. Arguably the notion of stream or flow that has been cultivated perhaps makes us forget and become unaware of the materiality of digital networks. The material nature of the network exhibits something that is unpredictable, unstable and discontinuous, which is beyond seemingly ‘natural’ breaks and beyond visible and apparent interruptions. Beyond different cultural instances however, the operative and technical dimensions of a running throbber should not be underestimated, as they can provide a specific perspective for further reflecting on how the experience of nowness is being organised computationally as streams.

In taking into consideration the operative and technical perspectives of network transmission, Florian Sprenger argues that the concept of a stream or a flow is a metaphor. He says:

The network structure of today’s communication channels and of their information stream is often understood as providing a direct connection between users and services or between two communication partners, even though there cannot be any direct connections on digital networks. The metaphor of the flow conceals the fact that, technically, what is taking place is quite the opposite. There is no stream in digital networks. ²²

Sprenger highlights the possible misconception of a flow or a stream, suggesting that there is a gap between the experienced and operative streams. He reminds us that two widely used concepts—flow and stream—in digital media are metaphors. Although metaphors help making sense of concepts and things, we have to consider the capacities and limitations that shape our understanding of computational culture (Markham, 2003). In this case, the metaphors potentially mislead anyone looking to understand the technical and operative processes that take place beneath a ‘stream’.

Micro-temporal analysis

The throbber is a sign of temporal rupture (Jack Self, 2016)

By taking a time perspective in examining the interruptions and ruptures of a throbber, this article draws upon Wolfgang Ernst’s notion of “micro-temporality”²³ that focuses on the detailed technical and operative processes of computation. The micro-temporal approach examines the nature and operation of signals and communications, mathematics, digital computation and dynamic network events within deep internal and operational structures. The added prefix ‘micro’ addresses the micro-operative processes that are not apparent to the immediate human register.

Following Foucault, Ernst’s notion of micro-temporality draws on the concept of discontinuity. ²⁴ In Foucault, discontinuity offers an alternative perspective to understanding knowledge beyond its stable form of narration and representation. Both Foucault and Ernst use discontinuity as a means to examine the gaps, silences and ruptures of things that go beyond signs or representational discourses. The focus on (micro-)temporality in this section is intended to shift gradually from a discursive to a non-discursive discussion of data streams that underlies deep processual micro-temporality.

To bring together concepts of discontinuity and micro-temporality is to offer an alternative perspective in examining data processing beyond a planetary scale. Streams can be understood as highly capitalised and as operating at massive scales under the contemporary conditions of a globalised economy that is disseminates to every part of the world. Alternatively, the notion of discontinuous micro-temporality highlights the micro-processes and gaps that are manifested within the newest presence-oriented feeds and their regular interruptions by a throbber. The concept of discontinuous micro-temporality, arguably, extends our understanding of a seemingly smooth stream and a steady temporal throbber.

Through his micro-temporal analysis Shintaro Miyazaki argues that there is rhythmical quality to algorithms (2012). This, in part, informs an understanding of a throbber beyond a regular tempo. Logically, a throbber is displayed when data is being received, loaded and buffered beyond an acceptable threshold of latency. Once it is below an acceptable threshold, content will be shown in lieu of an animated throbber. Micro-temporality, in the context of streams, can be understood as the variation and latency of data processing. In order to express the rhythms and differences of networked data traffic, I have coded a throbber that was able to display varying speeds. In Figure 4, lines 16 and 25 are the major code that controls the speed of each display of an ellipse.

Instead of using an animated image, I have written code to describe all the visual elements of a throbber, such as position, circles/shapes, colours and rotations, exploring whether there is a possibility for a throbber to perform in a different way and questioning why it has to be displayed in a regular tempo.

To further understand the operational logic behind a throbber, I began to investigate how data is transmitted through and within the machine and its networked architecture, taking a techno-engineering perspective to understand how things operate at an epistemic level. In particular, I engage with engineering concepts such as socket networking, buffering, buffer playback, TCP/IP protocols and packet-switching that are essential to understand the operative and technical processing of streams. The following four sections offer an analysis of digital signal processing, data packets and network protocols, buffer and buffering that lead to the absence of data.

Digital Signal Processing

As opposed to the continuous-time signals of analogue systems, the digital adopts the model of discrete-time with independent variables in signal processing. This model means that each discrete state is countable and measurable according to a distinct value and can be represented by a sequence of numbers. However, the signals are discrete in time, alluding to the value between two discrete-time instances is not defined. Therefore, the flow of data that we experience through a screen is discrete in its nature.

Inside a computer machine a processor’s clock ²⁵fundamentally coordinates the instruction and the processing of data in the form of binary signals. The clock signal can be expressed in a square wave of a clock circuit driven by an oscillating signal that is produced by a quartz crystal oscillator. ²⁶ A clock cycle refers to the amount of time between two pulses of an oscillator and the signal changes from 0 to 1 and then back to 0. Computational time is highly related to speed of processing. In modern 21st Century processors the clock rate is usually measured in Megahertz (MHz) or Gigahertz (GHz), which refers to a clock cycle or clock tick. For example, a clock speed of 5 GHz allows the computer to process 5 thousand million clock cycles/ticks per second. To run the program in Figure 4, for example, it takes less than a second because a computer can execute more than an instruction per second. ²⁷

Following the von Neumann architecture that was first initiated in 1945, John von Neumann designed a computer architecture consisting of a processing unit that contained an arithmetic logic unit, a control unit and a memory unit for performing arithmetical operations, operational sequence control and data and instruction storage. This is also known as a stored-program computer. These units are coordinated by a central clock, ²⁸executing computer instructions in a precise manner.

The appearance and disappearance of a throbber is rendered by code, instructing when a throbber should be displayed on a screen. However, computer instructions are more than source code. In computer science and engineering the Fetch-Execute cycle is used to describe how a Central Processing Unit (CPU) performs code instructions through a series of steps that are executed within clock cycles. A simple calculation like ‘5+6’ includes further micro-instructions such as the copying of the memory location, the storing of the instruction and the individual values of 5 and 6 (in bits pattern format), calculating the sum of the values and writing the calculated result. The high-level instruction breaks into many micro-instructions by fetching and executing values from and in the memory space. The micro-instructions are highly ordered. For example, the values of 5 and 6 must be stored before they can be retrieved for the next process. The instruction pointer (also known as a program counter) is used to keep track of the instruction sequence. This pointer is incremented after fetching an instruction and storing the memory address of the next instruction to be executed. The computer will continue repeating the cycles which fetch instructions and data from memory and then execute them one after another in sequence until the final instruction is reached. ²⁹

This cyclic process of fetching and execution is coordinated by the clock in which one fetch cycle might take more than one clock cycle to complete. The time to finish one fetch cycle is measured in clock cycles. In short, executing code instructions involves the reading and writing of memory (memory is used here in a broad sense that includes the main computer memory, instruction registers and memory buffer registers, etc.), generating a sequence of micro-operational steps and the actual computation. The display of a throbber on a screen is not an exception. All of the code instructions are operated across on/off states, generally known as ‘flip-flops’ and at the level of the quartz-crystal circuit via logic gates used to store and control data flow.

Underneath a throbber is the entanglement of data, source code, micro-instructions and many other entities. The reading and writing of memory is one of the key operations that take place in the CPU. Since the CPU involves multiple units, there are many components involved in the process of a fetch-execute cycle. The clock determines the access of memory. The writing of values in the internal memory can only be done on a clock edge, referring to the oscillating time between 0 to 1 or 1 to 0. It is significantly small when it compares to the whole clock cycle. According to computer scientists David A. Patterson and John L. Hennessy, this is called “an edge-triggered clocking methodology,” alluding to “any values stored in a sequential logic element are updated only on a clock edge”. ³⁰ In other words, the micro-temporality of instructions is driven by the internal clock as there are things that have to be done exactly at a specific time. However, what is important is that if the memory is not properly stored within a designated clock cycle “garbage data” ³¹ will be read instead, causing system failure.

This mismatch between writing and reading is an important concept in understanding what is occurring behind a running throbber. If a stream, a video stream for example, cannot arrive and write on the memory on time it is possible that the CPU will read garbage data. Since there is the logic of buffering as manifested in a throbber display (the detailed logic of buffering will be discussed in the later section), this minimises the reading of garbage data. This section starts to introduce the micro-operations that are happening within a machine. The signal processing suggests that streams operate in the digital nature of discrete signals. Importantly, the machine clock forms a basic infrastructural activity of contemporary technology, organising and maintaining the sequences and components of computation that are essential in performing operational tasks through micro-instructions.

Such micro-processes are extremely difficult to perceive as they involve hidden processes that fall beneath the threshold of human sensory perception. However, invisible events are intertwined with wider cultural forces that are always in process with what we see and how we experience the now. The micro-temporal perspective allows us to be attentive to how time is structured and organised computationally and differently. Precisely, it considers how computational time makes critical decisions to determine the operation of things (the case of clock edges for instance). Therefore, micro-temporality is also about considering time-critical processes. As Ernst puts it, “[time-criticality refers] to a special class of events where exact timing and the temporal momentum is decisive for the processes to take place and succeed at all”. ³² This section begins to unfold the micro-processes of computation to understand the temporal dynamic nature of technical media. The next section will continue to focus on this subject but will discuss technological network transmission that is the core of a stream’s delivery.

Data Packets and Network Protocols

In the late 1960s, ARPANET, the world’s first packet switching network was introduced laying the groundwork which led to the development of the Internet as it has developed today. The concept of ‘packet switching’ was fundamental to understanding how data is organised and flowed. A data stream was chopped into smaller blocks as ‘packets,’ which were then sent via a communications channel in and through different routes, rates and sequences, known as packet switching (Barad, 2002). According to Baran, one of the inventors of the packet switched computer network, real-time connections between sender (transmitting end) and user (receiving end) are an illusion. Instead a sufficiently fast data rate gives only a sense of real-time connection between a sender and receiver. Fundamentally, the routing of a data packet transmits through different sites. Although a selected path is based on “adaptive learning of past traffic,” there are real-time decisions that have to be made to locate the shortest path due to the dynamics of network conditions. In other words, data travels “via highly circuitous paths that could not be determined in advance”. ³³

The Internet includes TCP/IP (Transmission Control Protocol/Internet Protocol), which are currently the major protocols for networked data transmission and entail the massive distribution of data over real-time connections. The Internet backbone stems from these protocols, consisting specific technical standards. Most data streams are transmitted via a TCP/IP connection, ensuring a reliable delivery through two major processes: a 3-way ‘handshake’ and ‘fragmentation’. ³⁴ Generally, establishing a communication channel between two connection points requires the handshaking process. The data packet is based on a sequence of numbers and computational logic (i.e. checksum) to reassemble and reformulate the sequence as the original sender’s message (this is called fragmentation).

A timer is implemented in the protocols of TCP/IP, maintaining a reliable connection by keeping track of packets that are transmitted within a certain time interval. It is an active agent that is implemented across any TCP/IP connection where a stream is operated, in particular to control the processes of network establishment and data transmission between a sender and receiver. Throughout the whole process of data packet transmission, a timer is set to wait for the return of an “acknowledgement” (referred to as ‘ACK’) from the corresponding side and “If the ACK is not received within a timeout interval, the data is retransmitted”. ³⁵ As such, there are always time-checking mechanisms implemented at the level of network protocol. Therefore, one of the important features of a timer is, according to Jonathan Postel who is the editor of the Internet Protocols Specification – RFC 793, to “govern the flow of data between TCPs.” ³⁶

In addition to a timer, there are different waiting times as expressed in a series of states of a connection lifetime: ‘LISTEN,’ ‘SYN-SENT,’ ‘SYN-RECEIVED,’ ‘ESTABLISHED,’ ‘FIN-WAIT1,’ ‘FIN-WAIT2,’ ‘CLOSE-WAIT,’ ‘CLOSING,’ ‘LAST-ACK,’ ‘TIME-WAIT’ and ‘CLOSED.’ Most of these states refer to different waiting times for things to be processed: for example, waiting for a connection request, matching a connection, confirming a connection and terminating a connection request. These states will change before and after certain actions. Using the three-way handshake as an example, if a client computer connects to a particular YouTube video link and requests to view the content a ‘SYNC’ request is sent from A (a client) to B (a server, Youtube) and the connection state at A is changed from ‘LISTEN’ to ‘SYNC-SENT.’ While B receives the request it will respond to A with both ‘SYNC’ and ‘ACK’ status in a message and the connection state for B is changed from ‘LISTEN’ to ‘SYN-RECEIVED.’ Finally A has to respond with an ‘ACK’ status again and then the connection state is now changed from ‘SYNC-SENT’ to ‘ESTABLISHED.’ When B receives the message from A the state changes from ‘SYNC-RECEIVED’ to ‘ESTABLISHED.’ The state of ‘ESTABLISHED’ is regarded as “the normal state for the data transfer phase of the connection”. ³⁷ The actual content, video for instance, is technically called ‘application data’ and can only be delivered in this state. This is the implementation of the three-way handshake process: first to synchronize (with a ‘SYNC’ request), then to acknowledge the receipt of this (with a ‘SYNC’ and ‘ACK’ respond) and finally returning the acknowledgement status (with an ‘ACK’). This process acts as a base, an established connection, and can be thought of a trust relationship that can be built upon to allow further data exchange. This relationship is expressed in human language, such as the greeting messages, ‘Client Hello’ and ‘Server Hello,’ as indicated in Figure 5. The timer implementation is important in building this trust relationship because every state change is required to be done at a certain time interval. This ‘ESTABLISHED’ state is time-critical, whether the application data can be delivered and received is essentially subjected to the prerequisite of this state in which streaming is no exception, where further exchange can be built upon on the basis of a trust relationship.

Beyond the micro-time dimension, packets are also spread across spaces. As Sprenger highlights in his earlier quote, “there cannot be any direct connections on digital networks”.³⁸ This implies that an actual network connection has more than two parties beyond the sender and receiver. According to the Protocol specifications (RFC 793 and RFC 791), ³⁹ there is a field called ‘Time to Live’ (TTL) that limits the lifespan of data within a connection. Data packet routing means that a connection between a sender and receiver that contains multiple switching computers. A route is made up of multiple hops where data passes through these switching computers. TTL is defined as the number of hops that a packet has to pass through before reaching its destination. This also means that if a packet passes through more than a defined number of hops the packet is discarded, alluding to the time to die as opposed to live. Therefore each packet has its own lifespan and its own state of life or death at a particular time.

TTL is also indicated in the video streaming experiment (Figure 7). The usual default value of TTL is 64 and the packet has travelled via 6 switching computers so as to reach the destination (where the value is indicated as 58 in Figure 7). For each switching computer, when it receives a data packet with a TTL value that is greater than 2, it will pass to the next and produce the decrement of 1 from the TTL value. A new TTL value is then sent to the next one. When a switching computer receives a data packet with a TTL value of 1, it means that the data packet has died. This is a form of checking mechanism to ensure the message will not route endlessly. The switching computer (usually in the form of a router) will inform the original sender if the value has exceeded the range. A forwarding machine will send back the message to the previous node as ‘ICMP_TTL_TIME_EXCEED.’ Then the previous machine will traverse with the message ‘ICMP_DEST_UNREACH’ together with the code ‘ICMP_TIME_EXCEED.’ In this backward route each hop will increase the value of TTL by 1 and send to the destination (the original sender). ⁴⁰
The checking mechanism and the decision of backward routing are monitored and executed in real-time.

This real-time execution is similar to what Chun describes within the context of hardware and software systems in which computation responds to the live conditions. She says,

hard and software real-time systems are subject to a ‘real-time’ constraint—that is, they need to respond, in a forced duration, to actions predefined as events. The measure of real time, in computer systems, is its reaction to the live—its liveness. ⁴¹

It is understood that decisions and reactions are required to execute beneath various real-time constraints. From the timer implementation in protocols to TTL handling, Chun’s notion of real-time constraints is further extended to technological networks, the complex connection between machines across distributed spaces. The example of the timer and the field TTL highlight the timely responses, transmission and decisions mechanism within a temporal network. These kind of responses, however, may not include direct human intervention and machines take charge of decisions in real-time and respond in a forced duration. Every decision, the routing decision via multiple hops for example, takes time. Decisions are made not only in real-time but also at micro-temporal intervals. Therefore, every decision made within protocols can be thought of as a micro-interruption instead.

The interruption of a transmission process is further elaborated in Sprenger’s notion of ‘micro-decisions.’ Through the analysis of TTL the role of the protocol is not simply to route and transmit data but it also computes, makes decisions and performs actions according to the value and logic that take place in real-time. As Sprenger puts it,

[there are] constant temporal interruptions, during which decisions are made about the further transmission of packets, and it situates the very stability of the network in these interruptions. ⁴²

He highlights the fact that micro-decisions are made through temporal interruptions. This also implies the notion of real-time is never instantaneous and data is not transmitted via multiple hops simultaneously. Each individual switching computer, situated in different continents, takes part in fulfilling the goal of information reaching the same destination.

These micro-decisions are important to make data transmission possible. Although these are made beyond the human sensory perception, Sprenger reminds us that the so-called real-time transmission is always interrupted even if only a nanosecond of time is required. Therefore, the notion of a real-time connection is never the same as having an instant and direct contact to the connected world. Such a connection is not a direct one between A to B but involves other interactions, such as switching computers, which are spatial as well as temporal. These are the active agents of a connection that can decide a packet’s state. In other words, there are no streams that flow in a smooth and one-way direction, rather there are unplanned micro-interruptions in multiple, or even backward, directions within a network connection. The experience of nowness consists of micro-interruptions that are not apparent to us. Consequently, the running of code executes decisions and produces micro-interruptions that engender the immanent live experience of watching a stream.

Buffer and Buffering

Further to the understanding of the nature of digital signal processing, the fundamental component of clock cycle and the protocols of network transmission, this section focuses on the buffer to elaborate the time-critical matter of buffering, the time when one sees a throbber on a screen.

A buffer is understood as a temporal storage that usually stores a small amount of data in physical memory. As demonstrated in the previous section, every packet segment that is sent via protocols requires the receipt of an acknowledgment, allowing the sender side to know which packet segment has been successfully delivered. This is also a major part of a reliable protocol, meaning the protocol “must recover from data that is damaged, lost, duplicated, or delivered out of order by the Internet communication system”.⁴³ Within a robust data flow control in protocols, the process of fragmentation with the checksum function that is inscribed in data packets makes it possible to reformulate a correct sequence that makes sense of the perceived content. This assembling process involves the use of a temporary buffer. Baran explains as follows:

On the transmitting end, the functions include chopping the data stream into packets, adding housekeeping information and end-to-end error control information to the out-going packets. On the receiving end, each multiplexing station uses terminating buffers temporarily assigned to each end addressee to unscramble the order of the arrived packets, and buffer them so that they come out as an error-free stream, only slightly but not noticeably delayed. ⁴⁴

What is interesting here is the barely noticeable delay time that gives the perception and illusion of a stream. A number of questions arise. How does the protocol synchronise such a delay so that things become unnoticeable? What is flow control (or the end-to-end error control) and how does it enable reliable housekeeping, as in a sense of packets coming out as an error-free stream even though there is packet damage, information which is lost, duplicated or out of order? How do these perpetual and invisible ruptures in packets enable us to conceive of a stream of discontinuity but not a perception of continuity? What is the role of the temporary buffer and its read/write logic, leading to an inscribed stream? Ultimately, what makes the buffer temporal and produces differences and how does it interact?

To address these questions, an investigation of the flow control in TCP architecture is necessary. One of the characteristics of a flow control is that it regulates the amount of data to be sent for each transmission through the concept of the ‘Siding Window Protocol Mechanism.’ Window size refers to the buffer size capacity that indicates the maximum amount of data that can be buffered. This requires notification to a receiver. The sender then can only send the amount of data within the indicated window size. The value of the window size should be decreased every time a new segment of the packet arrives at the receiver. When the value drops to zero it means that the receiver’s buffer size is full and the receiver is not able to handle any additional data at the moment. Unless the receiver has processed the buffered data and sent an ‘ACK’ message back to the sender the window size value will be increased accordingly. Theoretically, all the received segment packets must send an ‘ACK’ message in return. Then a sender can continue to send the remaining data segment. According to scientist Christoph Meinel and scholar Harald Sack, who specialise in Internet systems and technologies, “the receiver regulates this maximum number over the sliding window protocol and adapts this size to fit its processing capacity”. ⁴⁵

To explain in details Figure 8 illustrates the sliding window at the transmitter side, including a data segment that is “already sent but not acknowledged” and “not yet sent but ready to send.” Once the sender receives the ‘ACK’ for message segment 5-8 segment 9-12 will transmit immediately, hence the sliding window will move to 9 and will cover through 15. In other words, the sliding window moves as acknowledgement arrives, consequently segment 13-15 will be included in the window. The sliding window adjusts the rate of data flow.

Fig 9 illustrates a different perspective of the process of data transmission, focusing on how the amount of data is organized and computed. Assuming the receiver sets a window size W as 750 and the sender sends data segment within 1-500 then receiver sends back an ‘ACK’ message with the value 500, meaning the previous segments (1-500) have been received. At this point the sender sends other segments from 501-750 to the other side. Since it takes time for the receiver to process the segments 501-750 it sends an ‘ACK’ as 750 but window size is set as 0, signaling the buffer capacity is full. It also means that the sender cannot send any further segments as the receiver does not have any additional capacity to process. After the receiver has processed 500 out of 750 segments from the buffer, it sends an ‘ACK’ with the new window value as 500 because the remaining 250 is still under processing. Then the sender sends another segment of 751-1250, once the receiver receives the segments, it immediately reports back with an ‘ACK.’ Again the window value is set to 0 because the buffer is now processing the old 250 segments as well as the new 500 segments with its full capacity. The governing of the flow control takes place within the windows’ restriction and the segment number in the ‘ACK’ field, and these are all determined during live processing. This flow control governs how (many), when and what acceptable data segments can be transmitted. When an error occurs during transmission it can be recovered by TCP via a retransmission mechanism. Data is automatically retransmitted if no ‘ACK’ is received within a certain time interval.

The dynamic size of a window is subjected to the activity of reading and writing of data in buffer memory. Window size is full when the data has filled the buffer memory and is waiting to be processed. Size is not a mere all (full) or nothing (empty) indicator but it can be varied dynamically when buffer data is processed partially or fully. The window size also impacts the counting logic of what to be sent and not to be sent from the sender’s perspective. By referencing the window size from the receiver the counting logic is automatically adjusted. Importantly, such sliding window protocols constitute the mechanisms of data flow control in which there are spatial-temporal assemblages that render a data stream.

In the process of data buffering there are micro-decisions also taken place. From the previously mentioned decisions of locating an efficient path to the movement of the window slice and further in relation to which segments of data to send, all of these decisions are made to control how data is transmitted. Even though it may not seem significant time is lost along the journey. This journey involves interruption at different times and spaces, as Sprenger remarks, “[t]he stream never flows uninterruptedly.” ⁴⁶ This constant interruption constitutes the notion of micro-temporality that is discontinuous and includes decision-making processes, controls and regulations that are programmed at the level of protocol and are inscribed in the stream. When micro-decisions take account of rules in network environments, a stream does not unfold continuously, but rather involves intensive calculable processes.

Buffering is a calculable process in which data is divided into segments and packets. From the articulation above with the processing of window buffer, as well as the respective input and output of the buffer, these involve the activities of storing, reading and processing. It is worth noting that the activities are not acting on the same bit and piece of data. While some data is stored in a buffer, other segments of data are being read and processed. From a system perspective, data is processed at the receiver’s end (as input data in the buffer) and is stored temporarily and locally until the data is further processed by the software application (as output data). This also means that software applications are not required to wait for the entire media file to be downloaded. ‘Just in Time’ (JIT) delivery is used in streaming media, allowing for the playback of partially received data temporarily stored in the client’s buffer. ⁴⁷ In this sense both the playback of buffer data and the receiving of the remaining data can be made simultaneously (and, in addition to the case of video and audio, this is also commonly experienced in loading any relatively large size file, such as a PDF or an image ⁴⁸ within a browser). Software applications, like a browser or media player, have the capability to load or play the partially downloaded data. The buffer is where software applications access the input data and process it as output data. In other words, the processing of data consists not only of the transferring part but rather, as Ernst reminds us, through “a coupling of storage and transfer in realtime.” He continues, “[w]hile we see one part of the video on screen, the next part is already loaded in the background”. ⁴⁹

More precisely, the viewer is not watching the content as data arrives, instead the viewer is watching the processed data that has arrived and is stored in the buffer. This process of temporal storage and playback gives us an understanding of the relation between buffer and stream, in which there is latency between data arrival (from the network), data storage (within internal memory) and data processing (inside a machine). Streaming is essentially about buffering that processes before we can perceive the audio and/or visual content.⁵⁰ A throbber is entangled with this latency, interacting with different pieces of data in different ways.Ideally, the “buffer empties itself at one end just as quickly as it fills up at the other end,” as described by Meinel and Sack.⁵¹ If there is transmission delay that is within a threshold time t, it is regarded as unnoticeable in playback. However, if the delay of the individual segment exceeds the threshold time t a throbber will then display. A program attempts to read and process the buffer but the data hasn’t arrived yet and this gap and rupture will lead to the appearance of a throbber. When this occurs we can perceive and experience discontinuous micro-temporality.

Normally a throbber is seen when loading a big chunk of data and is commonly seen in video sites mostly due to the instability or low bandwidth of a network which causes a delay in the arrivial of a data segment (i.e. it exceeds the threshold time t). Buffering is highly related to time as it allows different rates to occur simultaneously, decoupling “time dependencies”⁵² between the input and output of data. As a result, data can be consumed and processed at a different rate by program applications. Data, in the case of streaming, is actively and constantly being stored (written) and removed (read) in the buffer at different speeds and rhythms, oscillating between the invisible and visible.

The absence of data

Proceeding from the operative logic of streaming this article has demonstrated that there are calculable processes, data transmissions and that the reading and writing of the buffer occurs at different rates. The operative logic is built into the infrastructure of networked technology. What has been written in the buffer will be automatically read and processed through running code. However, technology does not guarantee that all the data is written in the buffer. Dropped frames (frames of video that are dropped during playback) are a relatively common experience in real-time communications and video streaming. Dropped frames impact upon the user’s viewing experience because frames disappear within a perceivable continuous stream. When an audio-visual is played back at the receiver’s side this introduces gaps in the stream and it produces glitches or jittery audible effects. This is different from displaying a throbber on a screen, where nothing can be seen on a screen despite the animated graphic. When experiencing dropped frames, one can still see or hear something but just not necessarily in good quality.

In some situations the issue of dropped frames is seamless because it does not create significant quality degradation. Such visible and invisible dropped frames are caused by packet loss, the absence of certain parts of data during data transmission across nodes and routers. Time lost, as mentioned above, includes micro-decision making as well as interruptions and delays. Indeed, packet loss is highly relevant to the notion of micro-temporality. The delay time for transmitting data not only includes ‘store-and-forward’ in each buffer nodes but also ‘queuing delays’ that are subjected to network congestion and are not predictable in advance.⁵³ Packets are required to queue up and wait for the transfer while the network is congested. Under streaming conditions, data is constantly transmitted from a sender to receiver across multiple sites. However, the amount of buffer space is limited at each site, which means a newly arriving packet potentially has no space to be stored while the stored packet is still queuing for its next routing. In this situation, “packet loss will occur-either the arriving packet or one of the already-queued packets will be dropped.” ⁵⁴

The robust design of network protocols consists of an automatic mechanism to detect and trigger retransmission for packet loss. However, for real-time conversational applications and media streaming platforms such as Skype and YouTube delay time for each packet is a critical issue as the transmission is required to be continuous. Both conversations and live concerts are unceasing. On the one hand the absence of data is crucial as packet loss is related to the degradation of quality and it could immediately impact the visual or audio quality. On the other hand, if data arrives with significant delay the application design at the receiver’s end is then required to determine if such data will still make sense in playback, in particular where conversation and data are constantly played-back as a stream. In deciding whether the data should be played-back or ignored, acceptable latency becomes a decision that is inscribed in the software and platform design.

Serious data loss may even result in the automatic termination of a connection—which also means the tolerance is unacceptable from the point of view of software design. The technical consequences of data loss is nothing new. If one has used Skype or other communication applications like weChat it is not uncommon to have the experience of glitches or jitter effects or a throbber displayed on a screen. What is of concern here is rather the cultural implications of these absence of data, or the potentiality of packet loss.

The absent data requires our attention. Firstly, the absence of data might be caused by a voluntary condition. It is possible for an application to discard late-arriving data that are within acceptable latency because it is insignificant to the entire user experience. Secondly, due to buffer capacity, data loss can occur anytime and at any site during the entire journey of a data transmission. Last but not least, when the network bandwidth cannot match the application’s processing rate there will be data loss. For example, a 50% data loss is encountered when a network has only a maximum bandwidth of 5 Mbps and the application requires 10 Mbps. When there is insufficient capacity to handle different rates at the receiver’s end, and hence, data loss will occur. ⁵⁵

As a result not all data is treated equally and able to arrive at the destination and take a perceptible form. Even though the presence of a stream is mediatised as audio and visuals through a screen there is still a possible absence of data. The absence of data, although it cannot be mediated in a perceptible form at the receiver’s end, is somehow interwoven into the presence of a stream in which a conversation or video playback is kept running. The point is that the mundane activity that we experience when we load, wait and stream through a screen is loaded with unperceivable gaps.

The now that we watched and experienced is conditioned by the absences of data and it is infrastructural specific and non-discursive, stemming from the materiality of the distributed network that is running dynamically. According to John Law and Vicky Singleton, when one experiences a reality, “whatever the form of its presence, this also implies a set of absences.” Their notion of presence and absence, as in this article, are understood as an entanglement not as a separate concept, in which “sets of present dynamics generated in, and generative of, realities that are necessarily absent.” ⁵⁶

Similarly, when a stream of data is sent via a distributed Internet network the logic of buffering and data processing are constantly performed through the presence and absence of data. A display of a throbber presents another reality, a reality that is conflated with an invisible material infrastructure and interruptions as well as the absence of material substrates. They are both micro-temporal as I have already demonstrated. Furthermore, a throbber and its underlying logic of data buffering involve discrete-time signaling—the milliseconds of time lost and the absence of data—presenting multiple realities which lie at the heart of time-dependent logics. Therefore, reality is not only a matter of continuous flow and the immediacy of a stream. Taking account of materiality such a notion of reality neither refers to the symbolic meaning of content, nor the feeling of presence, nor immediate data delivery but rather to a tension that is expressed between continuity and discontinuity through the performativity of code.

This is to say that when taking into account packet loss, the nowness of a stream is also about an absent present. In this article, the use of discontinuous micro-temporality explicates the invisibility of computational culture by shifting our attention from the cultural understanding of a throbber and what is visible on a screen to invisible micro-events that are running in the background; events are not separated but entangled as absent/present.

Absent data is rarely mentioned in the commercial products that frame contemporary digital culture inasmuch as it possibly relates to quality degradation or may be regarded as unnoticeable. Technically there are some parts of a stream not necessarily perceptible even though data is continuously being sent. Within a stream there are these discontinuous forces constituting a continuous presence. Sometimes the forces seem to be strong but other times they are weak; in some case they are more visible and at other times they are unnoticeable. The notion of discontinuity pays attention to the gaps, ruptures and pauses that are interwoven within the continuous flow of a data stream. From the display of a running throbber to its disappearance while a stream is presented, discontinuous micro-temporality highlights the forces and presence of micro-decisions and micro-interruptions that reconfigure the liveness of a stream and how we experience nowness. Speculating and working through the visual coded icon of a throbber allows reflection on perpetually changing cultural and social conditions. On the one hand, the existence of a throbber is a by-product of a commercial application that informs users to wait for an unknown period of time. On the other hand, through the use of a throbber in developing various data query services, such as live streaming, big data analysis, social media platforms, data predictions and transactional applications, this cultural icon offers a critical space for speculating and reflecting on how time is being organised and how the now is presented and made operative. A throbber is a cultural phenomenon that appears in almost every application that operates within a live computational environment. It is not only a technical or visual object but is also entangled with other cultural, technical and operative processes that render the unknowable more knowable.

Acknowledgements

This is a much-extended version of previously published articles entitled ‘Microtemporality: At The Time When Loading-in-progress’ (Soon, 2016) and ‘Executing Micro-temporalities’ (Soon, 2017), as well as the updated version of my PhD thesis chapter titled ‘Executing Micro-temporal Streams’ (Soon, 2016). Its conceptualisation developed through seminars and conferences on ‘execution’ in 2015 and 2016. I am thankful to the team of Critical Software Thing, respondents and participants in giving valuable feedback. Thanks are due to Geoff Cox, Christian Ulrik Andersen, Jane Prophet, Femke Snelting, especially Helen Pritchard, Eric Snodgrass and Magdalena Tyżlik-Carver who have edited the short version of this article in their book titled Executing Practices.

References

Albers, M. C. (1996). Auditory cues for browsing, surfing, and navigating. Proceedings of the 3rd International Conference on Auditory Display (ICAD 1996), Palo Alto, California.

Baran, P. (2002). The beginnings of packet switching: Some underlying concepts. IEEE Communications Magazine, 40(7), 42-48.

Berry, D. M. (2011). The Philosophy of Software: Code and Mediation in the Digital Age. Basingstoke: Palgrave Macmillan.

Berry, D. M. (2012). The Social Epistemologies of Software. Social Epistemology, 26(3-4), 379-398.

Berry, D. M. (2013). Introduction: What is Code and Software? Life in Code and Software: Mediated Life in a Complex Computational Ecology. Open Humanities Press.

Broida, R. (2010, July 14). Stop Frustrating Pauses in YouTube Videos. PCWorld. Retrieved from http://www.pcworld.com/article/201089/Stop_Frustrating_Pauses_in_YouTube_Videos.html

Burrell, M. (2004). Fundamentals of Computer Architecture. New York: Palgrave Macmillan.

Chun, W. H. K. (2008). On “Sourcery,” or Code as Fetish. Configurations, 16(3), 299-324.

Chun, W. H. K. (2017). Updating to Remain the Same: Habitual New Media. MIT Press.

Claypool, M., & Riedl, J. (1998). End-to-End Quality in Multimedia Applications. In B. Furht (Ed.), Handbook of Multimedia Computing. Boca Raton, London.

Ellis, J. (1992/[1982]). Visible Fictions: Cinema: Television: Video (2 ed.). London, New York: Routledge.

Ernst, W. (2006). Dis/continuities: Does the Archive Become Metaphorical in Multi-Media Space? In W. H. K. Chun & T. Keenan (Eds.), New media, old media : a history and theory reader. New York ; London: Routledge.

Ernst, W. (2013). Media Archaeology: Method and Machine versus History and Narrative of Media.Minneapolis: University of Minnesota Press.

Ernst, W. (2013b) Ernst on Time-Critical Media: A mini-interview/Interviewer: J. Parikka. Machinology. Retrieved from https://jussiparikka.net/2013/03/18/ernst-on-microtemporality-a-mini-interview/

Farman, J. 2017. ‘Fidget Spinners’. REAL Life, Available at: http://reallifemag.com/fidget-spinners/ Accessed February 15, 2018.

Feuer, J. (1983). The Concept of Live Television: Ontology as Ideology. In E. A. Kaplan (Ed.), Regarding Television: Critical Approaches (pp. 12-22). Washington, DC: University Press of America.

Foucault, M. (1972). The Archaeology of Knowledge and the Discourse on Language. New York: Pantheon Books.

Frabetti, F. (2015). Software Theory : A Cultural and Philosophical study. London: Rowman & Littlefield International.

Fuller, M. (2003). Behind the blip : essays on the culture of software. New York, London: Autonomedia;Pluto.

Galloway, A. R. (2004). Protocol : how control exists after decentralization. Cambridge: MIT Press.

Health, S., & Skirrow, G. (1977). Television: A World in Action. Screen, 18(2), 7-59.

Hyde, R. (2004). Write great code Volume 1 : understanding the machine. San Francisco: No Starch Press.

Kurose, J. F., & Ross, K. W. (2013). Computer Networking: A Top-Down Approach. Pearson Education.

Knuth, D. (1989) 1991. “Theory and Practice.” Theoretical Computer Science 90 (1): 1-15.

Laplante, P. A. (2000). Dictionary of Computer Science, Engineering and Technology. CRC Press.

Law, J., & Singleton, V. (2005). Object Lessons. Organization, 12(3), 331-355.

 

Markham, A, 2003. ‘Metaphors of Internet: Tool, Place, Way of Being’. Paper presented at the conference of the International Association of Internet Researchers in Toronto, Canada, October 2003. Available at: http://markham.internetinquiry.org/writing/MarkhamTPW.pdf

 

Meinel, C., & Sack, H. (2013). Internetworking: Technological Foundations and Applications. Berlin: Springer.

Miyazaki, S. (2012). Algorhythmics: Understanding Micro-temporality in Computational Cultures. Computational Culture, (2). Retrieved from http://computationalculture.net/article/algorhythmics-understanding-micro-temporality-in-computational-cultures

Patterson, D. A., & Hennessy, J. L. (2007). Computer Organization and Design: The Hardware/Software Interface (3 ed.). Burlington: Morgan Kaufmann.

Pereira, F., & Ebrahimi, T. (2002). The MPEG-4 Book. Prentice Hall.

Postel, J. (1981). Transmission Control Protocol – Darpa Internet Program Protocol Specification (RFC 791). [Specification]. Retrieved from Information Sciences Institute: https://tools.ietf.org/html/rfc793

Postel, J. (1981b). Internet Protocol – Darpa Internet Program Protocol Specification (RFC 793) [Specification]. Retrieved from Information Sciences Institute: https://tools.ietf.org/html/rfc791

Roebuck, K. (2011). Virtual Desktops : High-impact Strategies – What You Need to Know: Definitions, Adoptions, Impact, Benefits, Maturity, Vendors. Dayboro: Emereo Publishing, 348-349.

Rosen, R. (2014). Internet Control Message Protocol (ICMP) Linux Kernel Networking: Implementation and Theory (pp. 37-61). Berkeley, CA: Apress.

Self, J. 2016. Beyond the Self. E-flux architecture. 2016. Available at: http://www.e-flux.com/architecture/superhumanity/68658/beyond-the-self/ Accessed February 15, 2018.

Sprenger, F. (2015). The Politics of Micro-Decisions: Edward Snowden, Net Neutrality, and the Architectures of the Internet. Lüneburg: meson press.

Stelter, B. (2011). Debate Web Stream Does Not Flow Smoothly for All [Blog post]. Retrieved from http://thecaucus.blogs.nytimes.com/2011/10/11/debate-web-stream-does-not-flow-smoothly-for-all/?_r=1.

Von Neumann, J. (1945). First Draft of a Report on the EDVAC (Contract No. W-670-ORD-4926)[Report]. Retrieved from http://www.wiley.com/legacy/wileychi/wang_archi/supp/appendix_a.pdf

Williams, R. (1974).Television : technology and cultural form. London: Fontana, Collins.

Notes