This is an original guest post by Ricardo Nuno Silva for VC Blog. Ricardo is a Portuguese applications developer with a longtime curiosity concerning the impact of digital technologies in everyday life. You can contact him at rsilva@pobox.com.

-

In the last few years many tools and techniques have been developed to help us visualize songs, music and sounds. This post is a showcase of some of these greatest tools. Each one is focused on a particular aspect of this challenging type of visualization.

One of the most common examples of software for sound visualization is the one used in media players. But they usually only translate sound frequencies to shapes and colors on the screen. They’ve been used extensively for leisure, relaxation and dance parties.

The tools in this showcase have a different approach, as they truly “understand” music in its individual notes. Some can be used in real time, while others need to do some number-crunching while analyzing each song.

Below each image there’s the name of the tool or technique, some great video examples, and a link to the author’s site.

If you know other great tools or videos of music visualization, please leave a comment below or via email. Thank you!

-

Static Visualization of Songs

The Shape of Song by Martin Wattenberg

Narratives 2.0 by Matthias Dittrich

-

Similar to Sheet Music

Don’t Be Sad by Brad Mehldau

MIDI Music Visualization Videos for Deaf and Hearing Impaired People by Eric Rangell.

Music Animation Machine (MAM) by Stephen Malinowski. See: Beethoven 5th Symphony.

-

Visualization of Instruments Output

Clavilux 2000 by Jonas Heuer.

Celeste Motus by the Abstract Birds. (via Pedro Custódio)

MuSA.RT - Music on the Spiral Array. Real-Time by Elaine Chew e Alex François.

-

Lyrics

TypeStar by Scott Garner.

-

Learning Games through Visualization

Synthesia (for piano) by Nicholas Piegdon.

-

Animating Virtual Instruments

MIDIJam by Scott Haag. See: MidiJam (I just died in your arms).

Pipe Dream by Animusic. See also: MIDIJam meets Animusic: Pipe Dream

-

Abstract Visualizations

Ljósið by Ólafur Arnalds

Just Colour by Jesper Brevik

-

See other music-related visualizations @ VisualComplexity.com | Music.

flame dragon, by peter blaskovic (created in flame painter)

As I was organizing my RSS feeds in feedly, I stumbled upon Gert K. Nielsen’s piece on Visual Journalism, written in March 22, 2010.  The venturesome title of Nielsen’s post was “The next big thing in infographics - five criterias and a solution“. Intriguing and stimulating. I was immediately on board. That is until I started reading his five recommendations and final proposed solution. You should read it and take your own conclusions, but I found Nielsen’s piece absolutely bewildering.

1. The first recommendation, on the need for computer generated infographics, reads more like a natural progression of the field rather than a recommendation, and is perhaps the most blunt of the list. The second and third criterias are on the other hand a bit more disconcerting.
2. “It must be beautiful”, Nielsen says in the beginning of his second suggestion. Nothing wrong with that, but you would expect some reflection on the benefits of aesthetics to follow that statement. However, Nielsen appears to be infatuated with aesthetics solely for its popularity… As he explains, “right now the interest is on presentation much more than the content”.
3. But the third criteria is even more baffling. “It has to be somewhat ambiguous”, states Nielsen. Yes, take a deep breath and read it again. And perhaps like me, you’ll wonder, what? But wait, Nielsen immediately comes to our rescue, fundamenting his view with a remarkable argument. “Describing things in black and white and sharp vector lines is too fanatic. Blends are much better suited to describe a complex situation”. Yes, let’s reconsider this fanaticism for objectivity, clarity and content. The future of infographics is ambiguousness!
4. (I didn’t quite understand this point, so if someone does please explain.)
5. Moving on to his fifth criteria, since I couldn’t grasp the fourth, Nielsen asserts “It needs to work in online presentations too”. This could be an interesting starting point to an analysis on the different contexts of use of infographics and the variety of platforms it could explore, but Nielsen falls short in his explanation, merely stating that infograthics could be integrated in online presentations “perhaps by moving or evolving over time”. A very light investigation, to say the least.

But perhaps the most disquieting part of the post was the solution proposed by Nielsen for the future of the field. As he explains: “The solution I came up with is particles in 3D-programs“. Brilliant! According to Nielsen, there’s no particular downside to 3D particles (think about clarity and legibility), apart from its demanding learning curve, or in other words, the time it takes to learn these “really tough concepts”. In his pursuit for ambiguousness it’s not entirely surprising that Nielsen fails to consider any other drawback to his formula. His proposed solution becomes slightly more tangible, when he presents an example of this vision: Flame. As he explains “the ability to paint with ‘flames’ fits right into my expectation of seeing graphics with an appearance that fits the current times”.

I will not expand too much on how I find this view seriously distressing, since I’ve done it before and again. But this leads to the growing confusion that Robert Kosara alludes in his latest post, The Visualization Cargo Cult. Gert Nielsen’s post, as puzzling as it might seem, is a reflection of a seriously disturbing view, that sees objective infographics as a thing of the past, and appealing ambiguousness as a much better fit for the “current times”. I just hope it doesn’t become a contagious meme.

NYT - Obama’s 2011 Budget Proposal: How It’s Spent

Biodiversity Tree

A zoomable treemap for the life records of the Natural Science Museum of Barcelona, by Bestiario.

Most VC readers must already be familiar with Data Flow 2, the most recent number of the growing Data Flow family, published in February 2010. Featuring several interviews with New York Times Graphics Editor Steve Duenes, Art+Com Director Joachim Sauter, and one with Andrew Vande Moere and myself, the book is an inspirational compendium of hundreds of projects. The work presents itself as a portfolio book, featuring an array of innovative approaches (many featured in VC), which are incredibly provocative and inspiring. Due to its coffee-table nature, the title doesn’t aim at an in-depth analysis or theoretical reflection on the displayed projects and defined categories, but acts primarily as a stimulating showcase of ideas.

As Andrew Vande Moere eloquently states in his review, the foreword doesn’t quite align with the book’s content, since most of its assertions for insightfulness are not necessarily substantiated in the variety of executions showcased throughout the book. Nonetheless, Data Flow 2 is a great source of inspiration for anyone working in the domain of data visualization.

A very appealing spline based 3D form in Processing that represents the bass frequency and puts it into motion.As Christian Bannister explains:

What would the bass look like? What would it be like to touch it and manipulate it directly and visually in real-time? These are some of the things I am trying to get at in this sketch.

Unfortunately the Call for Participants is now closed, but nonetheless this initiative should be interesting to follow. Synthetic Aesthetics aims to bring creative practitioners and those who are expert at studying, analyzing and designing the synthetic/natural interface together with the existing synthetic biology community to help with the work of designing, understanding and building the living world.

From this thought-provoking premise:

Biology has become a new material for engineering. From the design of biological circuits made from DNA to the design of entire systems, synthetic biology is very much interested in making biology something that can be designed.

The project asks:

Can collaborations between synthetic biology and design inform and shape the developing field of synthetic biology—the engineering of new and existing biological entities for useful means? What insights can design offer in designing microscopic entities for a human-scale world? Can design learn from synthetic biology?

(Originally written in May, 2009 - Extinct section in VC)

Recursive cycles of innovation happen within many different areas, most notably in the domains of Art and Design. Even those who are completely out of touch with fashion can observe how the field is constantly rediscovering its past, recycling ideas, and incessantly mixing new trends with old influences. In Economics, it’s a well-known fact that business trends and stock markets expose processes that tend to repeat itself in a more or less regular fashion. In this domain, the most notorious study was developed by Nikolai Kondratiev, a Russian economist who observed a series of long 50-year cycles in modern world economy. The Kondratiev waves, as they were later called, consist of alternating periods of high and low sectoral growth, which for most cases have proved to be accurate since the end of the 18^th century. In fact, from the different angles you can look at history, there’s always someone who is ready to point out a specific recursive cyclical pattern.

But even though we are accustomed to this type of process in many fields, we always think technology, and in particular the computing industry, is immune to it. After all, our technological progress is made of several vertiginously rising paths that share the absence of a review mirror. There’s no point in looking back, or even considering that some aspects of the past might re-occur, simply because there’s nothing to learn from it. Nonetheless, the act of uncovering patterns and potential cycles, particularly in an industry that prides itself of its continuous fresh innovation, is an extremely appealing exercise.

The pattern I’m about to describe is divided in 3 periods, starting at the foundation of computing history and ending with a set of strong indicators of a third new cycle. It tries to make the case that even though individual technological components evolve in a really fast and unique pace, the way in which they interrelate and behave might follow some level of cyclical occurrence. Separating these 3 stages are roughly two periods of 25 years. The first cycle started in the late 1950s, with the widespread of the mainframe-computing model, followed by the second stage in the beginning of the 1980s, with a succession of events that lead to the emergence of the highly powerful laptop computer. Finally, the latest and most recent cycle has just started. Lead by Cloud Computing and the Netbook phenomenon, everything seems to indicate this will be a major movement for many years to come. From an initial centralized model, to a dispersion of increasingly independent machines, the new drift foresees storage and computing drainage from many portable computers and the return to a model based on data centrality. The main distinction this time is that instead of the mainframe, the “Cloud” emerges as its central interconnected hub. Although recurring cycles might be a noticeable pattern for how data is stored and accessed, there’s still a unique common thread to all these stages: a continuous straight progression towards mobility.

First Cycle | The Central Mainframe

Characterized by one central computer, responsible for most of the storage and processing power, linked to a series of satellite terminals, the mainframe-computing model has been a key protagonist in the history of the modern computer since the late 1950s. By then people accessed and interacted with immensely large mainframes through a variety of linked terminals that suffered significant changes over time. From early punchcards and teleprinters, to latter video computer displays with their familiar green and amber screens, interactive computer terminals through the 1960s and 70s had one thing in common: its powerless unintelligence and dependency on the crucial mainframe.

Second Cycle | The rise of the laptop and its Portability Effect

By the end of the 1970s, specialized terminals, as the precursors of modern-day portable computers, were becoming smarter. Initially packed with terminal emulation software, these machines were detaching itself from the almighty mainframe and becoming self-sufficient entities with their own processing capability. This process opened the path for the desktop computer, with early pioneers like Apple II and IBM 5150 leading the way. The course of computing mobility had just started and it would be just a matter of time before laptop computers started to materialize and eventually replace desktop computers.

In most part, the computing industry in the past 25 years has seen laptops dramatically increase their computing power and rival traditional desktop PCs. In 1986, battery-powered portable computers had about 2% of market share worldwide[1]. Today there are more laptops than desktops in businesses and general use, and in 2008, more laptops than desktops were sold in the US[2]. Even though some mainframes have evolved into the supercomputers of modern age, uncovering important aspects of science, like the structure of cosmos or the vast neuronal network of the human brain, the true hero of this story is the laptop. These tiny compact boxes have become potent full-fledged machines with the added benefit of portability – an essential attribute in an increasingly mobile world. But how long will the mobile processing power rush last? Or has it in fact reached a tipping point? Will the hero of the last decade be partially or entirely replaced with its new weaker adversary: the Netbook?

Third Cycle | Cloud Computing: The Personal Mainframe

Cloud Computing is seen as the next computing trend and the key driver for the third cycle of data centrality. It can simply be described as a “style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet”[3]. This model is extremely in tune with our contemporary lifestyle. We currently access the web through a variety of devices, with different features, shapes and sizes. And while the amount of access points keeps increasing, the ability to synch content between them is still an immense headache, where rarely (if ever) we see a satisfactory user experience. The Cloud paradigm substantially alleviates this problem, by increasingly relying on services, information and applications stored on online servers – the vast Cloud landscape – that can then be accessed at anytime from anywhere, as long as there is an online connection.

On an enlightening special report by The Economist, entitled “Let it rise”[4], it’s asserted that the Cloud is already a common phenomenon, where 69% of Americans connected to the web use some type of “cloud service” including web-based e-mail or online data storage. Many companies are following this feverish movement and in the same article by The Economist, Irving Wladawsky-Berger compares it to the Cambrian explosion some 500m years ago, when the rate of evolution speeded up, in part because the cell had been perfected and standardised, allowing evolution to build more complex organisms.

Another indicator of this turning point is the Netbook. In part driven by a global economic downturn, the Netbook phenomenon might prove to be a long-lasting craze. Characterized as a lightweight, economical and energy-efficient laptop, especially suited for wireless communication and Internet access, this new mobile computer has been all over the news lately. In a recent article from Newsweek magazine[5], they uncovered a growing market trend in Japan, as more consumers are opting for netbook computers. While PC sales went down 4 percent on the fourth quarter of 2008 in Japan, sales of netbooks shot up 43 percent. The recession has been an important driver for this consumer shift, since people have become more sensitive to price, but the growth of cloud computing is its vital ingredient. There’s also an undeniable rational deduction behind this behavioral change. Many users start questioning themselves if they actually need all the bustling speed and storage, when their computers are mostly used for emailing and web browsing.

On the diagram shown above we can observe a series of laptops and netbooks linked to a central Cloud, which is in turn surrounded by a multiplicity of abstract devices. Many of these future devices will not require a vast processing capability, since they will work as rendering windows for the same online service - flowing incessantly through all of them. The role of “windows for services” might be what awaits many future mobile computers, including mobile phones. This points out to the growing value and significance of online services and applications, as the vital glue across many systems and platforms.

Conclusion

Predictions always feel like empty promises and there can be no certainty on what the future holds. Is the Netbook a predecessor of a future class of dumb terminals entirely dependent on the Cloud? Is Cloud Computing really going to be the next big thing? If so, how long will it last? Will it prove to be a long lasting shift or will people grow increasingly wary of their privacy and lack of ownership and return to a similar model as we have today, and in the process instigate a fourth cycle of data centrality?

In the end of January I was in Paris for a couple of events: a talk at ESAG - École Supérieure de Design, d’Art Graphique et d’Architecture Intérieure and one-week workshop at ENSAD - École Nationale Supérieure des Arts Décoratifs. The lecture at ESAG was great and the lengthy discussion that followed, with the packed audience of inquisitive students, was extremely involving.

The workshop at ENSAD was a longer and very fruitful engagement, part of IDN: Identité numérique mobile (DMI: Digital Mobile Identities), a new program of ENSAD Lab - a research unit for creation and innovation gathering graduate research students and professionals to collaborate and discuss on the contemporary challenges of design. Lead by Remy Bourganel and Etienne Mineu, IDN is meant to investigate the flows, the emerging patterns and representations that qualify a new digital mobile identity. In this context, students at the workshop explored different ways of analyzing and visualizing different social dimensions relevant to them. Some of their initial studies can be seen here.

*UPDATE* - The UK Met Office data was made available today: http://bit.ly/7mWJbx (scroll to the bottom) or direct link: All.zip (3.7MB). Here’s the corresponding station codes per country.

All our eyes are now set in Copenhagen, in what’s in my view one of the most important meetings ever held. Following the overhyped data fraud scandal, which is being targeted by many skeptics as the “Climategate”, the UK Met Office decided to make available the data for more than 1,000 weather stations from across the world, in order to hush divergent voices. The dataset, to be released this week, is the subset of stations evenly distributed across the globe and provides a “fair representation of changes in mean temperature on a global scale over land”, said the Met Office in a statement. “We are confident this subset will show that global average land temperatures have risen over the last 150 years.”

The data has not yet been made public, but once it does I will update this post. In case you cannot wait for this dataset, the group of scientists at RealClimate.org have recently put together a cohesive list of datasources, from innumerous satellites and stations, on sea levels, sea temperature, surface temperature, aerosols, greenhouse gases, and many more. In a blog post announcing the list, the group states:

Much of the discussion in recent days has been motivated by the idea that climate science is somehow unfairly restricting access to raw data upon which scientific conclusions are based. This is a powerful meme and one that has clear resonance far beyond the people who are actually interested in analysing data themselves. However, many of the people raising this issue are not aware of what and how much data is actually available.

This represents a great momentum for all of us involved in Visualization at large to be part of the solution and deliver a clear unequivocal view on what’s happening with our planet. Regardless of how you label your practice, Information Visualization, Data Visualization, Information Design, Visual Analytics, or Information Graphics, this is ultimately a call for everyone dealing with the communication of information for human reasoning. Let’s roll up our sleeves!