Images of Architecture, Architecture of Images: Computer Reveals Patterns in Cities

Posted: September 21, 2012 | Author: Jean Mendoza | Filed under: Uncategorized |Leave a comment »

In his famous 1903 essay “The Metropolis and Mental Life,” sociologist Georg Simmel wrote that “with each crossing of the street” the city bombards its inhabitants with “rapid crowding of changing images, the sharp discontinuity in the grasp of a single glance, and the unexpectedness of onrushing impressions.” Simmel’s notion of the city as a chaotic jumble of images resounds to this day. But a new study suggests that if one looks closely enough, architectural features emerge that contribute to a city’s distinctive “look and feel.”

Combining computer science and urban architecture, a team of researchers at Carnegie Mellon University and INRIA/Ecole Normale Supérieure looked to Google to unlock patterns in visually complex places like cities. They developed an algorithm that can automatically detect and analyze Google Street View images of twelve cities around the world. They found that these images revealed stylistic architectural elements unique to those cities, such as Paris’ traditional street signs, balustrade windows, and balcony support as distinct from London’s neoclassical columns, Victorian windows, and cast-iron railings.

This study is one of early attempts to apply data mining to images. Data mining uses computer science and statistics to make sense of large amounts of data (for example, economists track the rise and fall of the stock market and epidemiologists analyze patient charts). Indeed, Simmel discerned early on the need to confront the flood of information we receive every day. The digital age has given rise to more data than we can keep up with. Images like photos and videos, which comprise almost 90 percent of web traffic (thanks to Facebook, Flickr, Youtube and imgur), remain largely untapped. Recognizing the potential opened by Google Street View to study cities, the researchers ventured into a particular brand of visual data mining that they’ve called “computational geo-cultural modeling.” The paper, entitled “What Makes Paris Look Like Paris?” and presented at the SIGGRAPH International Conference on Computer Graphics and Interactive Techniques on August 9, uses Paris as its primary example.

Mining through the virtual landscape of twelve cities for key Parisian elements is like “finding a few needles in a haystack,” the researchers wrote. They collected tens of thousands of Google Street View images and further divided each of those into 25,000 square patches. The researchers wanted to hunt down objects that were both frequent (occur often in Paris) and discriminative (are found only in Paris). For example, trees and cars appear everywhere in Paris, but so do in other cities; the Eiffel Tower is unique to Paris, but since there’s just one, it can tell us little about the city as a whole.

Programmed to recognize objects in images (similar to face detection in cameras and Facebook), the algorithm randomly sampled the patches of images to identify matches, starting with nearby neighbors (images from inside as opposed to outside of Paris). Through repeated sorting, the algorithm was able to build clusters of similar patches by filtering out uninteresting images like sidewalks or the sky found in all cities until elements common but unique to Paris remain.

The researchers found that it is not landmarks like the Eiffel Tower and Arc de Triomphe, but ordinary street signs, window railings, balcony supports, lampposts, and doors that best characterize the city. The traditional blue/green/white signs and special style of lampposts that mark Parisian streets are difficult to find anywhere else. “The visual minutiae of daily urban life,” as the researchers put it, that often escape us may actually define our surroundings.

The program can also spot even subtler patterns. For instance, balcony railings are commonly found in main boulevards, while window railings dot the smaller side streets of Paris. And while arch-supporting columns have made Place des Vosges famous, they also appear in the more recent Marché Saint-Germain.

Similarities across cities also emerged, possibly indicating cross-cultural exchange. In the five European cities studied (Paris, Barcelona, Milan, London, and Prague), double arches are found everywhere except for London. Cast-iron balcony railings also appear frequently in Paris, Barcelona, and Milan, while railings in London and Prague are made of stone.

The algorithm had greater trouble characterizing American cities, mostly coming up with cars and road tunnels. The researchers suspect this failure is due to the “melting pot” of architectural styles and the dominance of cars in America.

This new tool may prove particularly useful for computer graphic modelers for films like Pixar’s Ratatouille, which needed to recreate Paris in an animated format. Beyond cities, this technology can potentially seek out patterns in nature, like fields and rivers, and even in home products, like cars and electronics, said the researchers. Can this tool also tap into Google Sky or Google Art Project to uncover patterns in the constellations or in paintings? The possibilities seem enticing.

So is the city, like Simmel once claimed, an onslaught of senseless images? Visual data miners say no. With further advances in this technology, we may perceive and understand our surroundings in a new light.

---