A Scientific American article brought mathematical knitting to my attention once again, and within the article was a link to Bridges, an organization which oversees the annual Bridges conference that explores the connections between art and mathematics. Following the link to their 2013 Conference, I found their Short Movie Festival. I’ve watched a number of these short films whose actions explore things like topological objects, fractals, geometric symmetries, and groups. These ideas are presented in visual works, movement performances, as well as in nature. I thoroughly enjoyed all of them and I found myself asking the question, “So where does mathematics live, really?” For example, associated with the short movie Dancing Braids by Ester Dalvit is the following note:
Braids can be described as configuration space of points in a disc. These can be visualized as dances: the positions of each dancer are translated into a strand of the braid, the time into a spatial dimension.
This movie is a small part of a long video about braid theory which is available here.
Or with Susan Gerofsky’s film, The Geometry of Longsword Locks, is this:
In traditional English longsword dancing, a team of dancers makes intricate moves while joined together by their wooden or metal ‘swords’. An impressive element of the dance is the variety of traditional geometric, symmetrical sword locks (often stars) created through the movements of all the dancers. The film showcases a longsword dance and the locks created by the physical algorithms of the conjoined dancers’ movement. After showing the dance, questions are offered to spark mathematical explorations by secondary or post-secondary students. These questions include topological and geometric ideas about crossings, angles and edges, and logic-related questions about categorizing lock types and discovering whether new locks could be created through analysis of the physical algorithms that create them. Slow-motion and repeated views help learners explore this rich source of geometry.
On the Simons Foundation website is yet another short video on Change Ringing.
The art or “exercise” of change ringing is a kind of mathematical team sport dating from the 1600s. It originated in England but now is found all over the world. A band of ringers plays long sequences of permutations on a set of peal bells. Understanding the patterns so they can be played quickly from memory is an exact mental exercise which takes months for ringers to perfect. Composers of new sequences must understand the combinatorics of permutations, the physical constraints of heavy bells, and the long history of the art and its specialized vocabulary. Change ringing is a little-known but surprisingly rich and beautiful acoustical application of mathematics.
According to The North American Guild of Change Ringers,
the earliest record we have of these is from 1668:Tintinnalogia: or, The Art of Ringing. Wherein Is laid down plain and easie Rules for Ringing all sorts of Plain Changes. Together with Directions for Pricking and Ringing all Cross Peals; with a full Discovery of the Mystery and Grounds of each Peal.
Perhaps we can ignore the effect of the subject tabs we learned to put in our notebooks when we were young and ask some new questions. Do these visual and musical experiences represent mathematical concepts or are mathematical concepts actually exploring the elements of these visual and musical experiences? I lean in the direction of the latter. In fact, I would argue that one of the major functions of the brain is to integrate experience. The dances shown in two of the short films are, in some sense, an impulsive integration of the things we hear, see and hold, that become shapes within the inherent unity of our experience. It can be said that mathematics ‘picks up’ on this impulse, and further explores that unity by investigating the paths that are born of these more impulsive harmonies. Mathematics is then distinguished by its symbolic representation of the flow of patterns created by our living – by the visual, and audio structures that the senses build, as well as the cognitive structures that develop with them. Braids and knots are two of the oldest human impulses to create new experience, and they are two of newest objects investigated by mathematics, which then further integrates them into what we know of number and quantity and symmetry.
There is one more thing, not so much related to the theme of this post, but worth a look. One of the short movies in the Bridges short movie festival is a poetic approach to the words real and complex that I think is really nicely done. You can go to it directly here.
Back in September, 1992 Semir Zeki wrote an article for what was then a special issue of Scientific American called Mind and Brain. In it he described what was known about how the brain produces visual images. I have referred back to the article many times because it highlights the philosophical implications of our current grasp of these processes. Right below the title of the article was this remark:
In analyzing the distinct attributes of images, the brain invents a visual world.
Near the end he makes an important observation:
The past two decades have brought neurologists many marvelous discoveries about the visual brain. Moreover, they have led to a powerful conceptual change in our view of what the visual brain does and how it accomplishes its functions. It is no longer possible to divide the process of seeing from that of understanding, as neurologists once imagined, nor is it possible to separate the acquisition of visual knowledge from consciousness. Indeed, consciousness is a property of the complex neural apparatus that the brain has developed to acquire knowledge.
Zeki’s investigation of the visual brain has lead to a significant amount of work on the neurobiology of aesthetics. He heads the Institute of Neuroaesthetics at University College London. VisLab, The Artificial Vision and Intelligent Systems Laboratory at the University of Parma, Italy, has contributed to the institute’s work. Within an introduction to the institute’s purpose, and with respect to Vislab in particular, there is the following statement:
Over the past few years Vislab has contributed to neuroesthetics by exploring visual art in relation to the known physiology of the visual brain.
Underlying the approach are three suppositions:
• that all visual art must obey the laws of the visual brain, whether in conception or in execution or in appreciation;
• that visual art has an overall function which is an extension of the function of the visual brain, to acquire knowledge;
• that artists are, in a sense, neurologists who study the capacities of the visual brain with techniques that are unique to them.
Very recently, Zeki co-authored a paper on The experience of mathematical beauty and its neural correlates. Neuroscientist, John Paul Romaya; physicist, Dionigi M. T. Benincasa; and mathematician, Michael Atiyah, were his co-authors. The paper was published in the journal Frontiers on February 13. Their study, was aimed at determining whether the beauty experienced in mathematics correlates with activity in the same part of the emotional brain (referred to as field A1 of the medial orbito-frontal cortex or mOFC) as the beauty derived from sensory or perceptually-based sources like visual art and music. Their results showed that mathematical beauty was correlated with activity in this part of the emotional brain, which raises some interesting questions related what our experience of beauty is all about.
Unlike studies that looked at the neurobiology of musical or visual beauty, this study required the recruitment of individuals with a fairly advanced knowledge of mathematics. And so, while it may be difficult to sort out, this effort, the author’s suggest,
…carried with it the promise of addressing a broader issue with implications for future studies of the neurobiology of beauty, namely the extent to which the experience of beauty is bound to that of “understanding.”
The study included 12 non-mathematical subjects, but the majority of these individuals indicated that they didn’t understand the equations and that they didn’t have an emotional response to an equation they may have found beautiful (despite the fact that some did rate particular equations as beautiful). Researchers were able to parse the components of the non-mathematicians’ judgment to some extent. Finding more intense activity in the brain’s visual areas for these subjects, confirmed their hunch that the beauty-rating from the non-mathematical participants was a judgment about the formal qualities of the equations – the forms displayed, their symmetries, etc.
The paper becomes even more interesting when the authors consider the implications of their work:
The experience of beauty derived from mathematical formulations represents the most extreme case of the experience of beauty that is dependent on learning and culture. The fact that the experience of mathematical beauty, like the experience of musical and visual beauty, correlates with activity in A1 of mOFC suggests that there is, neurobiologically, an abstract quality to beauty that is independent of culture and learning. But that there was an imperfect correlation between understanding and the experience of beauty and that activity in the mOFC cannot be accounted for by understanding but by the experience of beauty alone, raises issues of profound interest for the future. It leads to the capital question of whether beauty, even in so abstract an area as mathematics, is a pointer to what is true in nature, both within our nature and in the world in which we have evolved. (emphasis added)
And then a quote from a talk given by Paul Dirac in 1939 (one of the subjects of an earlier post of mine), where Dirac advices physicists to look first at promising mathematical ideas, and to consider beauty over simplicity.
There is no logical reason why the (method of mathematical reasoning should make progress in the study of natural phenomena) but one has found in practice that it does work and meets with reasonable success. This must be ascribed to some mathematical quality in Nature, a quality which the casual observer of Nature would not suspect, but which nevertheless plays an important role in Nature’s scheme. . . What makes the theory of relativity so acceptable to physicists in spite of its going against the principle of simplicity is its great mathematical beauty… The theory of relativity introduced mathematical beauty to an unprecedented extent into the description of Nature. . . We now see that we have to change the principle of simplicity into a principle of mathematical beauty. The research worker, in his efforts to express the fundamental laws of Nature in mathematical form, should strive mainly for mathematical beauty.
What I find very encouraging is that this paper suggests, in yet another way, a coupling of the body with its world that mathematics may yet have a hand in helping to reveal.
The Platonic tradition would emphasize that mathematical formulations are experienced as beautiful because they give insights into the fundamental structure of the universe (see Breitenbach, 2013). For Immanuel Kant, by contrast, the aesthetic experience is as well grounded in our own nature because, for him, “Aesthetic judgments may thus be regarded as expressions of our feeling that something makes sense to us” (Breitenbach, 2013). We believe that what “makes sense” to us is grounded in the workings of our brain, which has evolved within our physical environment…Hence the work we report here, as well as our previous work, highlights further the extent to which even future mathematical formulations may, by being based on beauty, reveal something about our brain on the one hand, and about the extent to which our brain organization reveals something about our universe on the other.
I was intrigued by a paper that came to my attention in the December 2013 issue of Philosophical Studies by Anna Farennikova in which she argues that we ‘see’ absence. In other words, seeing that something is not there is as much a product of our visual system as seeing an object. The example with which she begins goes like this. “If someone steals your laptop at a cafe, you may see its absence from your table.” Can you “see” something that is not present? Farennikova rejects the idea that the visual information is simply the table (without the laptop) and that the absence of the laptop is quickly deduced. She argues, instead, that our visual system includes mechanisms for ‘seeing’ the absence of something, making the case that “in addition to representing objects, perception represents absences of objects.” You might be tempted to say “what difference does it make if I ‘see’ the absence of something or ‘judge’ the absence of something. But Farennikova explains the difference that it makes:
The phenomenon of seeing absence can thus serve as an adequacy-test for a theory of perceptual content. If experiences of absence are possible, then we have another reason (following Siegel) to reject the view that perceptual content is restricted to colors and shapes.
This is a question that addresses what it means to perceive. And this is exactly why it interests me. The argument she builds is one that necessarily considers the variation in sensory experience, particularly in visual experience. There are subtleties in the distinction between sensory experience and higher level cognitive experience. Observing these nuances inevitably leads to a careful evaluation of what it means ‘to perceive,’ which is important to some of the arguments I’ve made about the nature of mathematics.
Farennikova does a fairly thorough job of anticipating her critics. And she is careful to distinguish the phenomenon that she is addressing from other experiences where there is a ‘failure to see.’ She draws attention to the fact that “many experiences of absence feel instantaneous and lacking in conscious effort.” She also points to the strong adaptive advantage of seeing absences.
To survive, we need to be reliably and efficiently informed not only about “what is present in the world, and where it is” (Marr 1982), but also about what is absent from the world and where it is absent. This reliability may require automaticity, which is a function of blocking interference from beliefs and higher cognitive states. If these reasons are correct, then the capacity to sensorily respond to the absence of things should be as primitive and fundamental to humans as the capacity to sensorily respond to the presence of things.
Farennikova’s argument relies on specifying the mechanism involved in experiences of absence, and showing that this mechanism is visual as opposed to cognitive. The model she proposes is a matching operation, where templates that have developed through experience in the visual system are matched with the sensory input of any given moment. These templates of absent objects are not images. They preserve some of the visual attributes of the object but they also hold more abstract information about how the structure of the object is organized. Templates are generated in sensory memory and, she explains, exist at a subpersonal level. They are not necessarily the same as conscious imagery and are not dissimilar to the processing considered commonplace in ordinary vision. Since all of these components are visual, it is reasonable to regard the entire process as visual.
Farennikova then appeals to the rich content view of seeing:
Theories of seeing have been tailored to the perception of material objects, so it is no surprise that absences fail to satisfy their criteria. But what justifies the assignment of genuine seeing only to material objects?
In some visual experiences, some properties other than spatial properties, color, shape, motion, and illumination are represented” (Siegel 2010 The contents of visual experience. Oxford University Press)
I also found a paper that takes issue with Farennikova by Jean-‐Rémy Martin, Université Paris and Jérôme Dokic, Ecole des Hautes Etudes en Sciences Sociales.
While agreeing with Farennikova that absence experiences are not reducible to high-level cognitive states such as beliefs, we reject the Perceptual View. Instead, we claim that absence experiences are neither strictly perceptual nor strictly cognitive. In particular, we propose that these experiences belong to the category of metacognitive (specifically metaperceptual) feelings, which reflect a specific kind of affective experience caused by subpersonal monitoring of (perceptual) processes.
Absence experiences for these authors are metaperceptual feelings of suprise. They argue that “mismatches at the narrow level of templates” are not enough to produce experiences of absence. “Rather, the adaptive function of the experience of incongruity suggest that it must be driven by whole expectations (with templates as proper parts).”
These things may all seem like tedious distinctions, but within this discussion there are actually some intriguing questions, like: When is the work of sensory processing over? When do our conscious minds appear to be at the helm? Is there anything in the middle? I’m inclined to say that even at the advanced level of cognition that we call mathematics, sensory processes are still at work. Not just in the reading of notation or the analyzing of images, but in the content of the mathematics itself, where templates, matches and mismatches are likely moving our minds eye to find out what’s there and what it means. Perhaps there is no cut off between sensory and higher-level cognitive processing. Rather there is some continuum of seeing/understanding whose depths are explored abstractly in mathematics and, in a more immediate way, by language.
Note: Farennilova has put together a series of images related to her research on her website : Seeing Absence
This is something of a follow-up to my last post. I checked out a series of links related to Max Tegmark in the last few days, having heard about the release of his first book Our Mathematical Universe. But I was also motivated by having observed that the latest conference organized by the Foundational Questions Institute (for which Tegmark is one of the directors) included prominent neuroscientists, Christof Koch and Giulio Tononi. This is not the first time a FQXi conference has included neuroscientists among their list of speakers. There are a series of threads that one can follow through Tegmark’s and Tononi’s work, but I would like to make a particular observation. Tegmark’s thesis in Our Mathematical Universe, and Tononi’s strategy in his 2008 paper on ‘Consciousness as Integrated Information,’ each rely on the significance of pure ‘relations,’ in how we analyze our experience as well as in how our experience is produced.
Tegmark has been arguing that the universe itself is a mathematical object or structure. His book is a full treatment of this idea. One of the keys to his defense of this idea is the claim that, as theories in physics have developed, their content has become more and more purely relational. In a 2007 paper that preceded the recent book, Tegmark explains that all of the physical theories that have been produced thus far have two components: mathematical equations and what he calls “baggage,” or the words that we give to the relations when we describe them.
However, could it ever be possible to give a description of the external reality involving no baggage? If so, our description of entities in the external reality and relations between them would have to be completely abstract, forcing any words or other symbols used to denote them to be mere labels with no preconceived meanings whatsoever.
A mathematical structure is precisely this: abstract entities with relations between them.
He then says later:
In other words, our successful theories are not mathematics approximating physics, but mathematics approximating mathematics.
With a more recent paper (Jan. 2014), Tegmark takes on the nature of consciousness. In Consciousness as a State of Matter, he brings principles of physics into a discussion of consciousness. He proposes the possibility that consciousness can be understood as a state of matter, like the states of matter we call a liquid, a solid and a gas and then begins an analysis of the properties that such a state of matter would have. When enormous numbers of particles are brought together, he explains, new and interesting emergent phenomena begin to happen. And while there are a large number of kinds of gasses, there is an independent substrate that they all share. These kinds of ideas can be brought to an analysis of the states of matter that define consciousness as well. One of the properties of memory, for example, is that it has many long-lived stable states. It also has dynamic properties. So the question becomes, can one take the ideas in neuroscience and use them to say something interesting about the physical world? Why do we perceive ourselves, for example, as living in a 3-dimensional space with a hierarchy of objects? How do we get there from the fundamental properties of matter described by modern physics?
Using some of the mathematics that describes physical systems, Tegmark tries to find the way that our experience would emerge from (he actually says “pop out”) of the mathematics. He calls it the ‘physics from scratch problem.’ Tegmark’s paper means to extend Tononi’s work on consciousness to more general physical systems by using information theory and Tononi’s idea of integrated information. He is convinced that the problems of neuroscience and the problems of physics are very strongly linked.
Can a deeper understanding of consciousness breathe new life into the century-old quest to understand the emergence of a classical world from quantum mechanics, and can it even help explain how two Hermitean matrices H and ρ lead to the subjective emergence of time? The quests to better understand the internal reality our mind and the external reality of our universe will hopefully assist one another.
Tononi’s paper finds experience to be the mathematical shape given to integrated information. Information is defined as the reduction of uncertainty. And it is the discrimination among alternatives that generates information. Tononi proposes a way to characterize experience with a geometry that describes informational relationships. The integration of information produces a ‘shape’ in what he calls qualia space, and a particular shape is a particular experience. The ‘space’ is defined using a set of axes each labeled with probabilities related to the states of a system in the brain (like visual systems) and the interactions among elements in the system. When a large number of elements and connections are at play, the dimension of the quailia space far exceeds three. For example, four elements with nine connections among them is a simple system, but it produces a 16-dimensional space. About these shapes Tononi writes that they are
often morphing smoothly into another shape as new informational relationships are specified through its mechanisms entering new states. Of course, we cannot dream of visualizing such shapes as qualia diagrams (we have a hard time with shapes generated by three elements). And yet, from a different perspective, we see and hear such shapes all the time, from the inside, as it were, since such shapes are actually the stuff our dreams are made of— indeed the stuff all experience is made of.
And then there’s this bit of poetry in the paper:
If one accepts these premises, a useful way of thinking about consciousness as a fundamental property is as follows. We are by now used to considering the universe as a vast empty space that contains enormous conglomerations of mass, charge, and energy—giant bright entities (where brightness reflects energy or mass) from planets to stars to galaxies. In this view (that is, in terms of mass, charge, or energy), each of us constitutes an extremely small, dim portion of what exists—indeed, hardly more than a speck of dust.
However, if consciousness (i.e., integrated information) exists as a fundamental property, an equally valid view of the universe is this: a vast empty space that contains mostly nothing, and occasionally just specks of integrated information —mere dust, indeed—even there where the mass-charge–energy perspective reveals huge conglomerates. On the other hand, one small corner of the known universe contains a remarkable concentration of extremely bright entities (where brightness reflects high levels of integrated information), orders of magnitude brighter than anything around them. Each bright “star” is the main complex of an individual human being (and most likely, of individual animals). I argue that such a view is at least as valid as that of a universe dominated by mass, charge, and energy.
In a talk given by Tegmark for the “Philosophy of Cosmology” project, he makes the claim that perhaps physical existence and mathematical existence are the same. The view of mathematics proposed by Tegmark and supported by Tononi seem to reverse the embodiment ideas first presented by George Lakoff and Raphael Nunez in their book, Where Mathematics Comes From. The idea analysis in the Lakoff/Nunez book rests on the claim that mathematical concepts develop, through effective metaphors, from fairly simple, fundamental, physical experience. In Tegmark’s world, at least, the mathematics comes first.
I just saw The Guardian’s Science Weekly podcast for November 11, 2013 which included a discussion with mathematician Edward Frenkel about his new book Love & Math: The Heart of Hidden Reality. I then listened to a Huffington Live segment from January 7 where Max Tegmark and Brian Greene talked about the link between mathematics and reality. Tegmark was speaking from the perspective of his new book, Our Mathematical Universe. I’ve only just ordered both books and so I haven’t read them yet. But I would like to say a few things about how each of the authors introduced their ideas. While I find both works encouraging, both bold attempts to reorient the popular view of mathematics, I’m struck by how different they are.
Edward Frenkel began by addressing the need to bring, to a broad audience, some heart-felt appreciation of the beauty in mathematics. The reason no one can see it, he suggests, is the fault of teachers, and not entirely because the ideas are complex. Equally complex ideas like space-time, quantum mechanical behavior,black holes, the Higgs particle, even DNA have found their way into the popular culture. There is no reason that the real subject of mathematics can’t be made similarly accessible for a popular audience. I very much agree.
Frenkel also spent a fair amount of time talking about the Langlands Program, which he referred to as the effort to find a ‘grand unified’ theory of mathematics. In this discussion, he imagined the different branches of mathematics as continents, fully separated land masses. When one finds (as Langland did) a way to translate questions from one area into questions in another, mathematics, Frankel tells us, becomes a teleportation device. While this work has some obvious pragmatic implications, Frenkel uses this idea to point out that the deeper impact of the Langlands Program is in how it reveals the way things (in a more general sense) are connected. Mathematics, he says, tells us about hidden structure that “we still don’t see.” “The more we know about mathematics,” he continues, “the more tools we will have to understand how the world works.” He likens mathematics to an unfinished jigsaw puzzle that’s giving us glimpses of a hidden reality whose final image we don’t know.
Notice he doesn’t say “the more mathematics we know,” but rather, “the more we know about mathematics.” One of the reasons that mathematics seems dry and uninteresting to so many is that it is rarely thought of as an exploration in itself, a search for new meaning and new possibilities or, as Frenkel puts it, for an image you have not seen before.
Frenkel also makes the argument that everything in our world is migrating to the digital. Three-dimensional printers “will be able to print everything out on demand, like a table and spoon, and so on.” The deepest level of physical reality is becoming a digital layer. In this world, he exclaims, “mathematics is going to be king!” And this is because “mathematics’ function is to order information.”
When asked about how the universe could be mathematical, Max Tegmark begins with an appeal to the progress made in modern physics. While the properties of familiar objects were once reduced by science to the properties of atoms, the properties of elementary particles have now been reduced to numbers. We give names to the numbers like spin and charge, but they’re really just numbers. But the way Tegmark speaks about the mathematical universe still feels like he’s giving that mathematical universe the kind of independent, objective reality which I think hampers the effectiveness of seeing things this way. I completely agree that what he proposes leaves the doors to our future understanding wide open. The only limits that exist would be the limits of our creativity and imagination. But because his mathematics still seems to stand outside of us, the question inevitably arises (and did during this interview) about whether consciousness or emotion will ultimately be described mathematically. He and Brian Greene seem optimistic that the answer to that question is yes. Frenkel, on the other hand, has already overlapped mathematics and love in a film he co-created called Rites of Love and Math.
I find Frenkel’s perspective more familiar. I haven’t yet seen the film nor read the book but I will do both shortly. I’m also interested in the conference, organized by the Foundational Questions Institute, and still underway today. It’s a conference on the Physics of Information, and includes a session on Mind, Brain, information and consciousness with neuroscientists Giulio Tononi and Christof Koch.
My views on mathematics share something with both Frenkel and Tegmark, but I’m also influenced by an idea described by Humberto Maturana and Francisco Zarela in their 1987 work, The Tree of Knowledge. A key to their understanding of cognition is their idea of ‘coupling.’ Structural coupling, in its most fundamental sense, is described as “a history of recurrent interactions leading to the structural congruence between two (or more) systems.” It is possible that we are beginning to glimpse something of our own coupling – the coupling of our conscious mind with the world we inhabit (a world that grows with our thoughts). And that may be why we can’t decide which side the math is coming from.
An article in the December issue of Scientific American gave me a new insect behavior to ponder and one that might reveal, in the insect’s biology, a distant cousin to the mathematical idea we call mapping. It seems that there are insects that have a talent for recognizing faces. Their talent has much in common with our own facial recognition abilities, except for the brain that carries it. It was a young graduate student, researching the social lives of a particular kind of wasp, who first noticed that she could distinguish individual wasps by their facial markings. Elizabeth Tibbetts (now an Associate Professor at the University of Michigan) then wondered if the wasps identified each other by these markings and began research directed at answering that question. What she found was that the wasps did respond to changes in facial markings and did use variation in facial patterns for individual recognition. Tibbetts further investigated this aspect of their behavior by trying to train them to learn to differentiate among patterns other than their own facial markings. The wasps quickly learned to accurately select among wasp faces, but had noticeably more difficulty discriminating among the other images. This is taken as strong evidence that wasps have neural systems specialized for wasp face recognition. Although honeybees do not use face recognition mechanisms in their own daily lives, researchers have successfully trained them to discriminate human faces. And further,
…although the bees learned faces slowly as compared with P. fuscatus wasps and humans, they were able to develop some ability to process faces holistically, even though they are not hardwired to do so, as P. fuscatus wasps and humans are. Second, honeybees were able to learn multiple view points of the same face and interpolate based on this information to recognize novel presentations. For example, after a bee learns the front and side view of a particular face, it will be able to correctly choose a picture of the same face rotated 30 degrees, even if it has not previously seen this particular image.
Inherent to facial recognition mechanisms is some idea of spatial arrangement as well as transformation (when presented with multiple views of the same face). So I searched out what has been understood about our own neural systems and found an article that Carl Zimmer wrote for Discover in January 2011. The article describes a model for understanding how our own facial recognition mechanisms work. The idea was first proposed 25 years ago by Tim Valentine and Vicki Bruce, two psychologists at the University of Nottingham, and it has been consistently gaining support. Zimmer reported on the work of cognitive neuroscientist Marlene Behrmann, at Carnegie Mellon University who, with her colleagues, had made some important observations when they set out to compare the brains of individuals who are face-blind to those who are face-sighted.
Valentine and Bruce argued that our brains do not store a photographic image of every face we see. Instead, they carry out a mathematical transformation of each face, encoding it as a point in a multidimensional “face space.” (emphasis added)
On a map of face space, you might imagine the north-south axis being replaced with a small-mouth- to-wide-mouth axis. But instead of three different dimensions, like the space we’re familiar with, face space may have many dimensions, each representing some important feature of the human face. Just as the ancient cosmos was centered on Earth, Valentine and Bruce argued that the facial universe is centered on the perfectly average face. The farther a face is from this average center, the more extreme it becomes.
This is not just a model for describing what happens. It is understood as a way to characterize how the brain can negotiate the infinite possibilities that emerge from our experience.
…it offers an elegant explanation for how we can store so many images of faces in our heads. By reducing a face to a point—creating a compact code for representing an infinite number of faces—our brains need to store only the distance and direction of that point from the center of face space. Face space also sheds light on the fact that we are more likely to correctly identify distinctive faces than typical ones. In the center of face space, there are lots of fairly average faces. Distinctive faces dwell far away from the crowd, in much lonelier neighborhoods.” (emphasis added)
This is exactly how our consciously defined mathematical ideas can operate in our experience. They corral infinite possibilities into well defined concepts whose behavior can be observed and interpreted. In this way it is an arrangement of ideas, that compacts experience and often extends the reach of cognition – of our ability to see and understand our world.
Another kind of learning is discussed in an excerpt from the book Incognito that also appeared in Discover in October of 2011 under the title Your Brain Knows a Lot More Than You Realize. The author of the piece is neuroscientist David Eagleman. Eagleman highlights learning that can only be accomplished with a kind of trial and error training. He describes what has come to be called chick sexing – the process of quickly determining the sex of chicks that have been hatched.
The Japanese invented a method of sexing chicks known as vent sexing, by which experts could rapidly ascertain the sex of one-day-old hatchlings. Beginning in the 1930s, poultry breeders from around the world traveled to the Zen-Nippon Chick Sexing School in Japan to learn the technique.
The mystery was that no one could explain exactly how it was done. It was somehow based on very subtle visual cues, but the professional sexers could not say what those cues were. They would look at the chick’s rear (where the vent is) and simply seem to know the correct bin to throw it in.
And this is how the professionals taught the student sexers. The master would stand over the apprentice and watch. The student would pick up a chick, examine its rear, and toss it into one bin or the other. The master would give feedback: yes or no. After weeks on end of this activity, the student’s brain was trained to a masterful—albeit unconscious—level.
Eagleman also discussed what he called flexible intelligence saying the following:
One of the most impressive features of brains—and especially human brains—is the flexibility to learn almost any kind of task that comes their way… This flexibility of learning accounts for a large part of what we consider human intelligence. While many animals are properly called intelligent, humans distinguish themselves in that they are so flexibly intelligent, fashioning their neural circuits to match the task at hand. It is for this reason that we can colonize every region on the planet, learn the local language we’re born into, and master skills as diverse as playing the violin, high-jumping, and operating space shuttle cockpits.
And a clear reference to mathematics is made in the article’s introduction:
Eagleman’s theory is epitomized by the deathbed confession of the 19th-century mathematician James Clerk Maxwell, who developed fundamental equations unifying electricity and magnetism. Maxwell declared that “something within him” had made the discoveries; he actually had no idea how he’d achieved his great insights. It is easy to take credit after an idea strikes you, but in fact, neurons in your brain secretly perform an enormous amount of work before inspiration hits. The brain, Eagleman argues, runs its show incognito. Or, as Pink Floyd put it, “There’s someone in my head, but it’s not me.
There are a few things going on here. First, I’m intrigued by the presence of what could be called ‘mappings’ in the biology of a creature so different from us. This suggests the possibility that a mathematics-like mechanism is not just the product of mammalian brain processes. I’m also impressed by the similarity between the face space model for face recognition and the conceptual compression that defines mathematical ideas in general. Finally, I’m always interested in the extent to which our sense making and even our mathematical insight happens outside of our awareness.
I’m planning to post something new this week, but I would also like to share the link to my guest blog for Scientific American that was posted last week. The title of the piece is: To What Extent Do We See With Mathematics?
Hope you enjoy it.
Much of the research done in cognitive science is designed to study the development of concepts – internal representations that define the idea-driven nature of modern human experience. And, in our experience, it’s difficult to mend the rift that’s been created between what we call thought and what we call reality. But a number of paths have opened up within various ‘embodied mind’ theses that are intended to correct the mistaken duality. I’ve covered some of them in previous blogs. I’ll refer here, again, to biologist and philosopher Humberto Maturana, a proponent of a closely related idea known as enactivism. In a paper that appeared in Cybernetics and Human Knowing in 2002, Maturana refuted the idea that language is a collection of abstract representations that correspond to concrete things, and made the following observation:
Part of the difficulty in understanding the relation between language and existence rests on the view of language as a domain of representations and abstractions of entities that pertain to a different concrete domain. Yet language is not so, languaging occurs in the concreteness of the doings of the observer in his or her actual living in the praxis of living itself…Nothing exists outside the networks of conversations through which we bring forth all that exists, from ourselves to the cosmos that makes us possible…
Languaging is action. And this is how I have come to see mathematics, as action.
Despite the development of embodiment theories (one of the most well known being the Lafoff/Nunez book Where Mathematics Comes From) debates over the objective reality of mathematics versus the imaginative reality of mathematics are hardly resolved. But the distinction between these alternatives is blurred when mathematics is seen as action. All of this came to mind today when I read about the recently discovered and oldest known sundial.
A tire size stone, found marking a Bronze Age grave, had been carved out to mark time with the movement of the sun. It was determined that the stone could reflect half hour increments. The burial ground in the Ukraine, where the stone was found, dates back to the 12th or 13th century B.C. An article posted on livescience in early October describes the find and some of the details about the stone.
To verify that the stone was a sundial, archeologist Larisa Vodolazhskaya calculated the angles that would have been created by the sun and shadows at that latitude. She confirmed that the carvings on the slabs marked the hours accurately. “They are made for the geographic latitude at which the sundials were found,” she said. And these ancient carvings rely on a sophisticated grasp of geometry.
The circular depressions, placed in an elliptical pattern, are hour marks of an analemmatic sundial; the largest groove on the plate, Vodolazhskaya said, marks where the vertical, shadow-casting gnomon would have been placed at the winter solstice.
Meanwhile, a long carved line transected by a number of parallel grooves in the center of the slab would have acted as a linear scale for a more traditional horizontal sundial, where the hours are marked by a gnomon’s shadow falling along hour lines. In this case, the horizontal sundial actually had two gnomons, Vodolazhskaya said. One gnomon tracked the time in the morning hours and early afternoon, and the second covered from late morning to evening, measuring time in half-hour increments. Ancient sundials with half-hour marks are rare, though one was discovered earlier this year at the Valley of the Kings in Egypt.
A thorough analysis of the stone can be found in a paper by Vodolazhskaya. The figures in the paper illustrate the impressive precision of the stone’s markings. The sophistication of the geometry employed and the sundial’s ability to measure the passage of time in half hour increments are taken as evidence that ancient Egypt had some influence on the people who inhabited the northern coast of the Black Sea. What strikes me, however, is something that may be unique to an ancient ‘tool’ like this one. In some sense, the intricate geometric structure exhibited by the stone exists only when the stone and the sun are taken together. It emerges from the directed light of the sun together with human perception, thought, and craft. This would be true of any ancient sundial. It just happens that this one made me particularly aware of it, perhaps because of the fine tuning between this one and the latitude of the location where is was found. In the right light, the mathematics isn’t fully isolated from the action of the sun and the sundial maker.
The mathematics isn’t describing the relationship between sunlight, shadow and time, it is the relationship between sunlight, shadow and time. And this is why I thought again about Humberto Maturanna. I think that mathematics, like language, is a kind of shared action, occurring within and among things in the world. It exists in various bodies that may or may not find reason to express it formally.
If you haven’t seen it, the more self-contained mathematical action in the little film Nature by Numbers is worth a look.
The Atlantic Monthly just did an interesting piece on Douglas Hofstadter, Pulitzer Prize-winning author of Gödel, Bach and Escher. Hofstadter’s 1979 book investigates the nature of human thought processes by looking at common themes in the work of the mathematician Gödel, the musician Bach and the artist Escher. In particular, it addresses the question of how meaning is born from apparently ‘meaningless’ elements – like the fundamental biological materials of the body and the brain, or the fundamental symbols of mathematics.
While once seen as groundbreaking work in artificial intelligence, the book is not about how to mechanically replicate human thought, but more about what thought ‘is,’ and the emergence of ‘meaning.’ MIT has offered a course on the text for high school students. Lecture videos and lecture notes for these courses can be found here. Early in one of the lectures, the point is made that the premise of Gödel, Bach and Escher is to show how the “I” of our experience, or the event of self-referencing in our experience, corresponds to the the self referencing that happens within mathematics.
James Somers, author of the Atlantic Monthly piece, focuses on an important shift in how Hofstadter’s work is understood.
Hofstadter seemed poised to become an indelible part of the culture. GEB was not just an influential book, it was a book fully of the future. People called it the bible of artificial intelligence, that nascent field at the intersection of computing, cognitive science, neuroscience, and psychology. Hofstadter’s account of computer programs that weren’t just capable but creative, his road map for uncovering the “secret software structures in our minds,” launched an entire generation of eager young students into AI.
But then AI changed, and Hofstadter didn’t change with it, and for that he all but disappeared.
The contrast between the progression of AI efforts and the aim of Hofstadter’s work is particularly relevant to my interests. Hofstadter’s persistence can be seen in the research of what was once called the Fluid Analogies Research Group or FARG at Indiana University. The group has been renamed The Center for Research on Concepts and Cognition. At the end of their research overview, they make the following statement:
We are staunch believers in the idea of using microworlds to study cognition, and the ideas behind our models are informed by many sources, ranging from biological metaphors (the brain is like an ant colony, thinking is like the parallel activity of enzymes in a single cell), to brain research, to the study of error-making, to the careful study of words and their halos, to the observation of our own smallish acts of creativity in various areas of life, to the study of how analogies have pervaded the greatest creative leaps made by physicists and mathematicians.
Lastly, a tiny comment on the philosophy behind FARG computer models. All of them are based on the idea that thinking is an extremely parallel, emergent phenomenon, as opposed to some kind of set of precise computational rules for manipulating abstract meaning-bearing symbols. In other words, we don’t see thinking as any kind of “logic” or “reasoning”, but as a kind of churning, swarming activity in which thousands (if not millions) of microscopic and myopic entities carry out tiny “subcognitive” acts all at the same time, not knowing of each other’s existence, and often contradicting each other and working at cross-purposes. Out of such a random hubbub comes a kind of collective behavior in which connections are made at many levels of sophistication, and larger and larger perceptual structures are gradually built up under the guidance of “pressures” that have been evoked by the situation. None of this activity is seen as being deterministic; rather, our models are all pervaded by randomness or “stochasticity”, to use a fancier term for the same idea. (emphasis added)
This philosophy is usually traced back to a computer program that Hofstadter built called Jumbo. The program was inspired by his interest in understanding what was happening when one solved a newspaper jumble. He had no interest in the program that would quickly solve them. He wanted to understand what was happening when we solved them. As Somers points out:
He had been watching his mind. “I could feel the letters shifting around in my head, by themselves,” he told me, “just kind of jumping around forming little groups, coming apart, forming new groups—flickering clusters. It wasn’t me manipulating anything. It was just them doing things. They would be trying things themselves.”
The architecture Hofstadter developed to model this automatic letter-play was based on the actions inside a biological cell. Letters are combined and broken apart by different types of “enzymes,” as he says, that jiggle around, glomming on to structures where they find them, kicking reactions into gear. Some enzymes are rearrangers (pang-loss becomes pan-gloss or lang-poss), others are builders (g and h become the cluster gh; jum and ble become jumble), and still others are breakers (ight is broken into it and gh). Each reaction in turn produces others, the population of enzymes at any given moment balancing itself to reflect the state of the jumble.
The key for me is the efficacy of what is here referred to as “subcognitive” acts, acts that have some randomness about them, and may even work at cross-purposes. Hofstadter’s philosophy is also evident in his vast collection of speech errors, or his record of instances of swapped syllables, like “hypodeemic nerdle.” Again Somers points out:
Correct speech isn’t very interesting; it’s like a well executed magic trick – effective because it obscures how it works.
The views on ‘thinking’ explored at the research center are provocative and carry large numbers of implications that will likely impact the age-old mind/body debate. They imagine ‘meaningfulness’ within the life of the whole of nature, of which mathematics is very much a part. I plan to spend a lot more time looking at work being done at the Center for Research on Concepts and Cognition and hopefully more time writing about it.
New Scientist published an article by Amanda Gefter in their August 15 issue which describes how and why the notion of infinity has come into question again. The distinction between a potential infinity (the process of something happening without end), and an actual infinity (represented, for example, by the set of real numbers) was disputed among mathematicians for a long time until Cantor brought new meaning to the nature of infinities in the set theory he created. Now infinities are part of the living tissue of mathematics. But infinities have thwarted the success of some physical theories and continue to do so. The survey of challenges from mathematicians and physicists alike make for an interesting read. One of the more reasonable claims comes from Max Tegmark of MIT.
When quantum mechanics was discovered, we realised that classical mechanics was just an approximation,” he says. “I think another revolution is going to take place, and we’ll see that continuous quantum mechanics is itself just an approximation to some deeper theory, which is totally finite.
I believe Riemann himself had not decided whether space was continuous or discrete.
One of the links within this piece was to another New Scientist article by Gefter, published in October of last year. The older piece had the title: Reality: Is everything made of numbers? In it, Amanda Gefter surveys some of emerging perspectives in physics that equate, in one way or another, mathematical reality with physical reality. She recalls Einstein’s fix for the equations that described an expanding universe, years before there was clear evidence that the equations were correct.
How did Einstein’s equations “know” that the universe was expanding when he did not? If mathematics is nothing more than a language we use to describe the world, an invention of the human brain, how can it possibly churn out anything beyond what we put in? “It is difficult to avoid the impression that a miracle confronts us here,” wrote physicist Eugene Wigner in his classic 1960 paper “The unreasonable effectiveness of mathematics in the natural sciences”
With respect to current physics investigations she says:
The prescience of mathematics seems no less miraculous today. At the Large Hadron Collider at CERN, near Geneva, Switzerland, physicists recently observed the fingerprints of a particle that was arguably discovered 48 years ago lurking in the equations of particle physics.
Gefter then tells us about how some prominent physicists, like Brian Greene and Max Tegmark, are tackling the riddle. She has more to say about Tegmark’s view than Greene’s, and Tegmark’s view is fairly extreme. Gefter quotes him:
I believe that physical existence and mathematical existence are the same, so any structure that exists mathematically is also real.
I’ve always liked this kind of talk. While the content of a remark like this might be difficult to specify, I suspect that it’s full of insight. Tegmark goes on to suggest that the mathematical structures that have no physical application in this universe correspond to other universes. But his ideas rest, in part, on his judgment that mathematical structures don’t exist in space and time. ” Space and time themselves,” he says, “are contained within larger mathematical structures.”
Now, one might argue that brains and computers exist in space and time, and without them there is no mathematics, or at least not the kind we used to thinking about. But I don’t want to move here into a difficult philosophical debate. Instead, I’d like to suggest that if we consider that everything that exists is known only in relationship, some new light might be shed on the question of how mathematics can do what it does. Physicists like Tegmark can seem to be replacing the physical subjects of their investigations with mathematical ones, putting the mathematics ‘out there.’ Cognitive scientists seem to be finding mathematical notions mirrored in sensory and learning processes, suggesting that mathematics is part of how the body learns about its world. But the body’s perceiving mechanisms are built entirely in relation to its surroundings, or more specifically, to the properties of things like air and light, or even gravity. If we can see mathematics as one of the body’s actions, action that stretches the reach of its perceiving and learning mechanisms, then perhaps we can imagine that it develops not ‘for’ the world we live in, or from it, but ‘with’ it. Like color, perhaps.