Synesthesia, Multimedia & the Caves of Altamira*

Synesthesia and Visual Music Links Symposium on Synesthesia, academic papers: Psyche's Symposium on Synesthesia. International Synaesthesia Association, from Britian. MIT site, was the first WWW page as far as I know. Richard E. Cytowic has finally done a web page, including a subsection on synesthesia. Synaesthesia || Doctor Hugo || Museums of the Mind || Synaesthesia & the Arts, experiment and research, a good overview of resources from Europe. Homepage of the Belgian Branch of the 'International Synaesthesia Association'. Leonardo Electronic Almanac (LEA), is "Leonardo/ISAST and MIT Press' electronic journal dedicated to providing a forum for those who are interested in the realm where art, science and technology converges." Most of the recent articles on music visualization have been published in Leonardo, a journal published by MIT Press. And now there's the Leonardo Synesthesia Special Project, which includes a widerange Leonardo online bibliography of more than 300 entries on synesthesia in arts and sciences. tonecolor, was Greg Jalbert's page on artists who are working on color music (it appears to be retired?). See Greg's program Bliss Paint and his article & links on generative art. Sandy Cohen's BINDU MUSIC VISUALIZATION HOME PAGE -- Sandy teaches a class on music visualization at SFSU. His site offers information about artists who have created visualized music. Some Quicktime samples from the pioneers are available are available from his Lobby site at UC Berkeley. iota is a non-profit arts organization dedicated to preserving and promoting the art of light and movement. The program is connected with artist Larry Cuba, celebrates visual music history to promote the growth of the worldwide community of contemporary practitioners. Based in LA, iota has been developing a library of of materials and a database of artists. There's even a store that carries the historical greats and mailing list. Ron Pellegrino's excellent Visual Music Site, part of the Lightshow WebRing. See his Sites of Music Visualizers., Online Visual Music Forum, excerpts from his history book of visual music, The Electronic Arts of Sound and Light . A VHS tape of his work is available through Graphics Press, Box 430, Cheshire, CT, 06410, USA (Graphics Press also publishes Edward Tufte's books on information design). Also check Ron's view of the state of VM. Stephen Malinowski writes as well as producing the Music Animation Machine. Rhythmic Light has a good tour with lots of references and source material on the history of visual music. Ylem's Art On The Edge, YLEM is a non-profit artists' networking organization centered in the San Francisco Bay Area. YLEM brings its members together six times a year at San Francisco's Exploratorium for forums presented by artists and scientists on contemporary issues. The Ultimate Video Feedback Page has interesting links on video feedback art and science. Liquid Music features lots of visual music. The History of Experimental Music in Northern California is in a large part a study of visual music. Good links too! VRML research projects -- check out the audio extensions work discussed by Bill Martens Contours of the Mind, an exhibition that coincided with Synaesthetica 94; a premiere conference specifically aimed at forging links between the fields of Computer Music and Computer Animation, hosted by the Australian Centre for the Arts and Technology. Synesthesia: Collaborative Biosignal Experience -- "This phenomena is particularly intriguing as it relates to current research on neurological networking/interaction as a critical tool in improving assessment, diagnosis, treatment and rehabilitation for the brain damaged population. Efforts are progressing on how to build optimal environments in which dendrite regrowth/new networking is encouraged and enchanced. In this project we will utilize individual biosignals (heart, breathing, muscle movement, etc.) to create and drive a virtual reality environment of graphics, sound, etc. First, we acquire their heartbeat and then feed it back as a recognizable sound. Then, we switch signals (auditory to visual). Synesthesia is a page by author Sean Day. The Synaesthesia Mailing List is a forum where synesthetes discuss their experiences. Subscribe by sending a message to the list administrator at daysa@muohio.edu. And archives were posted in January 2002. There are a lot of links at The Lycos Network psychology area and at Psychology & the Arts.

Visual and Auditory Research Links Retina Reference & Vision pointers by Lance Hahn at UPenn Digital Literacy: Visual Communication and Computer Images by Paul Martin Lester, CSU, Fullerton -- an overview Illusionworks, a comprehensive collection of optical and sensory illusions. Grand Illusions -- "takes an enquiring approach to optical illusions and scientific toys, and the principles that lie behind them. From the psychology of seeing to the mysteries of magic mirrors, new articles with a general appeal will appear regularly." The Media Lab, at the Massachusetts Institute of Technology. The Ultimate Spatial Audio Index Auditory Perception --"A multimedia presentation of selected topics in Auditory Perception, including auditory demonstrations, discussion and experiments in perception." New CDs mentioned on this site: Examples in Auditory Scene Analysis by Dr. Al Bregman & Musical Illusions and Paradoxes by Dr Diana Deutsch. PHYSICS AND PSYCHOPHYSICS OF MUSIC -- Course Notes from David Worrall, a composer and Head of TheAustralian Centre for theArts and Technology at The Australian National University ACM Special Interest Group on Sound and Computation & WWW Pages Relating to Sound Computation Center for Research in Computing and the Arts, at the University of California, San Diego. Center for New Music and Audio Technologies, at the University of California, Berkeley. Center for Computer Research in Music and Acoustics, at Stanford University. Institut de Recherche et Coordination Acoustique/Musique (IRCAM), in Paris.

Synesthetic Software (The VJs are taking over, so don't miss AudioVisualizers.com and similar resources are listed at the bottom of this section)   One fun development has been the addition visual plug-ins for MP3 players -- they all seem to have them and APIs for scripting. There have been a few CD-ROMs with A/V synchronization experiments. One was by EBN, Emergency Broadcast Network, whose enhanced CD-ROM, "Telecommunication Breakdown," and live shows with video walls expore video sampling through a variety of devices. Another title with experimentation and speculation was Marc Canter's Media Band. A number of other CD-ROM products use graphics to explore music. Maxis, publisher of SimCity, also produces SimTunes, which includes kid-oriented games that use cartoon to teach music. The Mixman CD-ROM is a virtual remixer but is limited to the provided tracks. OM Records produces enhanced CD-ROMS that also feature visual remixing interfaces (OM's drag and drop "mix id up" feature and strategy is detailed by Jonathan E. in MicroTimes). Coldcut's "Let Us Replay" comes with a demo software of their program VJAMM that allows you to remix Ninja Tunes videos on your computer. And, for there's a DJ/VJ support list called eyecandy. A number of developers are working on software for music visualization including Kai Krause and Peter Gabriel. Opcode Max is provides a very powerful environment for synchronizing media, even for live performances and installations. Other authoring programs, like Macromedia Director, provide a some of control for synchroniztion of media types, but are difficult to learn. A future version of Apple's QuickTime called "QuickTime Interactive" will make this ability ubiquitous. Adobe After Effects 5.0 supports expressions, which allow users to synchronize any parameter with any other parameter. So a range of audio effects can be tied to graphic properties with drag & drop controls or with Javascript code. You can also use the old internal AE scripting tool Motion Math if you have the Production Bundle. Trapcode makes a VM plug-in for AE called Sound Keys (Mac/Win), which uses the audio spectrum of a layer to generate a stream of keyframes. "With Motion Math, you can isolate frequency bands using the audio filters in AE prior to using layerad.mm; Sound Keys is an attempt to make it *easy* to make very audio-driven animations." Trish Meyer: "Not only can you easily isolate bass, mid, treble areas, but the keyframes generated are somehow 'smoother' than those generated by MM." Pixound, a browser plug-in, has several functions. According to WebMonkey, "the main gist is this: When you run your mouse over an image with a corresponding Pixound (.pxd) file, the colors (not the coordinates, but the colors) are translated into specific sounds - instantly and continuously. You're not downloading a single sound track, that plays and then stops. It's more like an instrument." Headspace has developed Beatnik to enhance audio on the Web. "The potential for interactivity results from Beatnik's support of a comprehensive set of JavaScript functions, which allow for a richer and more personalized musical experience than other multimedia delivery solutions. Beatnik allows a web page to play music not only upon the opening of a web page, but also on an event such as a "mouse click" or "mouse over." These events can also trigger individual notes, sampled voices, or sound effects; start or stop music; and change tempo, volume, pitch, or mix. This is revolutionary in that it allows and even encourages direct musical interactions with web pages, instead of the current use of the Internet as a playback-only system." Metasynth, (at IRCAM), from the creator of KPT Bryce, "is a program which makes it possible to synthesize sounds (represented as frequencies evolving in time) directly from an image coming from any source whatever (graphics, digital photo, drawn directly or derived from the analysis of an existing sound)." This is very cool software! Look out for MathematicArt and Midi Kaleido too. The makers of MetaSynth, U & I Software make similar Mac-only programs too: Videodelic, VTRACK, and Artmatic. Bliss Paint -- "Bliss Paint is a cutting edge real-time animation tool for the creation of images for video and multimedia production, texture and background generation, interactive performances, logo animation, image processing, fine arts, and more. Colors flow through your animations as you generate images by drawing from a large library of animated shapes and patterns. MIDI instruments may be used to control the color synthesizer and shape selection in Bliss Paint. Sound input through a microphone or line input may be used to control color synthesis. With sound and MIDI input, Bliss Paint animations respond rhythmically to live music or music from your stereo. Many of Bliss Paint's features are designed for live performance animation for music." The Music Animation Machine --"Music moves, and can be understood just by listening. But a conventional musical score stands still, and can be understood only after years of training. The Music Animation Machine bridges this gap, with a score that moves -- and can be understood just by watching." Bomb "is a visual-musical instrument. It uses alife [articial life], and is alife. It runs on your PC and produces animated organic graphics in response to the keyboard, audio music, or on its own. You can download it from this web site, and run it on your PC, Mac or SGI...David Zicarelli, the man behind Max, has made Bomb into a Max object/plugin so you can control the graphics with midi or anything else." Visual Music Tone Painter -- "is a new art form which merges sound, light, and touch. This exciting program converts signals from synthesizers or other MIDI instruments into a visual display in real time - bringing music to life by allowing the user to paint with sound. Choosing from a palette of gorgeously coloredgeometric shapes ranging from simple circles, polygons, spirals and waves to more complex forms such as letter forms, three dimensional spinning cubes, epitrochoids, and lissajous curves, the player creates mandalas of light and sound. These visual forms respond not only to pitch but also subtle nuances of finger pressure and release as they dance across the screen in instantaneous reflection of the musical gesture. These musical dynamics are reflected through visual transformations in size, movement, hue, saturation and value. As the music develops, seemingly simple geometries build into moire patterns, strobing, and other multi-layered effects. The resulting synesthesia hypnotizes and engages the faculties of users ranging from 1 year olds to highly sophisticated musicians, exploding blocks to creativity as the players explore the variety of fascinating interactions between sound and color. Musical novices are inspired to improvise and lose their fear of playing an instrument." SoundHack, shareware for the Macintosh, includes a binaural filter from HRTFs of Durand Begault. SoundArt, is an "entertainment that uses your PC's sound card to listen to music and produce mesmerizing visual imagery. It transforms music into graphics and creates captivating performance art." PlayerPro -- "Play and edit MOD music from the Amiga & IBM and save the files (with instruments, partitions, etc.); record new instruments, and view the music as it plays on an oscilloscope and a virtual keyboard. MIDI files too." It's multiplatform, and there's also a Netscape plug-in available. SoundView -- "Shareware that allows you to view real-time time-wave and frequency spectrum info from any Macintosh sound input device. You can record and playback 8-bit sound, as well." Spektrum-- "Looks and acts like a 256-channel spectrum analyzer for any normal 22-KHz Mac sound file. You can also record new sound samples, play and analyze them at your command." IBVA (this one is spooky...) -- "IBVA reads your brain waves in real time and allows you to use them to trigger images, sounds, other software or almost any electronically addressable device through its MIDI and serial control functions. With the network and modem features of the IBVA, your brain waves can be analyzed and control equipment from anywhere in the world." Soundtoys, "a new site featuring sound toys and multimedia experiments, is seeking artists, multimedia developers, digital musicians and creative programmers who may want to contribute sound toys, texts, games, generative music, interactive environments, Shockwave, etc. In addition to exposure through the Web site, the work will be promoted worldwide at festivals and on a CD-ROM. Please include information about when the work was created, who made it, any technical details, contact details, etc." aestesis makes Windows software for realtime video, and carries community news. Steim makes Mac software for video sampling and playback as well as a tool to take video information and convert it into Midi messages. Onadime makes software for music visualization in real-time. Here's MacAddict review of Composer. PixelToy (Mac shareware) can make lava lamp and MP3 visualizer-type animations. ArKaos VJ was developed as a real-time multimedia tool for professional VJs, installation, and event artists. Artists can work with their own image and movie files along with a variety of different effect types and sequences. The software enables users to adjust a variety of parameters and it supports full MIDI suites both on-screen and off, as well as music authoring programs like Cubase. ArKaos's software has been used by David Bowie, The Chemical Brothers, Jean-Michel Jarre, Fatboy Slim, and U2,and by corporations like Ford and Apple. Apparently this product was Steinberg's X<>Pose, a "visual sampling program that lets users trigger playback of up to 88 QuickTime or PICT files directly from the Mac keyboard or any MIDI device."   ||||see also these more dedicated sites||| Ron Pellegrino's Recommended Sites of Music Visualizers TOUECCANS is a complete but aging guide to eye candy programs, animated and non-animated. Electronica-Optica -- VJs & Visual Artists VJCentral.com -- "Community Site for VJs by VJs" "VJ & Video Performance Art Haven - The Evolution of the DJ" contains many resources for VM hardware and software. AudioVisualizers.com "provides a comprehensive forum for video performance artists who run the gamut from high intensity rave video mixers, to experimental interactive installation artists, and other likeminded creative folks..." The number of VJ programs is now amazing. They also provide a history which hardware facts.

Famous synesthetes Famous synesthetes include (according to Cytowic and Ackerman): Nabakov, Faulkner, Baudelaire, Joyce, Kandinsky, Hockney, Scriabin, Rimsky-Korsakov, Messiaen. Some of their experiences are described at the ISA site.

Synesthetic Media: A Select Bibliography Abraham, Ralph, "Visual Music Instruments & Chaos," Reality Hackers , Winter 1988. Ackerman, Diane. A Natural History of the Senses, Random House, c1990 Art in Cinema, Symposium catalog, San Francisco Museum of Art, 1947. Barren-Cohen, Simon and John E. Harrison, editors, Synesthesia: Classic and Contemporary Readings, Blackwell, 1997. Bendazzi, Giannalberto, Cartoons: One Hundred Years of Cinema Animation , Indiana, 1994. Calvin, William, The Cerebral Code: Thinking a Thought in the Mosaic of the Mind, MIT, 1996. Cecil, Paul, Flickers of the Dreamachine, Codex (UK), 1996 (caution: "Flickers" is Brion Gysin's dark ride). Chion, Michel, Audio-Vision: Sound in the Screen, Columbia University Press, 1994. Critchley, MacDonald, "Ecstatic and synesthetic experience during music perception," in Music and the Brain , M. Critchley, ed., C.C. Thomas, 1977. Cytowic, Richard, Synesthesia: Union of the Senses , Springer-Verlag, 1989. Cytowic, Richard, The Man Who Tasted Shapes , Putnam, 1993. Dann, Kevin, Bright Colors Falsely Seen: Synaesthesia and the Search for Transcendental Knowledge , Yale, 1998. Falk, David, Dieter Brill and David Stork, Seeing the Light: Optics in Nature, Photography, Color, Vision and Holography , Wiley, 1986. Forster, E.M., "The Machine Stops," in The Eternal Moment and Other Stories , 1928 (Hartcourt Brace). Remember: "only connect." Gerstner, Karl, Forms of Color , MIT, 1986. Hesse, Herman, Magister Ludi (some translations: The Glass Bead Game), 1943 (Henry Holt). Huxley, Aldous, Brave New World , 1932, Harper. (Remember the feelies?). Jenny, Hans, Cymatics (lit.- matters pertaining to waves), Basel: Basilius Press, 1967 (vol.1) and 1974 (vol.2). Short 1986 video available from MACROmedia, PO Box 279, Epping, NH, 03042. Konigsberg, Ira, "Cave Paintings and the Cinema," Wide Angle 18 (2), April 1996. Kramer, Gregory (ed.), Auditory Display: Sonification, Audification and Auditory Interfaces, Santa Fe Institute/Addison-Weslay, 1992. Kruger, Myron, Artificial Reality , Addison-Wesley, 1983. Land, Edwin, "The Retinex Theory of Color Vision," The Perceptual World , Irvin Rock, ed., Freeman, 1990. Laurel, Brenda, Computers as Theatre , Addison-Wesley, 1991. Lawlor, Robert, Sacred Geometry , Crossroad, 1982. Lewis-Williams, J. David, and Thomas Dowson, "The Signs of All Times: Entoptic Phenomena in Upper Paleolithic Art", Current Anthropology, 29:201-245, 1988. Lings, Martin, The Qur'anic Art of Calligraphy and Illumination , Interlink, 1987 (orig. 1976). Meggs, Plillip B., A History of Graphic Design, Van Nostrand Rheinhold, 1992. Moritz, William, "Abstract Film and Color Music," in The Spiritual in Art: Abstract Painting 1890-1985, Abbeville, 1986. Moritz, William, VISUAL MUSIC: Larry Cuba's Experimental Film , Mediagramm, ZKM Karlsruhe, July 1996. Murphy, Michael and Rhea White, The Psychic Side of Sports , Addison-Wesley, 1978. Peacock, Kenneth, "Instruments to Perform Color-Music: Two Centuries of Technological Experimentation," Leonardo 21(4), 1988. Pfeiffer, John, The Creative Explosion: An Inquiry into the Origins of Art and Religion , Harper & Row, 1982. Popper, Frank, Art of the Electronic Age , Abrams, 1993. Popper, Frank, Kinetic Art: Origins and Development , New York Graphics Society, 1968. Rossing, Thomas, Science of Sound , Addison-Wesley, 1982. (see Chladni patterns made with optics). Rouget, Gilbert, Music and Trance , Chicago, 1982. Russett, Robert and Cecil Starr, Experimental Animation , DeCapo, 1988. Schafer, R. Murray, The Tuning of the World: Toward a Theory of Soundscape Design , Pennsylvania, 1980. Settgast, Mary, Plato Prehistorian , Lindisfarne, 1990. Sitney, P. Adams, Visionary Film, Oxford, 1974. (see chap. 8, Absolute Animation) White, Randall, "The Earliest Images: the Origins and Evolution of Ice Age "Art," Institute of Ice Age Studies, On-Line Library, New York University. A critique was posted by Eric Pettifor. Whitney, John, Digital Harmony: On the Complementarity of Music and Visual Art , Byte, 1980. Winfree, Arthur, The Timing of Biological Clocks , Scientific American Books, 1987. Youngblood, Gene, Expanded Cinema , Dalton, 1970. (a book on synesthetic cinema!). Youngblood, Gene, CALCULATED MOVEMENTS: AN INTERVIEW WITH LARRY CUBA, Video and the Arts Magazine, Winter 1986.   There are more references at : the MIT site a good annotated bibliography from Leonardo Electronic Almanac Sean Day's Bibliography

* "The Caves of Altamira" is a song by Steely Dan.

"The Poem of the Cloak" celebrates the Prophet Muhammad (saw) who first received revelation

in a cave on the Mountain of Light.

Back to Synesthetic Media Connections

Back to MSP HomePage

version 04/14/03 (aefilter at yahoo.com)