Scientific Foundations of Cinematography with Gordon Arkenberg

Scientific Foundations of Cinematography

“First study science, and then follow the practice born of that science. Those who fall in love with practice without science are like pilots who board a ship without rudder or compass, who are never certain where they are going.”

– Leonardo da Vinci, Treatise on Painting

The scientific foundations of the visual arts are treated briefly, if at all, in classes and books about cinematography. In most filmmaking programs, the fundamental experiments, concepts, and questions in science are treated discursively – they are a means to an end, a footnote to the artistry of creating images.

The lectures in Scientific Foundations of Cinematography take Leonardo’s dictum to heart. Instead of assuming that ‘an ounce of effort is worth a ton of theory’, a deep theoretical dive reveals that ‘an ounce of theory is worth a ton of effort’. There is no better guide than reviewing the legacy of experts (historical and modern) in physics, mathematics, and engineering. As we explore every dimension of a scientific topic, the science in cinematography will become a lodestone that transforms your inspiration into action.

Educator (and 1st AC) Gordon Arkenberg has been teaching Science of Cinematography since 2010 in the undergraduate Film/TV department at NYU/Tisch School of the Arts. The five lectures in Scientific Foundations focus on topics where he finds the most confusion and misinformation. Each lesson will explore the history that informs our current understanding through seminal and rarely seen experiments. No mathematical or scientific prerequisites are necessary, just bring your curiosity. 

1. Day One - Color Theory: The Rainbow from Eye to Mind

In February 1790, Johann Wolfgang Goethe decided to finally return the optical equipment loaned to him by a professor and friend. In a gesture of farewell, Goethe decided to take one last look through the prism to see Newton’s famous spectrum. When he held it up to his eye he reportedly exclaimed “Newton is wrong!” What did Goethe see? Did his visual experience really disprove the scientific genius of Newton? Regardless, Goethe did not return the optical accoutrement but vigorously pursued an exploration into color phenomenon that lasted until his death.

Light is not colored per se. There is no such thing as a ‘red photon’, and a wavelength of 650 nanometers is not ‘red’. The science of color is both physical and psychological – studied through the discipline of psychophysics. This lecture will follow a historical and ideological thread that emerged from the conceptual dichotomy between Isaac Newton and Johann Goethe. (The alternate thread is followed in the subsequent lecture, Colorimetry.) The psychophysical exploration of color was passed from Goethe to Arthur Schopenhauer, developed by physiologist Ewald Hering, and pursued by vision scientists into the 21st century.

The nature of color, as a psychophysical experience, is best understood by witnessing the experiments discovered and discussed by physicists, philosophers, chemists, and engineers. From seeing the ‘inverted’ spectrum, colored shadows, and Edwin Land’s famous full color image using two projectors, you will literally not believe your eyes.

2. Day Two - Colorimetry: the History and Foundations

Physicist and mathematician J.J. Koenderink recently wrote that “colorimetry has been a closed subject since the 1920’s. All that happened later are just additional conventions. By their very nature these are in academic bad taste. In a sense colorimetry is a ‘dead science’ then.” This sounds radically at odds with the omnipresence of CIE 1931xy diagrams in lighting control software and the reputed solution chromaticity coordinates bring to matching colors across different technologies.

This lecture sets aside the engineering conventions of the 20th century to focus on the formal basis of colorimetry established in the 19th century. This is a mathematical and geometric view of color Isaac Newton introduced, Hermann Grassmann formalized, and James Clerk Maxwell experimentally verified. Key experiments will be reproduced, and attendees should be prepared to calculate and graph chromaticity coordinates. (No worries, the math level is basic arithmetic with details for the algebra curious.)

By its conclusion, this historical journey through the inception of color science will empower your understanding of modern colorimetry diagrams. Most critically, it will help you see colorimetry as a tool that, in the right situation, can solve complex problems.

3. Day Three - Camera Sensors: Through the Lens of Information Theory

Following World War II, a revolutionary theory of communication obtained dominance called information theory. Emerging from the mind of Claude Shannon, it was swiftly applied to a range of fields from physics to linguistics. For those in the photographic arts, information theory began to inform the way physicists and engineers were assessing and developing imaging technology. Open any serious book on camera technology from the mid-20th century and information theory is widely discussed. Where did it go? Is it no longer useful?

Information theory provides key principles for understanding and assessing camera image performance. Its ideas and methods moved manufacturers away from psychophysical methods to physico-mathematical models. The principles it established are now so thoroughly baked into camera assessment that their origin is rarely referenced. If you have researched a camera’s signal-to-noise ratio or looked up a modulation transfer function, you have information theory to thank.

This lecture will apply ‘old’ concepts to new cameras. By looking at data generated from modern cinema cameras, patterns in their performance will emerge that clarify the nature and qualities of electronic imaging. This empowers cinematographers to interpret advertising claims, design and perform their own tests, and assess image quality from fundamental principles. Finally, the class will look at the return of information theory in camera design through recent work by engineers calculating the Shannon information capacity of digital cameras.

4. Day Four - Human Vision: Are You a Camera?

Properties of the human visual system are commonly analogized through camera specifications. For example, we often hear that the 50mm focal length is the closest match to the human eye, 24fps provides ‘natural’ motion, and we can see 10 million colors. Are these quantities really validated by vision science?

“Are You a Camera?” dives into comparing popular claims about our visual system against the physiological and psychophysiological research. The lecture will subject its attendees to experiments in order to establish focal length, angle of view, f-number, resolution, dynamic range, color depth, and frame rate of the visual system.

By its conclusion, we can establish the aspects of human vision that are reducible to quantities, and those that defy simplification. Understanding your own vision will clarify what is meant by ‘photographic seeing’, which in turn opens new dimensions in how you engage with photographic technology.

5. Day Five - Linear Perspective: A Journey in Dimensions

The camera with a pinhole or lens intercepts rays from a three-dimensional world and projects them onto a two-dimensional plane. In which case, one would imagine that photographers and cinematographers would be well acquainted with the principles of linear perspective. Then, why do some claim that perspective has nothing to do with focal length when zooming clearly changes our perception of space within the image? How is it possible that different formats and focal lengths can create the same perceptual experience?

The problem with exploring perspective exclusively through a camera is that critical degrees of freedom can be missed or ignored - the photographic system does too much work for you. To remedy this situation, this lecture returns to the birth of linear perspective with Leon Battista Alberti’s notion of tracing the world onto a window. Recreating this concept physically in the classroom will allow us to understand every component of a linear perspectival system.

Following from the core concepts of projective geometry, we will explore the perspectival distortions that unnerved Leonardo da Vinci, and which cinematographers exploit today. We will visit popular perspective ‘explanations’ to uncover how their conclusions are incomplete and see novel experiments that erase the effects of the dolly-zoom. This journey through dimensions will introduce a new verbal and visual language that will change your perspective on…well, perspective.

*Note: All classes are ticketed separately. Education and IATSE discounts are avalaible upon request. By purchasing all 5 separate workshops, attendees can receive a discounted bulk rate. Please inquire via [email protected].