See What You Hear: Audio Calibration for Video Digitization

I’ve always been amused by the way a certain professional field frequently goes out of its way to shout “we don’t understand audio” to the world. (Association of Moving Image Archivists, Moving Image Archiving and Preservation, Moving Image Archive Studies, Museum of the Moving Image, etc. etc.)

“But there’s no good word to quickly cover the range of media we potentially work with,” you cry! To which I say, “audiovisual” is a perfectly good format-agnostic term that’s been in the public consciousness for decades and doesn’t confuse my second cousins when I try to explain what I do. “But silent film!” you counter, trying to use clever technicalities. To which I say, silent film was almost never really silent, home movies were meant to be narrated over, and stop being a semantic straw man when I’m trying to have a hot take over here!

The point is: when working in video preservation and digitization, our training and resources have a tendency to lean toward “visual” fidelity rather than the “audio” half of things (and it IS half). I’m as guilty of it as anyone. As I’ve described before, I’m a visual learner and it bleeds into my understanding and documentation of technical concepts. So I’m going to take a leap and try and address another personal Achilles’ heel: audio calibration, monitoring, and transfer for videotape digitization.

I hope this to be the first in an eventual two-part post (though both halves can stand alone as well). Today I’ll be talking about principles of audio monitoring: scales, scopes and characteristics to watch out for. Tune in later for a breakdown of format-by-format tips and tricks that vary depending on which video format you’re working with: track arrangements, encoding, common errors, and more. My focus, for now, is on audio in regards to videotape – audio-only formats like 1/4″ open reel, audio cassette, vinyl/lacquer disc and more bring their own concerns that I won’t get into right now, if only for the sake of scope crawl (I’ve been writing enough 3000+ word posts of late). But much if not all of the content in this post particularly should be applicable to audio-only preservation as well!

Big thanks to Andrew Weaver for letting me pick his brain and help spitball these posts!

The Spinal Tap Problem

Anyone who has ever attended one of my workshops knows that I love to take a classic bit of comedy and turn it into a buzz-killing object lesson. So:

Besides an exceptional sense of improvisational timing, what we have here is an excellent illustration of a fundamental disconnect in audio calibration and monitoring: the difference between how audio signal is measured (e.g. on a scale of 1 to 10) and how it is perceived (“loudness”).

There are two places where we can measure or monitor audio: on the signal level (as the audio passes through electrical equipment and wires, as voltage) or on the output level (as it travels through the air and hits our ears as a series of vibrations). We tend to be obsessed with the latter – judging audio based on whether it’s “too quiet” or “too loud”, which strictly speaking is as much a matter of presentation as preservation. Cranking the volume knob to 11 on a set of speakers may cause unpleasant aural side effects (crackling, popping, bleeding eardrums) but the audio signal as recorded on the videotape you’re watching stays the same.

To be clear, this isn’t actually any different than video signal: as I’ve alluded to in past posts, a poorly calibrated computer monitor can affect the brightness and color of how a video is displayed regardless of what the signal’s underlying math is trying to show you. So just as we use waveform monitors and vectorscopes to look at video signals, we need “objective” scales and monitors to tell us what is happening on the signal level of audio to make sure that we are accurately and completely transferring analog audio into the digital realm.

Just like different color spaces have come up with different scales and algorithms for communicating color, different scales and systems can be applied to audio depending on the source and/or characteristic in question. Contextualizing and knowing how to “read” exactly what these scales are telling us is something that tends to get lost by the wayside in video preservation, and what I’m aiming to demystify a bit here.

Measuring Audio Signal

So if we’re concerned with monitoring audio on the signal level – how do we do that?

All audiovisual signal/information is ultimately just electricity passed along on wires, whether that signal is interpreted as analog (a continuous wave) or digital (a string of binary on/off data points). So at some level measuring a signal quantitatively (rather than how it looks or sounds) always means getting down and interpreting the voltage: the fluctuations in electric charge passed along a cable or wire.

In straight scientific terms, voltage is usually measured in volts (V). But engineers tend to come up with other scales to interpret voltage that adjust unit values and are thus more meaningful to their own needs. Take analog video signal, for instance: rather than volts, we use the IRE scale to talk about, interpret and adjust some of the most important characteristics of video (brightness and sync).

We never really talk about it in these terms, but +100 IRE (the upper limit of NTSC’s “broadcast range”, essentially “white”) is equivalent to 700 millivolts (0.7 V). We *could* just use volts/millivolts to talk about video signal, but the IRE scale was designed to be more directly illustrative about data points important to analog video. Think of it this way: what number is easier to remember, 100 or 700?

Only one of those numbers has an emoji, is all I’m saying.

Audio engineers had the same conundrum when it came to analog audio signal. Where it gets super confusing from here is that many scales emerged to translate voltage into something useful for measuring audio. I think the best way to proceed from here is just to break down the various scales you might see and the context for/behind them.

dBu/dBv

Decibel-based scales are logarithmic rather than linear, which makes them ideal for measuring audio signals and vibrations – the human ear is more sensitive to certain changes in frequency and/or amplitude than others, and a logarithmic scale can better account for that (quite similar to gamma correction when it comes to color/luminance and the human eye).

sighhhhhhhhh

The problem is that decibels are also a relative unit of measurement: something can not just be “1 dB” or “1000 dB” loud; it can only be 1 dB or 1000 dB louder than something else. So you can see quite a lot of scales related to audio that start with “dB” but then have some sort of letter serving as a suffix; this suffix specifies what “sound” or voltage or other value is serving as the reference point for that particular scale.

An extremely common decibel-based scale for measuring analog audio signals is dBu. The “u” value in there stands for an “unterminated” voltage of 0.775 volts (in other words, the value “+1 dBu” stands for an audio wave that is 1 decibel louder than the audio wave generated by a sustained voltage of 0.775 V).

In the analog world, dBu is considered a very precise unit of measurement, since it was based on electrical voltage values rather than any “sound”, which can get subjective. So you’ll see it in a lot of professional analog audio applications, including professional-grade video equipment.

Confusingly: “dBu” was originally called “dBv”, but was re-named to avoid confusion with the next unit of measurement on this list. So yes, it is very important to distinguish whether you are dealing with a lower-case “v” or upper-case “V”. If you see “dBv”, it should be completely interchangeable with “dBu” (…unless someone just wrote it incorrectly).

dBV

dBV functions much the same as dBu, except the reference value used is equivalent to exactly 1 volt (1 V). It is also used as a measurement of analog audio signal. (+1 dBV indicates an audio wave one decibel louder than the wave generated by a sustained voltage of 1 V)

…sure!

Now…why do these two scales exist, referenced to slightly different voltages? Honestly, I’m still a bit mystified myself. Explanations of these reference values delve quite a bit into characteristics of electrical impedance and resistance that I don’t feel adequately prepped/informed enough to get into at the moment.

What you DO need to know is that a fair amount of consumer-grade analog audiovisual equipment was calibrated according to and uses the dBV scale instead of dBu. This will be a concern if you’re figuring out how to properly mix and match and set up equipment, but let’s table that for a minute.

PPM and VU

dBu and dBV, while intended for accurately measuring audio signal/waves, still had a substantial foot in the world of electrical signal/voltage, as I’ve shown here. Audio engineers still wanted their version of “IRE”: a reference scale and accompanying monitor/meter that was most useful and illustrative for the practical range of sound and audio signal that they tended to work with. At the time (~1930s), that meant radio broadcasting, so just keep that in mind whenever you’re flipping over your desk in frustration trying to figure out why audio calibration is the way it is.

The BBC struck in this arena first, with PPM (peak program meter), a highly accurate meter and scale intended to show “instant peaks”: the highest point on each crest of an audio wave. These meters became very popular in European broadcasting environments, but different countries and engineers couldn’t agree on what reference value, and therefore measurement scale, to use. So if you come across a PPM audio meter, you might see all kind of scales/number values depending on the context and who made it: the original BBC PPM scale went from 1 to 7, for instance (with 6 being the “intended maximum level” for radio broadcasts) while EBU PPM, ABC (American) PPM, Nordic PPM, DIN PPM, CBC PPM, etc. etc., might all show different decibel levels.

Why

In the United States, however, audio/radio engineers decided that PPM would be too expensive to implement, and instead came up with VU meters. VU stands for Volume Units (not, as I thought myself for a long time, “voltage units”!!! VU and V are totally different units of scale/measurement).

VU meters are averaged, which means they don’t so much give a precise reading of the peaks of audio waves so much as a generalized sense of the strength of the signal. Even though this meant they might miss certain fluctuations (a very quick decibel spike on an audio wave might not fully register on a VU meter if it is brief and unsustained), this translated close enough to perceived loudness that American engineers went with this lower-cost option. VU meters (and the Volume Unit scale that accompany it) are and always have been intended to get analog audio “in the ballpark” rather than give highly accurate readings – you can see this in the incredibly bad low-level sensitivity on most VU meters (going from, say, -26 VU to -20 VU, for instance, is barely going to register a blip on your needle).

So I’ve lumped these two scales/types of meter together because you’re going to see them in similar situations (equipment for in-studio production monitoring for analog A/V), just generally varying by your geography. From here on out I will focus on VU because it is the scale I am used to dealing with as an archivist in the United States.

dBFS

All of these scales I’ve described so far have related to analog audio. If the whole point of this post is to talk about digitizing analog video formats…what about digital audio?

Thankfully, digital audio is a little more straightforward, at least in that there’s pretty much only one scale to concern yourself with: dBFS (“decibels below [or in relation to] Full Scale”).

Whereas analog scales tend to use relatively “central” reference values – where ensuing audio measurements can be either higher OR lower than the “zero” point – the “Full Scale” reference refers to the point at which digital audio equipment simply will not accept any higher value. In other words, 0 dBFS is technically the highest possible point on the scale, and all other audio values can only be lower (-1 dBFS, -100 dBFS, etc. etc.), because anything higher would simply be clipped: literally, the audio wave is just cut off at that point.

Tipping the Scales

All right. dBu, dBV, dBFS, VU….I’ve thrown around a lot of acronyms and explanations here, but what does this all actually add up to?

If you take away anything from this post, remember this:

0 VU = +4 dBu = -20 dBFS

The only way to sift through all these different scales and systems – the only way to take an analog audio signal and make sure it is being translated accurately into a digital audio signal – is to calibrate them all against a trusted, known reference. In other words – we need a reference point for the reference points.

In the audio world, that is accomplished using a 1 kHz sine wave test tone. Like SMPTE color bars, the 1 kHz test tone is used to calibrate all audio equipment, whether analog or digital, to ensure that they’re all understanding an audio signal the same way, even if they’re using different numbers/scales to express it.

We worship you, o Lord our savior

In the analog world, this test tone is literally the reference point for the VU scale – so if you play a 1 kHz test tone on equipment with VU meters, it should read 0 VU. From there, the logarithms and standards demand that 0 VU is the same as +4 dBu. That is where the test tone should read if you have equipment that uses those scales.

dBFS is a *little* more tricky. It’s SMPTE-recommended practice to set 1 kHz test tone to read at -20 dBFS on digital audio meters – but this is not a hard-and-fast standard. Depending on the context, some equipment (and therefore the audio signals recorded using them) are calibrated so that a 1 kHz test tone is meant to hit -18 or even -14 dBFS, which can throw the relationship between your scales all out of whack.

(In my experience, however, 99 times out of 100 you will be fine assuming 0 VU = -20 dBFS)

Once you’re confident (relatively) that everyone’s starting in the same place, that makes it possible to proceed from there: audio signals hitting between 0 and +3 VU on VU meters, for example, should be hitting roughly between -20 dBFS and -6 dBFS on a digital scale.

Note that these are all logarithmic scales based on different logarithms – so they are never going to line up one-to-one except at the agreed-upon reference point. That is, +4 dBu may be equal to 0 VU, but +5 dBu is not equal to 1 VU. When it comes to translating audio signal from one system and scale to another, we can follow certain guidelines and ranges, but there is always going to be a certain amount of imprecision and subjectivity in working with these scales on a practical level during the digitization process.

Danger, Will Robinson

Remember when I said that we were talking about signal level, not necessarily output level, in this post? And then I said something about how professional equipment calibrated to the dBu scale versus how consumer equipment calibrated to dBV scale? Sorry. Let’s circle back to that for a minute.

Audio equipment engineers and manufacturers didn’t stop trying to cut corners when they adopted the VU meters over PPM. As a cost-saving measure for wider consumer releases, they wanted to make audio devices with ever-cheaper physical components. Cheaper components literally can’t handle as much voltage passing through them as higher-quality, “professional-grade” components and equipment.

The height of professionalism

So many consumer-grade devices were calibrated to output a 1 kHz test tone audio signal at -10 dBV, which is equivalent to a significantly lower voltage than the professional, +4 dBu standard.

(The math makes my head hurt, but you can walk through it in this post; also, I get the necessary difference in voltage but no, I still don’t really understand why this necessitated a difference in the decibel scale used)

What this means is: if you’re not careful, and you’re mixing devices that weren’t meant to work together, you can output a signal that is too strong for the input equipment to handle (professional -> consumer), or way weaker than it should be (consumer -> professional).  I’ll quote here the most important conclusion from that post I just linked above:

If you plug a +4dBu output into a -10dBV input the signal is coming in 11.79dB hotter than the gear was designed for… turn something down.

If you plug a -10dBV output into a +4dBu input the signal is coming in 11.79dB quieter than the gear was designed for… turn something up.

Unbalanced audio signal/cables are a big indicator of equipment calibrated to -10 dBV: so watch out for any audio cables and connections you’re making with RCA connectors.

Consumer-grade blogging

The reality is also that after a while many professional-grade manufacturers were aware of the -10 dBV/+4 dBu divide, and factored that into their equipment: somewhere, somehow (usually on the back of your device, perhaps on an unmarked switch) is the ability to actually swap back and forth between expecting a -10 dBV input and a +4 dBu one (thereby making any voltage gain adjustments to give you your *expected* VU/dBFS readings accordingly). Otherwise, you’ll have to figure out a way to make your voltage gain adjustments yourself.

The lessons are two-fold:

  1. Find a manual and get to know your equipment!!
  2. You can plug in consumer to professional equipment, but BE CAREFUL going professional into consumer!! It is possible to literally fry circuits by overloading them with the extra voltage and cause serious damage.

Set Phase-rs to Stun

There’s another thing to watch out for while we’re on the general topic of balanced and unbalanced audio, which are the concepts of polarity and phase.

Polarity is what makes balanced audio work; it refers to the relation of an audio signal’s position to the median line of voltage (0 V). Audio sine waves swing from positive voltage to negative voltage and vice versa -precisely inverting the polarity of a wave (i.e. taking a voltage of +0.5 V and flipping it to -0.5 V) and summing those two signals together (playing them at the same time) results in complete cancellation.

Professional audio connections (those using XLR cables/connections) take advantage of this quality of polarity to help eliminate noise from audio signals (again, you can read my post from a couple years back to learn more about that process). But this relies on audio cables and equipment being correctly wired: it’s *possible* for technicians, especially those making or working with custom setups, to accidentally solder a wire such that the “negative” wire on an XLR connector leads to a “positive” input on a deck or vice versa.

This would result in all kinds of insanely incorrectly-recorded signals, and  probably caught very quickly. But it’s a thing to possibly watch out for – and if you’re handed an analog video tape where the audio was somehow recorded with inverse polarity, there are often options (both on analog equipment or in digital audio software, depending on what you have on hand) that are as easy as flipping a switch or button, rather than having to custom solder wires to match the original misalignment of the recording equipment.

This is where phase might come into play, though. Phase is, essentially, a delay of audio signal. It’s expressed in terms of relation to the starting point of the original audio sine wave: e.g. a 90 degree phase shift would result in a quarter-rotation, or a delay of a quarter of a wavelength.

In my experience, phase doesn’t come too much into play when digitizing audio – except that a 180 degree phase shift can, inconveniently, look precisely the same as a polarity inversion when looking at an audio waveform. This has led to some sloppy labeling and nomenclature in audio equipment, meaning that you may see settings on either analog or digital equipment that refer to “reversing the phase” when what they actually do is reverse the polarity.

You can read a bit more here about the distinction between the two, including what “phase shifts” really mean in audio terms, but the lesson here is to watch your waveforms and be careful of what your audio settings are actually doing to the signal, regardless of how they’re labelled.

Reading Your Audio

I’ve referred to a few tools for watching and “reading” the characteristics of audio we’ve been discussing. For clarity’s sake, in this section I’ll review exactly, for practical purposes, what tools and monitors you can look at to keep track of your audio.

Level Meters

Level meters are crucial to measuring signal level and will be where you see scales such as dBu, dBFS, VU, etc. In both analog and digital form, they’re often handily color-coded; so if after all of this, you still don’t really get the difference between dBu and dBFS, to some degree it doesn’t matter: level meters will warn you when levels are getting too hot by changing from green to yellow and eventually to red when you’re in danger of getting too “hot” and clipping (whatever that equivalent point is in the scale in question).

Waveform

Waveforms will actually chart the shape of an audio wave; they’re basically a graph with time on the x-axis and amplitude (usually measured in voltage) on the y-axis. These are usually great for post-digitization quality control work, since they give an idea of audio levels not just in any one given moment but over the whole length of the recording. That can alert you to issues like noise in the signal (if, say, the waveform stays high where you would expect more fluctuation in a recording that alternates loud and quiet sections) or unwanted shifts in polarity.

Waveform monitors can sometimes come in the form of oscilloscopes: these are essentially the same device, in terms of showing the user the “shape” of the audio signal (the amplitude of the wave based on voltage). Oscilloscopes tend to be more of a “live” form of monitoring, like level meters – that is, they function moment-to-moment and require the audio to be actively playing to show you anything. Digital waveform monitors tend to actually save/track themselves over time to give the full shape of the recording/signal, rather than just the wave at any one given moment.

Spectrogram

Spectrograms help with a quality of audio that we haven’t really gotten to yet: frequency. Like waveforms, they are a graph with time on the x-axis, but instead of amplitude they chart audio wave frequency.

If amplitude is perceived by human ears as loudness, frequency is perceived as pitch. They end up looking something like a “heat map” of an audio signal – stronger frequencies in the recording show up “brighter” on a spectrogram.

Spectrograms are crucial to audio engineers for crafting and recording new signals, “cleaning up” audio signals by removing unwanted frequencies. As archivists, you’re probably not actually looking to mess or change with the frequencies in your recorded signal, but they can be helpful for checking and controlling your equipment; that is, making sure that you’re not introducing any new noise into the audio signal in the process of digitization. Certain frequencies can be dead giveaways for electrical hum, for example.

The More You Know

This is all a lot to sift through. I hope this post clarifies a few things – at the very least, why so much of the language around digitizing audio, and especially digitizing audio for video, is so muddled.

I’ll leave off with a few general tips and resources:

  • Get to know your equipment. Test audio reference signals as they pass through different combinations of devices to get an idea what (if anything) each is doing to your signal. The better you know your “default” position, the more confidently you can move forward with analyzing and reading individual recordings.
  • Get to know your meters. Which one of the scales I mentioned are they using? What does that tell you? If you have both analog and digital meters (which would be ideal), how do they relate as you move from one to the other?
  • Leave “headroom”. This is the general concept in audio engineering of adjusting voltage/gain/amplitude so that you can be confident there is space between the top of your audio waveform and “clipping” level (wherever that is). For digitization purposes, if you’re ever in doubt about where your levels should be, push it down and leave more headroom. If the choice is between capturing the accurate shape of an audio signal, albeit too “low”, there will be opportunity to readjust that signal again later as long as you got all the information. If you clip when digitizing, that’s it – you’re not getting that signal information back and you’re not going to be able to adjust it.

“What is the difference between level, volume and loudness?”, David Mellor

“What is the difference between dBFS, VU and dBU [sic] in Audio Recordings?”, Emerson Maningo

“Understanding & Measuring Digital Audio Levels”, Glen Kropuenske

“Audio” error entries on the AV Artifact Atlas

Color is Math

I know, I hate to break it to you. I’m not thrilled about it either – one of my favorite memories from high school is walking straight up to my Statistics teacher on the last day of senior year and proudly announcing that it was the last time I was ever going to be in a math class. (Yes, I do feel a bit guilty about the look of befuddled disappointment on his face, but by god I was right)

But it’s true: at least when it comes to video preservation, color is math. Everything else you thought you knew about color – that it’s light, that you get colors by mixing together other colors, that it’s pretty – is irrelevant.

LIES

Just now, you thought you were looking at color bars? Wrong. Numbers.

nooooooooooooooooooooooooooooooooooooooo

When I first started learning to digitize analog video, the concept of luminance and brightness made sense. Waveform monitors made sense. A bright spot in a frame of video shows up as a spike on the waveform in way that appeases visual logic. When digitizing, you wanted to keep images from becoming too bright or too dark, lest visual details at these extremes be lost in the digital realm. All told, pretty straightforward.

Vectorscopes and chrominance/color information made less sense. There were too many layers of abstraction, and not just in “reading” the vectorscope and translating it to what I was seeing on the screen – there was something about the vocabulary around color and color spaces, full of ill-explained and overlapping acronyms (as I have learned, the only people who love their acronyms more than moving image archivists and metadata specialists are video engineers).

I’d like to sift through some of the concepts, redundancies, and labeling/terminology that threw me off personally for a long time when it came to color.

CIE XYZ OMG

Who in the rainbow can draw the line where the violet tint ends and the orange tint begins? Distinctly we see the difference of the colors, but where exactly does the one first blendingly enter into the other? So with sanity and insanity.

Herman Melville, Billy Budd

I think it might help if we start very very broad in thinking about color before narrowing in on interpreting color information in the specific context of video.

In the early 20th century, color scientists attempted to define quantitative links between the visible spectrum of electromagnetic wavelengths and the physiological perception of the human eye. In other words – in the name of science, they wanted to standardize the color “red” (and all the other ones too I guess)

Though we all know it’s the best one, right

Given the insanely subjective process of assigning names to colors, how do you make sure that what you and I (or more importantly, two electronics manufacturers) call “red” is the same? By assigning an objective system of numbers/values to define what is “red” – and orange, yellow, green, etc. etc. – based on how light hits and is interpreted by the human eye.

After a number of experiments in the 1920s, the International Commission on Illumination (abbreviated CIE from the French – like FIAF) developed in 1931 what is called the CIE XYZ color space: a mathematical definition of, in theory, all color visible to the human eye. The “X”, “Y” and “Z” stand for specific things that are not “colors” exactly, so I don’t even want to get into that here.

The important thing about the CIE XYZ color space – you can’t really “see” it. The limitations of image reproduction technology mean there will always be tangible limits standing between you and the full CIE XYZ gamut (outside of, maybe, a color scientist’s lab, but I’m not even convinced of that). Consider that graph above: even though it’s colored in for illustrative purposes, that’s not actually every color you can possibly see in there. The actual wavelengths of light produced, and therefore colors represented, by your computer’s LED screen encompasses a much much more limited range of values.

tl;dr your computer screen, compared to the mathematical limits of the natural world – sucks.

That’s to be expected though! Practical implementations of any standard will always be more limited than an abstract model.

*cough*

So even if it basically only exists in theory (and in the unbounded natural world around us), the CIE XYZ color space and its definition of color values served as the foundation for most color spaces that followed it. A color space is the system for creating/reproducing color employed by any particular piece of technology. Modern color printers, for example, use the CMYK color space: a combination of four color primaries (cyan, magenta, yellow, and “key”/black) that when mixed together by certain, defined amounts, create other colors, until you have hard copies of your beautiful PowerPoint presentation ready to distribute.

Again, just like CIE XYZ, any color space you encounter is math – it’s just a method of saying this number means orange and this number means blue and this number means taupe. But, just like with all other kinds of math, color math rarely stays that straightforward. The way each color space works, the way its values are calculated, and the gamut it can cover largely depend on the vagaries and specific limitations of the piece of technology it’s being employed with/on. In the case of CIE XYZ, it’s the human eye – in the case of CMYK, it’s those shitty ink cartridges on your desktop laser printer that are somehow empty again even though you JUST replaced them.

More lying liars who lie

So what about analog video? What are the specific limitations there?

Peak TV: Color in Analog Video

Video signals and engineering are linked pretty inextricably with the history of television broadcasting, the first “mass-market” application of video recording.

So tune in to what you know of television history. Much like with film, it started with black-and-white images only, right? That’s because it’s not just budding moving image archivists who find brightness easier to understand and manipulate in their practical work – it was the same for the engineers figuring out how to create, record, and broadcast a composite video signal. It’s much easier for a video signal to tell the electron gun in a CRT television monitor what to do if frame-by-frame it’s only working with one variable: “OK, be kinda bright here, now REALLY BRIGHT here, now sorta bright, now not bright at all”, etc.

Compare that to: “OK, now display this calculated sum of three nonlinear tristumulus primary component values, now calculate it again, and again, and oh please do this just as fast when we just gave you the brightness information you needed”.

So in first rolling out their phenomenal, game-changing new technology, television engineers and companies were fine with “just” creating and sending out black-and-white signals. (Color film was only just starting to get commonplace anyway, so it’s not like moving image producers and consumers were clamoring for more – yet!)

Who needs color video when you have these two, said America

But as we moved into the early 1950s, video engineers and manufacturing companies needed to push their tech forward (capitalism, competition, spirit of progress, yadda yadda), with color signal. But consider this – now, not only did they need to figure out how to create a color video signal, they needed to do it while staying compatible with the entire system and market of black-and-white broadcasting. The broadcast companies and the showmakers and the government regulation bodies and the consumers who bought TVs and everyone else who JUST got this massive network of antennas and cables and frequencies and television sets in place were not going to be psyched to re-do the entire thing only a few years later to get color images. Color video signal needed to work on televisions that had been designed for black-and-white.

From the CIE research of the ’20s and ’30s, video engineers knew that both the most efficient and wide-ranging (in terms of gamut of colors covered) practical color spaces were composed by mixing values based on primaries of red, green, and blue (RGB).

No no no no notorious

But in a pure RGB color space, brightness values are not contained on just one component, like in composite black-and-white video – each of the three primary values is a combination of both chrominance and luminance (e.g. the R value is a sum of two other values that mean “how red is this” and “how bright is this”, respectively). If you used such a system for composite analog video, what would happen if you piped that signal into a black-and-white television monitor, designed to only see and interpret one component, luminance? You probably would’ve gotten a weirdly dim and distorted image, if you got anything at all, as the monitor tried to interpret numbers containing both brightness and color as just brightness.

This is where differential color systems came into play. What engineers found is that you could still create a color composite video signal from three component primaries – but instead of those being chrominance/luminance-mixed RGB values, you could keep the brightness value for each video frame isolated in its own Y′ component (which Charles Poynton would insist I now call luma, for reasons of…..math), while all the new chroma/color information could be contained in two channels instead of three: a blue-difference (B′-Y′) and a red-difference (R′-Y′) component. By knowing these three primaries, you can actually recreate four values: brightness (luma) plus three chroma values (R, G, and B). Even though there’s strictly speaking no “green” value specified in the signal, a color television monitor can figure out what this fourth value should be based on those differential calculations.

For television broadcasting, keeping the luma component isolated meant that you could still pipe a color composite video signal into a black-and-white TV, and it would still just display a correct black-and-white image: it  just used the values from the luma component and discarded the two color difference components. Meanwhile, new monitors designed to interpret all three components correctly would display a proper color image.

This basic model of using two color difference components for video was so wildly efficient and successful that we still use color spaces based on this model today! Even as we passed from analog video signals into digital.

Making it possible with adapters and converters, to, say, play Fallout 4 on a B&W TV

Lookup Table: “YUV”

…but you may have noticed that I just said “basic model” and “color spaces”, as in plural. Uh oh.

As if this wasn’t all complicated enough, video engineers still couldn’t all just agree on one way to implement the Y′, B′-Y′, R′-Y′ model. Engineers working within the NTSC video standard needed something different than those working with PAL, who needed something different than SECAM. The invention of component video signal, where these three primaries were each carried on their own signal/cable, rather than mixed together in a single, composite signal, also required new adjustments. And digital video introduced even more opportunities for sophisticated color display.

I mean just look at this gamut

So you got a whole ton of related color spaces, each using the same color difference model, but employing different math to get there. In each case, the “scale factors” of the two color difference signals are adjusted to optimize the color space for the particular recording technology or signal in question. Even though they *always* basically represent blue-difference and red-difference components, the letters change because the math behind them is slightly different.

So here is a quick guide to some common blue/red difference color spaces, and the specific video signal/context they have been employed with:

  • Y′UV = Composite analog PAL video
  • Y′IQ = Composite analog NTSC video
  • Y′DBDR = Composite analog SECAM video
  • Y′PBPR = Component analog video
  • Y′CBCR = Component digital video – intended as digital equivalent/conversion of YPBPR, and also sometimes called “YCC”
  • PhotoYCC (Y′C1C2) = Kodak-developed digital color space intended to replicate the gamut of film on CD; mostly for still images, but could be encountered with film scans (whereas most still image color spaces are fully RGB rather than color-difference!)
  • xvYCC = component digital video, intended to take advantage of post-CRT screens (still relatively rare compared to Y′CBCR, though), sometimes called “x.v. Color” or “Extended-gamut YCC”

And now a PSA for all those out there who use ffmpeg or anyone who’s heard/seen the phrase “uncompressed YUV” or similar when looking at recommendations for digitizing analog video. You might be confused at this point: according to the table above, why are we making a “YUV” file in a context where PAL video might not be involved at all???

As Charles Poynton again helpfully lays out in this really must-read piece – it literally comes down to sloppy naming conventions. For whatever reason, the phrase “YUV” has now loosely been applied, particularly in the realm of digital video and images, to pretty much any color space that uses the general B′-Y′ and R′-Y′ color difference model.

I don’t know the reasons for this. Is it because Y′UV came first? Is it because, as Poynton lays out, certain pieces of early digital video software used “.yuv” as a file extension? Is it a peculiar Euro-centric bias in the history of computing? The comments below are open for your best conspiracy theories.

By which I mean Kirov conspiracy theories or GTFO

I don’t know the answer, but I know that in the vast vast majority of cases where you see “YUV” in a digital context – ffmpeg, Blackmagic, any other kind of video capture software – that video/file almost certainly actually uses the Y′CBCR color space.

Gamma Correction, Or How I Learned To Stop Worrying and Make a Keyboard Shortcut for “Prime”

Another thing that I must clarify at this point. As evidenced by this “YUV” debacle, some people are really picky about color space terminology and others are not. For the purposes of this piece, I have been really picky, because for backtracking through this history it’s easier to understand precise definitions and then enumerate the ways people have gotten sloppy. You have to learn to write longhand before shorthand.

So you may have noticed that I’ve been picky about using proper, capitalized subscripts in naming color spaces like Y′CBCR. In their rush to write hasty, mean-spirited posts on the Adobe forums, you may see people write things like “YCbCr” or “Y Cb Cr” or “YCC” or any other combination of things. They’re all trying to say the same thing, but they basically can’t be bothered to find or set the subscript shortcuts on their keyboard.

which, I mean, I don’t entirely blame ’em

In the same vein, you may have noticed these tiny tick marks (′) next to the various color space components I’ve described. That is not a mistake nor an error in your browser’s character rendering nor a spot on your screen (I hope). Nor is it any of the following things: an apostrophe, a single quotation mark, or an accent mark.

This little ′ is a prime. In this context it indicates that the luma or chroma value in question has been gamma-corrected. The issue is – you guessed it – math.

Curse you, Mathemagician

The human eye perceives brightness in a nonlinear fashion. Let’s say the number “0” is the darkest thing you can possibly see and “10” is the brightest. If you started stepping up from zero in precisely regular increments, e.g. 1, 2, 3, 4, etc., all the way up to ten, your eyes would weirdly not perceive these changes in brightness in a straight linear fashion – that is, you would *think* that some of the increases were more or less drastic than the others, even though mathemetically speaking they were all exactly the same. Your eyes are just more sensitive to certain parts of the spectrum.

Gamma correction adjusts for this discrepancy between nonlinear human perception and the kind of linear mathematical scale that technology like analog video cameras or computers tend to work best with. Don’t ask me how it does that – I took Statistics, not Calculus.

The point is, these color spaces adjust their luma values to account for this, and that’s what the symbol indicates. It’s pretty important – but then again, prime symbols are usually not a readily-accessible shortcut that people have figured out on their keyboards. So they just skip them and write things like… “YUV”. And I’m not just talking hasty forum posts at this point – I double-dare you to try and reconstruct everything I’ve just told you about color spaces from Wikipedia.

Which, come to think of it, the reason why “YUV” won out as a shorthand term for Y′CBCR may very well be just that it doesn’t involve any subscripts or primes. Never underestimate ways programmers will look to solve inefficiencies!

Reading a Vectorscope – Quick Tips

This is all phenomenal background information, but when it gets down to it, a lot of people digitizing video just want to know how to read a vectorscope so they can tell if the color in their video signal is “off”.

Here we have a standard SMPTE  NTSC color bars test pattern as it reads on a properly calibrated vectorscope. Each of the six color bars (our familiar red, green, and blue, as well as cyan, yellow, and magenta) are exactly as they should, as evidenced by the dots/points falling right into the little square boxes indicated for them (the white/gray bar should actually be the point smack in the center of the vectorscope, since it contains no chroma at all).

The saturation of the signal will be determined by how far away from the center of the vectorscope the dots for any given frame or pixel of video fall. For example, here is an oversaturated frame – the chroma values for this signal are too high, which you can tell because the dots are extending out way beyond the little labeled boxes.

Meanwhile, an undersaturated image will just appear as a clump towards the center of the vectorscope, never spiking out very far (and a black-and-white image should in fact only do that!)

Besides saturation, a vectorscope can also help you see if there is an error in phase or hue (both words refer to the same concept). As hopefully demonstrated by our discussion of color spaces and primaries, colors generally relate to each other in set ways – and the circular, wheel-like design of the vectorscope reflects that. If the dots on a vectorscope appear rotated – that is, dots that should be spiking out towards the “R”/red box, are instead headed for the “Mg”/magenta box – that indicates a phase error. In such a case, any elephants you have on screen are probably going to start all looking like pink elephants.

The problem is, even with these “objective” measurements, color remains a stubbornly subjective thing. Video errors in saturation and phase may not be so wildly obvious as the examples above, making them a tricky thing to judge, especially against artistic intent or the possibility of an original recording being made with mis-calibrated equipment. Again, you can find plenty of tutorials and opinions on the subject of color correction online from angry men, but I always just tell everyone to use your best judgement. It’s as good as anyone else’s!

XKCD #1492

(If you do want or need to adjust the color in a video signal, you’ll need a processing amplifier.)

Pixel Perfect: Further Reading on Color

I’ve touched on digital video and color here but mostly focused on pre-digital color spaces and systems for audiovisual archivists and others who work with analog video signals. Really, that’s because all of this grew out of a desire to figure out what was going on with “uncompressed YUV” codecs and “-pix_fmt yuv420p” ffmpeg flags, and I worked backwards from there until I found my answers in analog engineering.

There’s so much more to get into with color in video, but for today I think I’m going to leave you with some links to explore further.

From the master of pixels, Charles Poynton:

On the subject of chroma subsampling:

 

On the subject of other color spaces beyond Y′CBCR

  • openYCoCg
    An experimental video codec from AV Preservation by Reto Kromer, based on the Y′COCG color space, which can convert to/from RGB values but isn’t based on human vision models (using orange-difference and green-difference instead of the usual blue/red)
  • Digital Photography School: sRGB versus Adobe RGB

 

XKCD #1882

About Time: Sifting Through Analog Video Terminology

In case it’s ever unclear, writing these blog posts is as much for me as it is for you, dear reader. Technical audiovisual and digital concepts are hard to wrap your head around, and for a myriad of reasons technical writing is frequently no help. Writing things out, as clearly and simply as I can, is as much a way of self-checking that I actually understand what’s going on as it is a primer for others.

And it case this intro is unclear, this is a warning to cover my ass as I dive into a topic that still trips me up every time I try to walk through it out loud. I’ll provide references as I can at the end of the piece, but listen to me at your own risk.

When I first started learning about preserving analog video, *most* of what I was reading/talking about made a certain amount of sense, which is to say it at least translated into some direct sensory/observable phenomenon – luminance values made images brighter or darker, mapping audio channels clearly changed how and what I was hearing, using component video rather than composite meant mucking around with a bunch more cables. But when we talked about analog signal workflows, there always seemed to be three big components to keep track of: video (what I was seeing), audio (what I was hearing) and then this nebulous goddamn thing called “sync” that I didn’t understand at all but seemed very very important.

The same way I feel when anyone tells me “umami” is a thing WHAT ARE YOU

I do not know who to blame for the morass of analog video signal terms and concepts that all have to do in some form with “time” and/or “synchronization”. Most likely, it’s just an issue at the very heart of the technology, which is such a marvel of mechanical and mathematical precision that even as OLED screens become the consumer norm I’m going to be right here crowing about how incredible it is that CRTs ever are/were a thing. What I do know is that between the glut of almost-but-not-quite-the-same words and devices that do almost-but-not-quite-the-same-thing-except-when-they’re-combined-and-then-they-DO-do-the-same-thing, there’s a real hindrance to understanding how to put together or troubleshoot an analog video digitization setup. Much of both the equipment and the vocabulary that we’re working with came from the needs of production, and that has led to some redundancies that don’t always make sense coming from the angle of either preservation or casual use.

I once digitized an audio cassette from the ’70s on which an audio tech was testing levels (…I hope?) by just repeating “time time time time” about 100 times in a row and ever since that day I’ve feared that his unleashed soul now possesses me. Anyway.

So I’m going to offer a guide here to some concepts and equipment that I’ve frequently seen either come up together or understandably get confused for each other. That includes in particular:

  • sync and sync generators
  • time base correction
  • genlock
  • timecode

…though other terms will inevitably rise as we go along. I’m going to assume a certain base level of knowledge with the characteristics and function of analog video signal – that is, I will write assuming the reader has been introduced to electron guns, lines, fields, frames, interlacing, and the basics of reading a classic/default analog video waveform. But I will try not to ever go right into the deep end.

Sync/Sync Generators

I know I just said I would assume some analog video knowledge, but let’s take a moment to contemplate the basics, because it helps me wrap my head around sync generators and why they can be necessary.

Consider the NTSC analog video standard (developed in the United States and employed in North, Central and parts of South America). According to NTSC, every frame of video has two fields, each consisting of 262.5 scan lines – so that’s 525 scan lines per frame. There are also 29.97 frames of video per second.

That means the electron gun in a television monitor has to fire off 15734.25 lines of video PER SECOND – and that includes not just actually tracing the lines themselves, but the time it takes to reset both horizontally (to start the next line) and vertically (to start the next field).

Now throw a recording device into the mix. At the same time that a camera is creating that signal and a monitor is playing it back, a VTR/VCR has to receive that signal and translate it into a magnetic field stored on a constantly-moving stretch of videotape via an insanely quick-spinning metal drum.

Even in the most basic of analog video signal flows, there are multiple devices (monitor, playback/recording device, camera, etc.) here that need to have an extremely precise sense of timing. The recording device, for instance, needs to have a very consistent internal sense of how long a second is, in order for all its tiny little metal and plastic pieces to spin and roll and pulse in sync (for a century, electronic devices, either analog OR digital, have used tiny pieces of vibrating crystal quartz to keep track of time – which is just a whole other insane tangent). But, in order to achieve the highest quality signal flow, it also needs to have the exact same sense of how long a second is as the camera and the monitor that it’s also hooked up to.

When you’re dealing with physical pieces of equipment, possibly manufactured by completely different companies at completely different times from completely different materials, that’s difficult to achieve. Enter in sync signal and sync generators.

Stay with me here

Sync generators essentially serve as a drumbeat for an analog video system, pumping out a trusted reference signal that all other devices in the signal chain can use to drive their work instead of their own internal sense of timing. There are two kinds of sync signals/pulses that sync generators have historically output:

Drive pulses were almost exclusively used to trigger certain circuits in tube cameras, and were never part of any broadcast/recorded video signal. So you’re almost certainly never going to need to use one in archival digitization work, but just in case you ever come across a sync generator with V. Drive and H. Drive outputs (vertical drive and horizontal drive pulses), that’s what those are for.

Blanking pulses cause the electron gun on a camera or monitor to go into its “blanking” period – which is the point at which the electron gun briefly shuts off and retraces back to the beginning of a new line (horizontal blanking) or of a new field (vertical blanking). These pulses are a part of every broadcast/recorded video signal, and they must be extremely consistent to maintain the whole 525-scan-lines-29.97-times-per-second deal.

Since blanking pulses are the vast majority of modern sync signals, you may also just see these referred to as “sync pulses”.

So the goal of sync generators is to output a constant video signal with precise blanking pulses that are trusted to be exactly where (or to be more accurate, when) they should be. Blanking pulses are contained in the “inactive” part of a video signal, meaning they are never meant to actually be visualized as image on a video camera or monitor (unless for troubleshooting purposes) – so it literally does not matter what the “active” part of a sync generator’s video signal is. It could just be field after field, frame after frame of black (often labeled “black burst”).  It could be some kind of test pattern – so you will often see test pattern generators that double as or are used as sync generators, even though these are separate functions/purposes, and thus could also be entirely separate devices in your setup.

Color bars – not *just* for geeky yet completely hard-to-use desktop wallpapers

(To belabor the point, test patterns, contained in the “active” part of the video signal, can be used to check the system, but they do not drive the system in the same way that the blanking pulses in the “inactive” part of a signal do. IMHO, this is an extremely misleading definition of “active” and “inactive”, since both parts are clearly serving crucial roles to the signal, and what is meant is just “seen” or “unseen”.)

Here’s the kicker – strictly speaking, sync generators aren’t absolutely necessary to a digitization station. It’s entirely possible that you’ll hook up all your components and all the individual pieces of equipment will work together acceptably – their sense of where and when blanking pulses should fall might already be the same, or close enough to be negligible.

But maybe you’re seeing some kind of  a wobbling or inconsistent image. That *could* be a sync issue, with one of your devices (the monitor, e.g.) losing its sense of proper timing, and resolvable by making sure all devices are working off the same, trusted sync generator.

You’ll see inputs for sync signal labeled in all sorts of inconsistent ways on analog video devices: “Sync In”, “Ext Sync”, “Ext Ref”, “Ref In”, etc. As far as I’m aware, these all mean the same thing, and references in manuals or labels to “sync signal” or “reference signal” should be treated interchangeably.

Time Base Correction

Sync generators can help line up devices with inconsistent timing in a system as a video signal is passed from one to another. Time Base Correctors (TBCs) perform an extremely similar but ever-so-confusingly-different task. TBCs can take an input video signal, strip out inconsistent sync/blanking pulses, and replace them entirely with new, steady ones.

This is largely necessary when dealing with pre-recorded signals. Consider a perfectly set up, synced recording system: using a CRT for the operator to monitor the image, a video camera passed a video signal along to a VTR, which recorded that signal on to magnetic tape. At the time, a sync generator made sure all these devices worked together seamlessly. But now, playing back that tape for digitization, we’ve introduced the vagaries of magnetic tape’s physical/chemical makeup into the mix. Perhaps it’s been years and the tape has stretched or bent, the metallic particles have expanded or compressed – not enough to prevent playback, but enough to muck with the sync pulses that rely on precision down to millionths of a second. As described with sync loss above, this usually manifests in the image looking wobbly, improperly framed or distorted on a monitor, etc.

https://bavc.github.io/avaa/artifacts/time_base_error.html

TBCs can either use their own internal sense of time to replace/correct the timing of the sync pulses on the recorded video signal – or they can use the drumbeat of a sync generator as a reference, to ensure consistency within a whole system of devices.

(Addendum): A point I’m still not totally clear on is the separation (or lack thereof) between time base correction and frame synchronization. According to the following video I found, a stand-alone frame synchronizer could stabilize the vertical blanking pulses in a signal only, resulting in a solid image at the moment of transition from one frame to another (that is, the active video image remains properly centered vertically within a monitor), but did nothing for horizontal sync pulses, potentially resulting in line-by-line inconsistencies:

So time base correction would appear to incorporate frame synchronization, while adding the extra stabilization of consistent horizontal sync/blanking pulses.

While, as I said above, you can very often get away with digitizing video without a sync generator, TBCs are generally much more critical. It depends on what analog video format you’re working with exactly, but whereas the nature of analog devices meant there was ever so slight leeway to deal with those millionth-of-a-second inconsistencies (say, to display a signal on a CRT), the precise on-or-off, one-or-zero nature of digital signals means analog-to-digital converters usually need a very very steady signal input to do their job properly.

You may however not need an external TBC unit. Many video playback decks had them built in, though of varying quality and performance. If you can, check the manual for your model(s) to see if it has an internal TBC, and if so, if it’s possible to adjust or even turn it off if you have a more trustworthy external unit.

Professional-quality S-VHS deck with built-in TBC controls

Technically there are actually three kinds of TBCs: line, full frame and full field. Line TBCs can only sample, store and correct errors/blanking pulses for a few lines of video at a time. Full field TBCs can handle, as the name implies, a full 262.5 (NTSC) lines of video at a time, and full frame TBCs can take care a whole frame’s worth (525, NTSC) of lines at a time. “Modern”, late-period analog TBCs are pretty much always going to be full frame, or even capable of multiple frames’ worth of correction at a time (this is good for avoiding delay and sync issues in the system if working without a sync generator). This will likely only come into play with older TBC units.

And here’s one last thing that I found confusing about TBCs given the particular equipment I was trained on: this process of time base correction, and TBCs themselves, has nothing to do with adjusting the qualities of the active image itself. Brightness, saturation, hue – these visible characteristics of video image are adjusted using a device called a processing amplifier or proc amp.

Because the process of replacing or adjusting sync pulses is a natural moment in a signal flow to ALSO adjust active video signal levels (may as well do all your mucking at once, to limit troubleshooting if something’s going wrong), many external TBC units also contain proc amps, thus time base correction and video adjustments are made on the same device, such as the DPS TBC unit above. But there are two different parts/circuit boards of the unit that are doing these two tasks, and they can be housed in completely separate devices as well (i.e. you may have a proc amp that does no time base correction, or a TBC that offers no signal level adjustments).

Genlock

“Genlock” is a phrase and label that you may see on various devices like TBCs, proc amps, special effects generators and more – often instead of or in the same general place you would see “Sync” or “Ref” inputs. What’s the deal here?

This term really grew out of the production world and was necessary for cases where editors or broadcasters were trying to mix two or more input video signals into one output. When mixing various signals, it was again good/recommended practice to choose one as the timing reference – all the other input signals, output, and special effects created (fades, added titles, wipes, etc. etc.) would be “locked” on to the timing of the chosen genlock input (which might be either a reference signal from a sync generator, or one of the actually-used video inputs). This prevented awkward “jumps” or other visual errors when switching between or mixing the multiple signals.

In the context of a more straightforward digitization workflow, where the goal is not to mix multiple signals together but just pass one steadily all the way through playback and time base correction to monitoring and digitization, “genlock” can be treated as essentially interchangeable with the sync signal we discussed in the “Sync/Sync Generators” section above. If the device you’re using has a genlock input, and you’re employing a sync generator to provide an external timing reference for your system, this is where you would connect the two.

Timecode

The signals and devices that I’ve been describing have been all about driving machinery – time base is all about coordinating mechanical and electrical components that function down at the level of milliseconds.

Timecode, on the other hand, is for people. It’s about identifying frames of video for the purpose of editing, recording or otherwise manipulating content. Unlike film, where if absolutely need be a human could just look at and individually count frames to find and edit images on a reel, magnetic tape provides no external, human-readable sense of what image or content is located where. Timecode provides that. SMPTE timecode, probably the most commonly used standard, identified frames according to a clock-based system formatted “00:00:00:00”, which translated to “Hours:Minutes:Seconds:Frames”.

Timecode could be read, generated, and recorded by most playback/recording/editing devices (cameras or VTRs), but there were also stand-alone timecode reader/generator boxes created (such as in the photo above) for consistency and stability’s sake across multiple recording devices in a system.

There have been multiple systems of recording and deploying timecode throughout the history of video, depending on the format in question and other concerns. Timecode was sometimes recorded as its own signal/track on a videotape, entirely separate from the video signal, in the same location as one might record an audio track. This system was called Linear Timecode (LTC). The problem with LTC was, like with audio tracks, the tape had to be in constant motion to read the timecode track (like how you can not keep hearing an audio signal when you pause it, even though you *can* potentially keep seeing one frame of image when a video signal is paused on one line).

Vertical Interval Timecode (VITC) fixed this (and had the added benefit of freeing up more physical space on the tape itself) by incorporating timecode into video signal itself, in a portion of the inactive, blanking part of the signal. This allowed systems to read/identify frame numbers even when the tape was paused.

Other systems like CTL timecode (control track) and BITC (timecode burnt-in to the video image) were also developed, but we don’t need to go too far into that here.

As long as we’re at it, however, I’d also like to quickly clarify two more terms: drop-frame and non-drop-frame timecode. These came about from a particular problem with the NTSC video standard (as far as I’m aware, PAL and SECAM do not suffer the same, but please someone feel free to correct me). Since NTSC video plays back at the frustratingly non-round number of 29.97 frames per second (a number arrived at for mathematical reasons beyond my personal comprehension that have something to do with how the signal carries color information), the timecode identifying frame numbers will eventually drift away from “actual” clock time as perceived/used by humans. An “hour of timecode” which by necessity must count upwards at a 30 fps rate such as:

00:00:00:28
00:00:00:29
00:00:01:00
00:00:01:01

played back at 29.97 fps, will actually be 3.6 seconds longer than “wall-clock” time.

That’s already an issue at an hour, and proceeds to get worse the longer the content being recorded. So SMPTE developed “drop-frame” timecode, which confusingly does not drop actual frames of video!! Instead, it drops some of the timecode frame markers. Using drop-frame timecode, our sequence would actually proceed thus:

00:00:00;28
00:00:00;29
00:00:01;02
00:00:01;03

With the “:00” and “:01” frame markers removed entirely from the timecode; except for every tenth minute, when they are re-inserted to make the math check out:

00:00:09;28
00:00:09;29
00:00:10;00
00:00:10;01

Non-drop-frame timecode simply proceeds as normal, with the potential drift from clock time. Drop-frame timecode is often (but not necessarily – watch out) identified in video systems using semi-colons or single periods instead of colons between the second and frame counts, as I have done above. Semi-colons are common on digital devices, while periods are common on VTRs that didn’t have the ability to display semi-colons.

I hope this journey through the fourth dimension of analog video clarifies a few things. While each of these concepts is reasonable enough on their own, the way they relate to each other is not always clear, and the similarity in terms can send your brain down the wrong alley quickly. Happy digitizing!

Resources

Andre, Adam. “Time Base Corrector.” Written for NYU-MIAP course in Video Preservation I, Fall 2017. Accessed 6/9/2018. https://www.nyu.edu/tisch/preservation/program/student_work/2017fall/17f_3403_Andre_a1.pdf

“Genlock: What is it and why is it important?” Worship IMAG blog. Posted 6/11/2011. Accessed 6/8/2018. https://worshipimag.com/2011/06/11/gen-lock-what-is-it-and-why-is-it-important/

Marsh, Ken. Independent Video. San Francisco, CA: Straight Arrow Books, 1974.

Poynton, Charles. Digital Video and HD: Algorithms and Interfaces. 2nd edition. Elsevier Science, 2012.

SMPTE timecode. Wikipedia. Accessed 6/8/2018.

Weise, Marcus and Diana Weynand. How Video Works: From Analog to High Definition. 2nd Edition. Burlington, MA: Focal Press, 2013.