I’ve continued working on the code to drive LED signals with an Arduino. I’d previously discussed my approach, and provided the code I was using at that time. I’ve learned a bit since then, and cleaned up the code significantly. I’ll provide a link to the current example program at the end of this post.

Fundamentally nothing has changed. I’m still planning to use NJI common-anode SMD LED signals (in fact, I’ve ordered them). What I did do was change the code so that a “bank” of two signals would always have both lit (meaning two of the four LEDs would be on when the bank was active) so that I could get through the full set of signals more quickly. One reason for this has to do with video camera shutter speeds. I think it’s worth saying a bit about that issue.

I’ve also made some changes to make the time wasted in turning the pins on and off less, since at these speeds that is becoming a significant percentage of the total LED cycle, and I need that time for the eventual Tram Controller program to be doing other things. These changes consisted of adding a library that provides faster versions of writeDigital and pinMode, as well as keeping track of what state pins are on, and not trying to change them unless the new state differs (this got rid of a number of “change disabled pin X to disabled” changes).

In my test program, when cycling at 8 milliseconds, I’m now spending just a quarter millisecond changing those pins with three banks (6 signals) in use. My One Point Five Meter line will only use four signals, as it doesn’t have the extended double-track section of the full Tram Line, which needs six. And so it will run even more efficiently.

Persistence of (Machine) Vision

As I’d noted in the earlier post, to fool the human eye into thinking that the LEDs are continuously lit rather than being pulsed, I need to cycle through the whole set of LEDs in less than about 20 milliseconds (msec). Slower than that, and the LEDs will seem to flicker slightly, with the effect becoming more pronounced as the times get longer (25 msec is noticeable, 30 is quite obvious).

But cameras aren’t eyes. Cameras take a picture every 41 msec (film), 33 msec (normal 30 frames-per-second, or fps, video), or 16.7 msec (60 fps video). The eye is fooled by these into seeing continuous motion. But those pictures are snapshots of an instant in time, displayed for the length noted above (or some fraction of it). If something happens during that “window” of 16.7, 33 or 41 msec but not in the instant recorded, the camera can’t display it (because it wasn’t recorded), and the eye won’t see it, so “persistence” won’t have a chance to act.

The length of that “snapshot” depends on the camera’s shutter speed, which turns out to be more complex to determine than you might think. But if you want to take video of the layout with operating signals (or other charlieplexed or multiplexed LEDs) and not have them flickering, it’s an important question to answer. And I do want to take such video.

Camera Sensors and Shutters

Video cameras today use two different kinds of sensors to record the light coming through the lens: CMOS and CCD. There’s an excellent description of them online, so I won’t do more than summarize the points that matter here. CMOS sensors are typically used with a “Rolling Shutter”, while CCD sensors are used with a “Global Shutter”.

Global shutters (CCD) expose the whole video frame for the duration of the image-taking. Rolling shutters (CMOS) divide the total time up and expose portions of the frame for shorter times, with the total fitting within the total time the frame is “exposed”. Rolling shutters, as a result, don’t work as good for video with a lot of motion. And CMOS sensors can exhibit other problems like smearing of bright lights in a line (sort of like lens flare, but in one line). As usual, you get what you pay for. Cheaper cameras tend to use CMOS sensors and rolling shutters, and have a number of problems with video artifacts. CCD sensors (with global shutters) cost more and work better, and their biggest problem (motion blur) makes their output look more like that of a film camera, which is generally counted as a plus.

In a CMOS sensor, the rolling shutter is actually an electronic rolling shutter, not a physical mechanism. A CMOS-equipped still camera may have a physical shutter in addition, but a camera used in “live view” more or for video will typically use only the electronic shutter. In an electronic shutter, the circuitry used to read information off the sensor is re-used, possibly on a line-by-line basis. The total time light is allowed to accumulate on each line is the same, but the first line will start and end its exposure well before the last one does.

As an aside, there has been a lot of investment in CMOS sensors due to reasons beyond this post, and the result is that they are cheaper and in some ways superior to CCDs and thus very commonly used in all kinds of applications. It’s not strictly “CMOS is bad”. In particular, CMOS does better in low-light conditions (usually).

In fact, even high-end Digital SLRs use CMOS sensors today: my Canon 5D mark III has one. So unless you’re a professional videographer, you’ll be dealing with rolling shutters. For LED driving, I’m not sure it matters. The problems I’ve seen related to timing don’t appear to be caused by the rolling shutter, but by the speed of the shutter itself. You could, however, see some odd behavior when panning with a rolling shutter. Some cameras do better at avoiding such problems than others, but I don’t think there’s going to be much you can do with the LED timing to address that.

Since a rolling shutter could only be active for a fraction of the “shutter speed” in each segment, it might effectively have a faster shutter speed than would be assumed from the time available to take a picture (e.g., in a 16.7 msec window, perhaps each row is only active for 8 msec). However, at least for my two cameras, this does not appear to be the case, as their behavior fit with the known or estimated shutter speeds they were using, and did not reflect a faster one.

So while there is reason to prefer a CCD sensor for video generally, the type of sensor really isn’t relevant to my concern over how quickly to cycle through my LEDs.

Frame Speed and Shutter Speed

Both kinds of sensor/shutters can have problems with lights that vary periodically. The typical culprit is older fluorescent lighting that flickered in time with the AC power frequency (1/60-second in the U.S., 1/50-second in some other locales). And the fix (for that anyway) was to run the shutter at the same speed as the lighting (i.e., 60 fps in the U.S., 50 fps in some other places). NTSC video uses 30 fps, and is still widely used, but it too is synchronized to 1/60-second lighting, just at half speed.

For cameras with settable shutter speeds, the recommendation is to use a shutter that’s half the duration of the interval between frames (the “180 degree rule”). For 30 fps that would be 1/60-second (16.7 msec), and for 60 fps it would be 1/120, typically rounded to 1/125-second (8 msec). For PAL video, 25 fps equates to 1/50-second (20 msec).

But cheaper cameras won’t let you set the shutter speed. They’ll set one internally based on the frame rate (e.g., 30 fps) and their own determination of exposure requirements. From my limited testing, this can vary up to the full duration of the frame, or close to it. My Sony camera, under dim lighting conditions, was clearly using a shutter close to 1/60-second (16.7 msec) even though it was recording at 60 fps.

What this all means is that depending on a camera, I could get away with a cycle speed of 16.7 msec, but not the 20 msec that will work for the human eye. The camera is always going to see flicker the eye misses. And if you shoot video at 60 fps, some cameras will need a cycle speed of 8 msec or less to avoid flicker (the equivalent for 50 fps video would be 10 msec).

Just to be sure, I ran some experiments to confirm that this was how it really worked.

Experimentation

I did some testing using a Canon 5D mark III DSLR with a CMOS sensor and the ability to set a specific shutter speed, and a Sony HD CX100 camcorder, also with a CMOS sensor, but only able to operate at a fixed 60 fps with an internally-set shutter speed (the specs say maximum shutter speed is 1/125-second, but I’m not sure they’re accurate).

Both of these use rolling shutters, but both apparently take steps to minimize the effect those have, although exactly what they are doing is unclear. I expect it’s some kind of “deskewing” to fix distorted vertical lines, and doesn’t eliminate the fact that the pixels aren’t on for the full shutter interval. But that doesn’t jibe with what I saw on the Sony, which seemed to be using the full window to record the LEDs (which weren’t moving, so they should only have been recorded for part of the time).

What I saw were two things: one expected and one a surprise. First, the LEDs behaved exactly as I expected with the Canon, showing a serious pulsing appearance when I set the cycle duration above the shutter interval (e.g., a 12 msec cycle with 1/125-second, or 8 msec, shutter), but appearing to be continuously lit for cycle speeds faster than it (8 msec or less).

What surprised me though was that at a cycle duration of 2.1 msec and a shutter speed of 8 msec, I saw some faint pulsing. This occurred with other combinations of shutter speed and cycle duration. My suspicion is that here I’m seeing a “beat frequency” effect, where multiples of the two work together to make it miss the LED every few frames (2.1 to 8 would be around 4 frames). The LED won’t appear to go fully dark, but its perceived intensity will vary. If that’s the case, the way to avoid it is to get the cycle duration as close to the shutter speed as possible, which is going to be problematic for cameras that don’t have a fixed shutter speed.

In order to test my cameras against the sensor-driving code, I shot a number of short clips at different cycle durations and camera settings. I’ve assembled some of these into a short video to illustrate the effect. Be aware that YouTube’s compression makes very small flickering of the LEDs hard to see (e.g., in the 2.1 msec cycle clip there’s a moderate flicker in my original video, but it’s really hard to see it in the posted copy).

Concluding Thoughts

I don’t really know what shutter speed my Sony is selecting. It it’s trying to avoid problems with older fluorescent lighting, it’s probably going to use multiples of 1/60-second exposure. And if the online specifications I found for it are accurate, that means it will run at either 1/60 (16.7 msec) or 1/125 (8 msec) and not any intermediate or faster speeds. So while I saw it working at 16 msec cycles, that may have been due to the fact that I was using low levels of light in my test (which was at my computer desk, a location I don’t have brightly lit).

My Canon on the other hand let me choose the shutter speed. I could have pushed the ISO higher and shot at 1/250 (4 msec) if I wanted to.

My takeaway is that I want to aim for an 8 msec cycle time on the LEDs (or just a hair under 8 msec to be safe), and that further to avoid what I think is the “beat frequency” problem, I need to avoid even divisors of that frequency (i.e., not 4, 2 or 1 msec cycles). I also want to avoid even divisions of 16.7 msec, which means avoiding 4.2 msec and 2.1 msec. So I plan to attempt to cycle my LEDs in around 7.5 milliseconds (although 3.5 would be nice, I don’t think it’s achievable). This is easy to do with my test program. But cycling that quickly with my big program, which has a lot more to do each time around the loop, is going to be more of a challenge.

The Example Code

I’ve created a simple test sketch, that I used to drive the Arduino in my test video (download it here). It has a lot of extra code in it for debugging and reporting timings, but you can extract the basic stuff to drive the LEDs fairly easily if you want to use it yourself. The wiring for the charlieplexing is described on my Arduino LEDs page, so I won’t repeat that part here.

The sketch is designed to use a library for fast manipulation of pins. See the comments at the top of the sketch to download the library, or to disable its use if you don’t want to use it. With six signals, the savings amounts to roughly 3 x 28 microseconds per cycle, which is just over a 1% of the cycle time, so it’s not really crucial to use it. It is a substantial fraction of the time spent on pins though, so if the program is getting busy and you need every microsecond, this will be helpful.

Tags: Arduino, Signals