Optical Flow vs Frame Blending: What are the Main Differences?

Working with slow-motion footage can feel like a struggle between creating the perfect shot and retiming the videos while maintaining the best quality possible. There are many things to consider to make slow-motion effects look good, and the most crucial one is shooting your videos at higher frame rates and with fast shutter speeds.

Frame rate and shutter speeds are important when you slow down a video because your software will stretch the frames to play it slower and extend the duration of the video. However, when it stretches the video clip, there will be missing frames in between if the video is in a low fame rate.

The solution in software like Premiere Pro is simply duplicating frames to fill the space between the available frames. This process is called frame sampling, but the result is far from slow motion and more like a choppy or stop motion video.

Adobe Premiere Pro, as well as other software like Final Cut Pro, DaVinci Resolve, and After Effects, comes with retiming and frame interpolation settings to create smooth slow motion, fast motion, speed ramps, time-lapse and other retiming effects for your video clips.

There are a few options to add smooth, slow-motion effects to your videos at low frame rates and slow shutter speeds without sacrificing quality. The most popular are frame sampling, frame blending, and optical flow.

In today's article, we'll explore frame blending vs optical flow. I'll highlight the main differences and help you decide how to choose between the two.

Let’s dive in!

What is Optical Flow?

If you search for slow-motion techniques, you often come across the term optical flow. This option is available in Adobe Premiere Pro and the latest version of other video editing programs and dedicated slow-motion plug-ins.

Several content creators and video editors affirm optical flow is the best method to add slow motion. But others say it can do wonders if you can make it work.

For video footage with low frame rates and shutter speeds, video editing software will try to fill in missing frames with different methods. Optical flow will look at the whole video, not just in the middle of the two frames where additional frames are missing. Based on that analysis, it'll do some very advanced calculations and estimations to create new frames in between.

Imagine a little machine scanning each frame from your footage and analyzing how each pixel in the image moves through the sequence. It will examine the trajectory of every individual pixel and come up with a very accurate estimation of where those pixels would be in the missing frames. Then, use that data to create new frames, making up for the missing ones.

The more data your video has, such as more frames, higher resolutions, good lighting, clean shots, etc., the better, as optical flow will have more data to produce new frames. When the video lacks information, the optical flow will add artifacts and warps, resulting in an unnatural slow-motion effect.

Many editors say optical flow adds artifacts to their clips because even the smartest AI will not be able to work with such little information and will work with what it has, trying to come up with something somewhat pleasantly.

What is Frame Blending?

When things don't work out with optical flow, frame blending is the second most used method to create smooth, slow-motion effects. It is less heavy in processing than optical flow but can often get similar or even better results. Most of the time, you'll have to try and see which option works for each video.

Frame blending is used when slowing down videos to a percentage that will cause them to lose frames because they have a lower frame rate or the video was slowed down over its limit.

The process of frame blending works similarly to a transition. Transitions take two frames and composite them over the other to create a seamless transition. When you use crossfade or dissolve transition, what it does is mix the two frames and overlap them to make the transition look smoother. One frame disappears as the other appears.

This is exactly what Premiere Pro does when using frame blending. It takes two frames in the slow-motion clip and adds a transition between the two. The stretched frames are filled with small dissolves in between as if placed between every two frames, but frame blending does it automatically.

Frame blending provides a smoother slow-motion effect than frame sampling, but its analysis is less comprehensive than the one used by optical flow. Frame blending sits in between optical flow and Frame Sampling: it won’t duplicate frames, but it won’t generate new frames either.

Like optical flow, frame blending will benefit the most from footage with enough data to analyze and frames to dissolve. The more frames, the fewer artifacts and blurriness it will add to the footage.

Main Difference Between Optical Flow and Frame Blending

The main difference in this optical flow vs frame blending battle lies in how each one processes the video footage and what they do to handle the missing frames in your video.

Frame blending takes two frames and blends them with a cross dissolve to fill the gaps between frames. These transitions between frames help refine the slow-motion effect but can occasionally create a blurry image.

Optical flow, on the other hand, analyzes the video and estimates where the image moves to generate those missing frames in the gaps. The results with optical flow will be smoother and more natural. However, if the footage is low quality, it can distort the image.

Both techniques can efficiently create a slow-motion effect, so I'm sure either one of these software tools will meet your demands.

How to Choose from Frame Blending and Optical Flow

Frame blending vs optical flow is more of a trial-and-error process. With practice, you can learn what type of videos will suit optical flow and which one's frame blending. In the meantime, I summarize a few scenarios where you would prefer to start with one over the other, depending on the footage.

• Dealing with Natural Motion Blur

  When your video clip presents motion blur, either natural or caused by editing or other effects, you can get better results for slow motion using frame blending. The blur added with frame blending will be less noticeable if there is already some blur.

  

  For optical flow, the blur can confuse its algorithm, and the estimation will be out of place, resulting in a distorted image.

  Winner: Frame blending.

• Low Frame Rate Footage

  Video footage with fewer frames will struggle more with frame blending.

  

  When frames are stretched, the crossfade may fill the gaps, but the looks of the video may turn as smooth as it could be. Optical flow will smooth the slow-motion sequence better with new frames.

  Winner: Optical flow.

• Retiming Speed

  When you slow down the video from 100% to up to 50%, either frame blending or optical flow should do well with little difference.

  

  Things start to change as you drop the speed to 30%, 20%, 10%, etc. Under these circumstances, the frames available will be more limited, and frame blending will be more obvious. For speeds lower than 50%, try optical flow first.

  Winner: Both might work, but optical flow would fill in the gaps more naturally if the speed is lower than the available frames.

Final Words

There are no rules on how and when to use frame blending or optical flow. It all boils down to understanding how each one works and experimenting with them on different videos to see how each affects different types of content.

Personally, what I do is try both techniques on a few seconds of the clip I'm working on and render that section to see the results. Based on what I see, I make decisions on which technique to use. For instance, if optical flow doesn't look good, I'll switch to frame blending or frame sampling.

Over time, as you gain more experience, you'll learn to think ahead when you're shooting to start making the proper settings in your camera so your footage is ready for optical flow or frame blending in post-production.

Good luck!