I use Sony Vegas to do my video editing and latest versions allow you to create your own visual effect plugins, quite easily using the OFX standard ( Basically that’s a C++ based DLL ( 32/64 bit version) on Windows ).

You can create many type of effects but the one I was interest with here was a Video Event FX ( video clip are called “video event” inside Vegas ) . The OFX framework gives you the ability to declare which kind of pixel formats ( RGB(A) , byte / float , etc.. ) you support and then ask for the memory buffer holding the image at any given time. You then have to produce your resulting image buffer and give it back to Vegas that will use it for display ( or for any other FX that is applied to your video clip in the timeline )

Most of effects you find inside Vegas or other editing software will apply to the image at frame F of a given video clip ( For example a blur effect will , read current frame image , apply a blur filter,  output the resulting image ).

What I wanted to add here is a bit of history, something like a motion blur FX ( which blends the N previous frames )  but this time I wanted to keep all the “light” as it would accumulate inside a long exposure picture. To do this, the process is in fact quite simple , every frame I compute the luminance of each pixels, if a pixel is brighter than all the pixels in the previous frames until now then I keep this one in the “memory buffer” which will keep track of all the “brightest pixels until now”

This gives this kind of result for a video :

Original Clip	Clip with “Keep Max Luminance Pixel”

The nice thing with this FX is that it really keeps the contribution of the light to the scene, so once the background is “lit” , you can walk in front of it and “disappear”  ( as long as you wear dark clothes/nothing that could appear be bright such as your glasses reflecting the light for example  )

Since my purpose was to have scene be progressively lit and the light to be moving without me on the video this allowed this kind of result

Original Clip	Clip with “Keep Max Luminance Pixel”

For the moment I only tested this plugin inside Vegas and did not add any options

The only drawback with the plugin is that you cannot jump forward/backward to any frame of your video clip as I need all the frames from the beginning of the clip until the current frame to produce the result.  So while playing / rendering this is ok since we advance one frame after the other but when seeking backward or a long time forward it can takes sometime to gather all the needed frames to produce the result. 

In order to improve the performance I switched my main algorithm to SSE to process more pixels in a shorter time but the real bottleneck is the time it takes to Vegas/OFX to “fetch” each image ( expecially when you are dealing with 1080p images ) so I choose to have the plugin to actually “skip” images in order to produce a draft image when you are seeking too far away

I’m still cleaning up a bit the plugin and intend to make it available here real soon for anybody interested…

For “coders” who would be interested in a bit more detail here is the code of the loop doing the main work ( parse all pixels, whatever pixel is brighter that the brightest so far is kept in the “history buffer” which is used as the result image )