NoMoSmooth (motion-adaptive spatio-temporal smoother)

While playing with code I wrote to apply 3x3 matrices temporally to 9 consecutive frames I was struck with an idea for a temporal smoother that might not be as suceptible to ghosting as others. Edit 3: See and on for a rewritten version implementing motion-adaptive spatial-temporal smoothing. Here's my first attempt at implementing it. It's proof of concept code, and as such is very (I mean extremely) slow and might not even do anything on your system other than crash. That said, it works ok on mine ;). See the for more information on the algorithm. As I said this is not intended to be a proper "release" of a filter. I'm more interested in receiving as much feedback as possible (positive or negative) than putting out something you can actually use in your encodes. If people think this filter shows promise I'll fix it up. Have fun and don't blame me ;). Edit: Yes, I had some difficulty coming up with a good name. The one I chose was inspired by TomsMoComp and I think is fairly appropriate. Feel free to suggest another ;). Edit 2: Removed links to old version.

It’s working well here — no crashes, and no artifacts. I like the aspect that you give up averaging beyond a certain time if that timepoint shows evidence for motion. It would be interesting to see the same trick used for spatial smoothing, in which an intervening pixel of a different color would prevent averaging with a pixel further away. The threshold of 2 is probably a little too low to make this effective. At that value, even very light noise is going to cut the averaging range short. At least for over the air stuff, you can expect a mean difference of about 4 to 7 (for luma and chroma combined) even in stationary areas. (Thanks go to folks over in the video capture forum for that number.) If you decide to put more effort into this sequential approach, you might eventually want to look up hidden Markov models (HMMs). You could use an HMM to infer the probability that motion began at any particular time in the sequence.

The threshold of 2 is probably a little too low to make this effective. That's just the default I settled for on for my test material, which is a slightly noisy DVD. I didn't try on captured material yet. You could use an HMM to infer the probability that motion began at any particular time in the sequence. Sounds interesting. I'll ask Dr Google :).

New version 

Oh - I see - thanks for the clarification :) Edit: btw, the results looks good - smooths temporal noise very well, without any ghosting, and the speed is very acceptable.

the results looks good - smooths temporal noise very well I'm testing the code with radii up to 15 -- obviously unrealistic -- with good results (except the speed ;)). Though when the radius gets very large there's more danger of artifacts from scene changes, though theoretically these shouldn't be too noticible. I'm contemplating adding an optional scene-change detector in there. At the moment I'm pondering an interleave scheme such as in TemporalSoften which should improve performance greatly (e.g. incrementing one pointer instead of one for each frame in the radius). and the speed is very acceptable. Either you're being kind or you've only tried low radii (3-5) or you're running it on a quad Xeon system :D. I don't think the speed is acceptable at the moment, though as I said interleaving will help that. I fear any improvements beyond that will have to wait until I've learned assembler (though this gives me good incentive to actually do it :)).

SansGrip, If you want someone to agree with you, yes it is very slow :) However I was more concerned about the lack of effect at the ends of a clip. The first frames included in the radius do not seem to get much change. I would have thought that they should at least by influenced by half the value of the radius frames rather than none at all.

The first frames included in the radius do not seem to get much change. At the moment the first radius / 2 frames don't get any change. I would have thought that they should at least by influenced by half the value of the radius frames rather than none at all. They will, if this filter turns out to be useful enough to continue development on :). Edit: Seems it is useful, so useful in fact that it's been in the core for ages :D. I'm somewhat red-faced now. I'll keep working on it and try to come up with something that makes it different ;). Hey -- at least I know I'm thinking the right things ;).

@ SansGrip: Seems that you suddenly removed it from your page; I hope I'll find your idea soon in another one of your projects when it is getting ready...

Seems that you suddenly removed it from your page; I hope I'll find your idea soon in another one of your projects when it is getting ready... That particular version was redundant. I'll post 0.0c if there's merit in the tweaks I'm making now :). @sh0dan & HSD: Thanks for the suggestions, I was thinking about them last night and am currently playing :).