Just a few things to add to AviSynth.nl , AviSynthShader should be there. , it should mention that ColorMatrix can cause banding due to rounding errors, and suggest 16-bit-depth alternatives such as Dither Tools and AviSynthShader for colorspace conversion without banding. By the way, does anyone know how the 'hints' parameter of ColorMatrix works internally? I'm wondering whether that could be implemented in AviSynthShader easily.
I don't know whether the SSE2/MMX introduces any more rounding errors than without, all I know is that the output of ColorMatrix can introduce some banding that is greatly reduced by using 16-bit processing. With DitherTools, there is still a little bit of banding but it is much smaller. ColorMatrixShader may have slightly more banding than DitherTools because of the YV12-YV24 conversion (unless you use DitherTools for that conversion) but it's close. There are others who know more about this.
My bad, I though it was clear I was referring to MisterX anyway Groucho's example without dithering (mode=-1) differences are almost 0, just some +/-1 noise on cplanes, even using sse2 or plain c
Sorry to be hypercritical, but this is relevant to a discussion of colormatrix. People should learn what proper color bars look like. Those color bars are incorrect (). It may be your screen shot and not your video; for instance, VirtualDub always decodes YUV sources for preview as Rec.601. For accurate preview of Rec.709 sources, you need to add the vdub filter AliasFormat (color space=Rec709).
It looks "incorrect" because I did a conversion from Rec601 into Rec709, and then VirtualDub reads it again as Rec601; but that doesn't really change anything.
I recall re-encoding an old black and white DVD video a while back. It was slightly "brown and white", so I used Tweak to make it black and white: Tweak(sat=0) I was really surprised at how much banding it caused. I've never quite got my head around why, and haven't come across a B/W DVD that bad since, but I did try it on the first screen shot from post #9 and it seemed to increase the banding. Can that be attributed to rounding errors as such? Or maybe it's just making the banding easier to spot? ConvertToYv12() ConvertToYv12() Tweak(sat=0)