Flipnote Shader
A Unity post effect that makes a 3D scene look like it was drawn in Flipnote Studio, the animation app on the Nintendo DSi and 3DS: dithered down to Flipnote's six colors, with sketchy outlines. I built it on my own over 16 days of a daily tech-art journal, in HLSL with Python for the color tables.
- Color: a 3D lookup texture maps any RGB color to the nearest of 231 dithered shades, and an 8×8 Bayer pattern draws it on screen.
- Outlines: a Sobel filter on the scene's depth and normals, with noise so the lines look hand-drawn.
- Controls: saturation, palette and outline detail.
- Use it: a .unitypackage you drop onto any URP camera as a post effect.
UnityHLSLURPC#Python
Color
Flipnote on the 3DS draws with six colors: black, white, red, blue, yellow and green. The DSi version has four. A Python script mixes every pair of them at 17 levels with an 8×8 Bayer matrix, which gives 231 shades, and saves them as a dither palette.
The shader then has to find the nearest shade for any color on screen. Searching the palette per pixel would be slow, so a second script precomputes the answer for the whole RGB cube as a 3D lookup texture (a LUT), and a C# editor script turns it into a Unity Texture3D. Its red and green channels hold the position of the matching shade in the dither palette. The shader reads that position, then uses the pixel's screen position (modulo 8) to pick the right texel of the pattern.
"Nearest" depends on how you measure it. I compared RGB, HSV and CIELAB distance. HSV came out wonky, the friends I showed preferred the RGB maps, and RGB is what the shader uses.
Muted scenes came out gray in a six-color palette, so a small shader saturates the scene before the palette step.
Outlines
Edge detection on the color image alone misses two objects of the same color and outlines shadows. So the Sobel filter runs on a depth-normal texture instead: the scene's normals multiplied by its depth.
The filter measures the change in each color channel and keeps the largest, which costs a little more than using brightness but still finds edges between colors of similar brightness.
A pseudo-random offset then nudges each outline sample, because Flipnote lines are sketchy rather than clean. The outlines are composited over the dithered color.
What I'd change
I noticed frame drops while making the demo video. Each pixel samples the lookup texture and then the dither palette; merging the two would cut a sample call, and the noise function could be cheaper.
Credits
The test scenes are Unity Asset Store packs by JustCreate, karboosx and Polygonal Mind. The Flipnote frog sprites are by thatmusicweeb. Videos by Acerola and Freya Holmér, and tutorials by Ronja, were my way into shaders.