unity – Displaying pixel artwork at any decision with subpixel motion

The Objective

I’ve a sport in Unity which makes use of pixel artwork. I am not following the usual guidelines of pixel artwork, nonetheless. I am pleased with sprites having totally different sized pixels, pixels rotating, and pixels not aligning with the pixel grid. An instance (not my video): https://www.youtube.com/shorts/FCJWPYqV0TI. I would additionally just like the participant to have the ability to run the sport at no matter decision they need whereas sustaining the identical vertical digital camera dimension, i.e. with a wider decision you’d see extra of the world horizontally, however not vertically. Additionally, particular to my sport, is the participant’s capacity to zoom the digital camera out and in.

Answer 1: A Commonplace Setup

The very first thing I attempted, since I wasn’t following the same old pixel artwork practices, was to simply use a regular 2D sport setup. I positioned my sprites within the scene, together with a digital camera. I set the filter mode on my sprites to level (no filter), set compression to none, and turned off all types of AA. I set my digital camera to comply with the participant in a clean movement, no pixel excellent digital camera right here since that is not the impact I am after. The outcomes:

Sprite shimmering. This publish is a incredible learn on why this occurs: What makes scaling pixel artwork totally different than different pictures?. You will even have jagged edges when rotating sprites.

So how will we remedy this? Often the reply is bilinear filtering. That removes these excessive distinction positions between pixels. You will additionally need to pad your sprite by 1 pixel, in order that the sides can mix with transparency. After all, for a sprite of this dimension, the outcomes are lower than perfect (I additionally did not add the padding on this instance, leading to harsh edges):

Answer 2: Bilinear Filtering Shader

So the subsequent resolution was to make a shader that took benefit of bilinear filtering, however solely utilized it the place wanted. There’s a nice video on this topic right here: https://www.youtube.com/watch?v=d6tp43wZqps. Right here is the shader code:

void PixelHD_float(UnityTexture2D Texture, float2 UV, out float4 Shade)
{
    float2 boxSize = clamp(fwidth(UV) * Texture.texelSize.zw, 1e-5, 1);
    float2 texel = UV * Texture.texelSize.zw - boxSize * 0.5;
    float2 offset = smoothstep(1 - boxSize, 1, frac(texel));
    float2 uv = (flooring(texel) + 0.5 + offset) * Texture.texelSize.xy;
    Shade = Texture.SampleGrad(Texture.samplerstate, uv, ddx(UV), ddy(UV));
}

Mainly what is going on is we’re getting the typical colour inside a field the dimensions of a single pixel on display. With this we simply set our Sprites to make use of bilinear filtering, and make a cloth that makes use of the shader. Then slap that shader onto the sprite renderers. The outcomes are very promising:

The shimmering is getting adequately subtle that you could’t actually make it out within the gif! Not excellent, however we’re getting there.

Answer 3: My Customized Shader

After weeks and weeks of analysis, trial and error, and in addition realizing I needed to interchange colours, I ended up making this shader myself:

#ifndef PIXELHD_INCLUDED
#outline PIXELHD_INCLUDED

bool UVInBounds(float2 UV)
{
    return UV.x >= 0 && UV.y >= 0 && UV.x <= 1 && UV.y <= 1;
}

float4 ReplaceColor(float4 Shade, float4 RedReplacement, float4 GreenReplacement, float4 BlueReplacement)
{
    if ((Shade.r > 0 ? 1 : 0) + (Shade.g > 0 ? 1 : 0) + (Shade.b > 0 ? 1 : 0) > 1)
    {
        return Shade;
    }
    else if (Shade.r > 0)
    {
        return float4(RedReplacement.rgb * Shade.r, RedReplacement.a * Shade.a);
    }
    else if (Shade.g > 0)
    {
        return float4(GreenReplacement.rgb * Shade.g, GreenReplacement.a * Shade.a);
    }
    else
    {
        return float4(BlueReplacement.rgb * Shade.b, BlueReplacement.a * Shade.a);
    }
}

float4 HandleFullyTransparent(float4 OriginalColor, float4 HorizontalNeighbor, float4 VerticalNeighbor, float4 CornerNeighbor, float2 OffsetDistance)
{
    // Preserve the unique colour if it isn't totally clear.
    if (OriginalColor.a != 0)
    {
        return OriginalColor;
    }
    // Take the horizontal neighbor's colour if the vertical neighbor is totally clear.
    else if (HorizontalNeighbor.a != 0 && VerticalNeighbor.a == 0)
    {
        return float4(HorizontalNeighbor.rgb, 0);
    }
    // Take the vertical neighbor's colour if the horizontal neighbor is totally clear.
    else if (VerticalNeighbor.a != 0 && HorizontalNeighbor.a == 0)
    {
        return float4(VerticalNeighbor.rgb, 0);
    }
    // Take the nook neighbor's colour if the horizontal and vertical neighbor is totally clear.
    else if (HorizontalNeighbor.a == 0 && VerticalNeighbor.a == 0)
    {
        return float4(CornerNeighbor.rgb, 0);
    }
    // Neither the horizontal or vertical neighbor is totally clear, so we take the closest colour.
    else if (OffsetDistance.x < OffsetDistance.y)
    {
        return float4(HorizontalNeighbor.rgb, 0);
    }
    else if (OffsetDistance.y < OffsetDistance.x)
    {
        return float4(VerticalNeighbor.rgb, 0);
    }
    // The horizontal and vertical neighbors are equal distance away, so we combine them.
    else
    {
        return float4((HorizontalNeighbor.rgb + VerticalNeighbor.rgb) * 0.5, 0);
    }
}

void PixelHDReplaceColor_float(UnityTexture2D Texture, float2 UV, float4 RedReplacement, float4 GreenReplacement, float4 BlueReplacement, out float4 Shade)
{
    // Convert UV coordinates into texel coordinates.
    float2 texelCoordinates = UV * Texture.texelSize.zw;
    
    // Get the underside left of the closest texel.
    float2 texel = spherical(texelCoordinates);
    
    // Get the change in uv per pixel.
    float2 uv_ddx = ddx(UV);
    float2 uv_ddy = ddy(UV);
    
    // Get the uv coordinates of the middle of the 4 surrounding texels.
    // Multiply by texel dimension to transform from texel coordinates to UV coordinates.
    float2 uv00 = (texel - 0.5) * Texture.texelSize.xy;
    float2 uv10 = (texel + float2(0.5, -0.5)) * Texture.texelSize.xy;
    float2 uv01 = (texel + float2(-0.5, 0.5)) * Texture.texelSize.xy;
    float2 uv11 = (texel + 0.5) * Texture.texelSize.xy;
    
    // Pattern the feel at our 4 factors.
    // If we're changing colours, mipmaps should not used as they pollute the info.
    float4 c00;
    float4 c10;
    float4 c01;
    float4 c11;
    if (all(RedReplacement == float4(1, 0, 0, 1)) && all(GreenReplacement == float4(0, 1, 0, 1)) && all(BlueReplacement == float4(0, 0, 1, 1)))
    {
        c00 = UVInBounds(uv00) ? Texture.SampleGrad(Texture.samplerstate, uv00, uv_ddx, uv_ddy) : 0;
        c10 = UVInBounds(uv10) ? Texture.SampleGrad(Texture.samplerstate, uv10, uv_ddx, uv_ddy) : 0;
        c01 = UVInBounds(uv01) ? Texture.SampleGrad(Texture.samplerstate, uv01, uv_ddx, uv_ddy) : 0;
        c11 = UVInBounds(uv11) ? Texture.SampleGrad(Texture.samplerstate, uv11, uv_ddx, uv_ddy) : 0;
    }
    else
    {
        c00 = UVInBounds(uv00) ? ReplaceColor(Texture.SampleLevel(Texture.samplerstate, uv00, 0), RedReplacement, GreenReplacement, BlueReplacement) : 0;
        c10 = UVInBounds(uv10) ? ReplaceColor(Texture.SampleLevel(Texture.samplerstate, uv10, 0), RedReplacement, GreenReplacement, BlueReplacement) : 0;
        c01 = UVInBounds(uv01) ? ReplaceColor(Texture.SampleLevel(Texture.samplerstate, uv01, 0), RedReplacement, GreenReplacement, BlueReplacement) : 0;
        c11 = UVInBounds(uv11) ? ReplaceColor(Texture.SampleLevel(Texture.samplerstate, uv11, 0), RedReplacement, GreenReplacement, BlueReplacement) : 0;
    }
    
    // Exit early if the result's totally clear.
    if (c00.a == 0 && c10.a == 0 && c01.a == 0 && c11.a == 0)
    {
        Shade = 0;
        return;
    }
    
    // Exit early if all colours are the identical.
    if (all(c00 == c10) && all(c00 == c01) && all(c00 == c11))
    {
        Shade = c00;
        return;
    }
    
    // Get the offset of the texel coordinates from the underside left of the texel.
    float2 offset = texelCoordinates - texel;
    
    // Deal with totally clear colours
    float2 offsetDistance = abs(offset);
    c00 = HandleFullyTransparent(c00, c10, c01, c11, offsetDistance);
    c10 = HandleFullyTransparent(c10, c00, c11, c01, offsetDistance);
    c01 = HandleFullyTransparent(c01, c11, c00, c10, offsetDistance);
    c11 = HandleFullyTransparent(c11, c01, c10, c00, offsetDistance);
    
    // Get the pixel dimension of a texel.
    float2 pixelSize = sqrt(uv_ddx * uv_ddx + uv_ddy * uv_ddy);
    
    // Get the ratio of texel dimension to pixel dimension.
    float2 texelToPixelRatio = Texture.texelSize.xy / pixelSize;
    
    // Multiply the offset by the texel to pixel ratio.
    // This implies if the texel takes up a whole lot of display house, then small modifications within the offset matter little or no.
    // If the texel takes up little or no display house, the offset issues far more.
    // We add 0.5 for the reason that offset is between 0 and 0.5, as a result of we rounded to the closest texel.
    // We clamp this worth in order that it may't go away the texel.
    float2 mix = saturate(offset * texelToPixelRatio + 0.5);
    
    // Interpolate between colours.
    Shade = lerp(lerp(c00, c10, mix.x), lerp(c01, c11, mix.x), mix.y);
}

#endif

The colour alternative occurs earlier than the bilinear filtering, then it handles any clear texels, then it performs bilinear filtering. The quantity of filtering is predicated on the ratio of texel dimension to pixel dimension on display. You do not even have to pad the sprites any extra, since I deal with all UV’s lower than 0 or higher than 1 as clear. The outcome was completely crisp pixel artwork with no shimmering. At this level I assumed my drawback was solved. Nonetheless, there was one obvious subject: Sprites may now not be seamless.

You possibly can see there are seams between background tiles, and really small seams between the character’s legs, toros, and head, since they’re all separate sprites. This may occasionally not seem to be a giant deal, for the reason that background is at present a strong colour, however that may ultimately get replaced with tiles which can not all the time share the identical colour at their borders.

That is in the end the place I received caught. At first I although the answer could be to not pattern alphas, making every edge that borders an alpha a tough edge as a substitute of a comfortable edge. I achieved that by doing this on the finish of the shader:

float w00 = (1 - mix.x) * (1 - mix.y) * (c00.a != 0 ? 1 : 0);
float w10 = (mix.x) * (1 - mix.y) * (c10.a != 0 ? 1 : 0);
float w01 = (1 - mix.x) * (mix.y) * (c01.a != 0 ? 1 : 0);
float w11 = (mix.x) * (mix.y) * (c11.a != 0 ? 1 : 0);
float totalWeight = w00 + w10 + w01 + w11;

// Interpolate between colours.
if (totalWeight > 0)
{
    Shade = (c00 * w00 + c10 * w10 + c01 * w01 + c11 * w11) / totalWeight;
}
else
{
    Shade = 0;
}

This truly does remedy the seams drawback, as sprites that share a border join correctly, however the shimmering returns since that a part of the sprite is now not being filtered. At this level I have been struggling for a couple of month on this, and would recognize any perception.