GameMaker: Clipping drawn graphics


Mouseover to activate, click to reposition the clipping star.

This is a tutorial about pretty much everything related to clipping drawn graphics in GameMaker.

That is, having drawn graphics only display inside a certain ("clip") area, be that a rectangle (UI regions, minimaps, etc.), circle, or a completely arbitrary shape (pictured above).

Also I'm trying new things so this post is nicely stuffed with interactive demos and snippets.

Rectangular clip (via surfaces)

Drawing sprites while clipping outside of rectangle in GameMaker.
Mouseover to view, click resize clip region

This one is the most common case and the one that is the easiest to implement:

  1. Create a clip-area-sized surface.
  2. Draw the graphics into it, offsetting them (either directly or via d3d_transform) by clip area's top-left corner=' coordinates.
  3. Draw the clip area surface at it's top-left corner' coordinates.

The code is straightforward too,

// create a surface if it doesn't exist:
if (!surface_exists(clip_surface)) {
    clip_surface = surface_create(clip_width, clip_height);
}
// clear and start drawing to surface:
surface_set_target(clip_surface);
draw_clear_alpha(c_black, 0);
// draw things here, subtracting (clip_x, clip_y) from coordinates:
draw_circle(mouse_x - clip_x, mouse_y - clip_y, 40, false);
// finish and draw the surface itself:
surface_reset_target();
draw_surface(clip_surface, clip_x, clip_y);

Where clip_surface is the surface ID used for clipping (can be set to -1 in Create), and clip_x\y\width\height define the clip' region.

Same approach is used in "scrollable content" example that I published a few years ago.

Rectangular clip (via shaders)

If you would prefer to use a shader over surface, you can.

For this you would add a shader with the following vertex code:

attribute vec3 in_Position;
attribute vec4 in_Colour;
attribute vec2 in_TextureCoord;
//
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying vec3 v_vPosition;
//
void main() {
    v_vPosition = in_Position;
    gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION]
        * vec4(in_Position.x, in_Position.y, in_Position.z, 1.0);
    v_vColour = in_Colour;
    v_vTexcoord = in_TextureCoord;
}

This is identical to the default "pass-through" code, except for one addition - it stores v_vPosition so that the fragment shader knows the coordinates of things being drawn. Fragment code, on other hand, would be as following:

varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying vec3 v_vPosition;
//
uniform vec4 u_bounds;
//
void main() {
    vec4 col = v_vColour * texture2D(gm_BaseTexture, v_vTexcoord);
    col.a *= float(v_vPosition.x >= u_bounds[0] && v_vPosition.y >= u_bounds[1]
        && v_vPosition.x < u_bounds[2] && v_vPosition.y < u_bounds[3]);
    gl_FragColor = col;
}

This pulls RGBA from the texture like a default pass-through shader would, but sets the alpha channel to 0 if the point is outside the rectangle (u_bounds).

Then, to use this, you would set the shader and pass in the rectangle' data:

// debug:
draw_circle(mouse_x, mouse_y, 40, true);
draw_rectangle(clip_x1, clip_y1, clip_x2, clip_y2, true);
// set up shader:
shader_set(shd_clip_rect);
var u_bounds = shader_get_uniform(shd_clip_rect, "u_bounds");
shader_set_uniform_f(u_bounds, clip_x1, clip_y1, clip_x2, clip_y2);
// draw things:
draw_circle(mouse_x, mouse_y, 40, false);
// finish:
shader_reset();

Where clip_ are the rectangle' bounds (much like with draw_rectangle).

A downloadable example of this is included at the end of the blog post for your convenience.

Rectangular clip for sprites

Drawing sprites while clipping outside of rectangle in GameMaker.
Mouseover to preview, click to move rectangle' points.

Sometimes, you might want to draw a clipped graphic (e.g. a cursor or an off-screen indicator) without creating a surface or adding a shader just for one thing.

For example, in Don't Crawl we have multiple layers of graphics (terrain tiles - splatter - entities - unscaled screen-space effects - entity overlays - cursors - gui), which required to clip some things to views despite them being drawn over them in the GUI events.

While rotated graphics would require a bit of fancy math to clip the drawn polygon, for a standard case, things are simple enough - calculate the resulting bounds of a graphic, check if it falls inside the clip area at all, draw a piece of it defined by intersecting rectangle if so:

/// draw_sprite_clip(sprite, subimg, x, y, clipx, clipy, clipw, cliph)
var s = argument0;
var sw = sprite_get_width(s);
var sh = sprite_get_height(s);
var sx = sprite_get_xoffset(s);
var sy = sprite_get_yoffset(s);
var si = argument1;
var _x = argument2;
var _y = argument3;
var cx1 = argument4;
var cy1 = argument5;
var cx2 = cx1 + argument6;
var cy2 = cy1 + argument7;
//
var bx1 = _x - sprite_get_xoffset(s);
var by1 = _y - sprite_get_yoffset(s);
var bx2 = bx1 + sprite_get_width(s);
var by2 = by1 + sprite_get_height(s);
//
switch (rectangle_in_rectangle(bx1, by1, bx2, by2, cx1, cy1, cx2, cy2)) {
case 1:
    draw_sprite(s, si, _x, _y);
    return true;
case 2:
    var lx1 = max(0, cx1 - bx1);
    var ly1 = max(0, cy1 - by1);
    var lx2 = sw + min(0, cx2 - bx2);
    var ly2 = sh + min(0, cy2 - by2);
    draw_sprite_part(s, si, lx1, ly1, lx2 - lx1, ly2 - ly1, _x + lx1 - sx, _y + ly1 - sy);
    return true;
}
return false;

Or, if you want scaling and/or blending, a slightly fancier version:

/// draw_sprite_clip_ext(sprite, subimg, x, y, xscale, yscale, color, alpha, rx, ry, rw, rh)
var s = argument0;
var sw = sprite_get_width(s);
var sh = sprite_get_height(s);
var sx = sprite_get_xoffset(s);
var sy = sprite_get_yoffset(s);
var si = argument1;
var _x = argument2;
var _y = argument3;
var mx = argument4;
var my = argument5;
var sc = argument6;
var sa = argument7;
var cx1 = argument8;
var cy1 = argument9;
var cx2 = cx1 + argument10;
var cy2 = cy1 + argument11;
//
var bx1 = _x - sprite_get_xoffset(s) * mx;
var by1 = _y - sprite_get_yoffset(s) * my;
var bx2 = bx1 + sprite_get_width(s) * mx;
var by2 = by1 + sprite_get_height(s) * my;
//
switch (rectangle_in_rectangle(bx1, by1, bx2, by2, cx1, cy1, cx2, cy2)) {
case 1:
    draw_sprite_ext(s, si, _x, _y, mx, my, 0, sc, sa);
    return true;
case 2:
    if (mx == 0 || my == 0) return true;
    var lx1 = max(0, cx1 - bx1) / mx;
    var ly1 = max(0, cy1 - by1) / my;
    var lx2 = sw + min(0, cx2 - bx2) / mx;
    var ly2 = sh + min(0, cy2 - by2) / my;
    draw_sprite_part_ext(s, si, lx1, ly1, lx2 - lx1, ly2 - ly1,
        _x + (lx1 - sx) * mx, _y + (ly1 - sy) * my,
        mx, my, sc, sa);
    return true;
}
return false;

Use is simple enough:

draw_sprite_ext(q, 0, mx, my, 1, 1, 0, -1, 0.1); // background sprite (debug)
draw_sprite_clip(q, 0, mx, my, x1, y1, x2-x1, y2-y1); // clipped sprite
draw_rectangle(x1, y1, x2, y2, 1); // clip border (debug)

Arbitrary clip area (via surfaces)


Mouseover to preview, click to move the clip-circle.

This is where things get interesting:

  1. Create a surface for the clip-mask.
  2. Fill the surface with an opaque black color.
  3. Cut out (via draw_set_blend_mode(bm_subtract)) hole(s) for seeing things through.
  4. Create another surface for clip area (same size).
  5. Draw the graphics into (and relative to) clip area surface.
  6. Cut out the mask surface out of clip area surface (again, via bm_subtract).
  7. Draw the clip area surface.

Therefore, mask-surface acts like a stencil for preventing clip area surface' contents from showing through where they shouldn't be.

The code is only slightly more complex:

if (!surface_exists(mask_surface)) {
    // create the mask-surface, if needed
    mask_surface = surface_create(256, 256);
    surface_set_target(mask_surface);
    draw_clear(c_black);
    draw_set_blend_mode(bm_subtract);
    // cut out shapes out of the mask-surface:
    draw_circle(128, 128, 70, false);
    //
    draw_set_blend_mode(bm_normal);
    surface_reset_target();
}
if (!surface_exists(clip_surface)) {
    // create the clip-surface, if needed
    clip_surface = surface_create(256, 256);
}
// start drawing:
surface_set_target(clip_surface);
draw_clear_alpha(c_black, 0);
// draw things relative to clip-surface:
draw_circle(mouse_x - clipx, mouse_y - clipy, 40, false);
// cut out the mask-surface from it:
draw_set_blend_mode(bm_subtract);
draw_surface(mask_surface, 0, 0);
draw_set_blend_mode(bm_normal);
// finish and draw the clip-surface itself:
surface_reset_target();
draw_surface(clip_surface, clipx, clipy);

Where clipx, clipy are the clip surface' top-left corner, and clip_surface\mask_surface are the surfaces for clip area and mask accordingly (to be set to -1 in Create-event). Surface and mask' sizes are constant (256x256) in this case.

This allows for noticeably more advanced effects - for example, in the demo at the start of the post, this approach is used with a constantly updating mask surface to fill the overlapping area between two rotating shapes.

Arbitrary clip area (via shaders)

As with other things, it is also possible to implement arbitrary clip masks via a shader.

So you would make a new GLSL ES shader, name it something like shd_clip_mask, and set it's vertex shader code to be as following:

attribute vec3 in_Position;
attribute vec4 in_Colour;
attribute vec2 in_TextureCoord;
//
varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying vec3 v_vPosition;
//
void main() {
    v_vPosition = in_Position;
    gl_Position = gm_Matrices[MATRIX_WORLD_VIEW_PROJECTION]
        * vec4(in_Position.x, in_Position.y, in_Position.z, 1.0);
    v_vColour = in_Colour;
    v_vTexcoord = in_TextureCoord;
}

The idea is the same as in clip rectangle shader - v_vPosition is used to determine the drawing position inside the fragment shader, but here it is used to determine the point in mask-texture to sample pixels from.

varying vec2 v_vTexcoord;
varying vec4 v_vColour;
varying vec3 v_vPosition;
//
uniform vec4 u_rect;
uniform sampler2D u_mask;
//
void main() {
    gl_FragColor = v_vColour
        * texture2D(gm_BaseTexture, v_vTexcoord)
        * texture2D(u_mask, (v_vPosition.xy - u_rect.xy) / u_rect.zw);
}

Then you would set the shader, give it the surface (or other texture) to sample mask pixels from, give it the clip region via uniforms, and any things drawn will be drawn with the said mask.

// create the mask surface if needed:
if (!surface_exists(mask)) {
    mask = surface_create(256, 256);
    surface_set_target(mask);
    draw_clear_alpha(c_white, 0);
    draw_set_color(c_white);
    draw_circle(128, 128, 70, false);
    surface_reset_target();
}
// debug drawing:
draw_set_color(make_color_rgb(94, 101, 124));
draw_circle(mouse_x, mouse_y, 40, true);
draw_circle(clipx + 128, clipy + 128, 70, true);
// set up the shader:
shader_set(shd_clip_mask);
var u_mask = shader_get_sampler_index(shd_clip_mask, "u_mask");
texture_set_stage(u_mask, surface_get_texture(clip_mask));
var u_rect = shader_get_uniform(shd_clip_mask, "u_rect");
shader_set_uniform_f(u_rect, clipx, clipy, 256, 256);
// draw things:
draw_circle(mouse_x, mouse_y, 40, false);
// finish:
shader_reset();

In conclusion

While GameMaker does not come with built-in functions for clipping graphics (although perhaps this will change in 2.x now that it is not tied to ancient DirectX9), feasible workarounds exist for basically any imaginable use case.

A sample project containing shaders (and code for some of the demos shown here) can be downloaded from itch.io.

Have fun!

Related posts:

23 thoughts on “GameMaker: Clipping drawn graphics

  1. here my curious question: what happen if you use that video method but with shader instead? my goal is to use the texture instead of color since it will be done with a checker texture, imitating the old school games that were often used in NES game, genesis/megadrive games or SNES games for example. here the video i mentioned early:

    https://youtu.be/AhUDFm7Xo1M

    • I could use a slightly more in-depth explanation or an illustration because I’m not sure how this relates to clipping.

  2. This was really informative! I was wanting to make a kaleidoscope effect by making a surface and a sprite of 4 triangles like a pinwheel and have it reflect but for the life of me couldn’t get it down. The first example you showed with the spinning stars seems to be the closest to how i envisions the kaleidoscope but i couldn’t get it to work. I tried drawing 4 surfaces and rotated them to make a pinwheel shape but couldn’t get them to reference the same source. Would you be able to shed some light on how I can figure out how to get it working?

    • For a kaleidoscope effect you wouldn’t even need fancy clipping would you? You could make a surface, draw whatever into it, and then draw 3 triangles with same texture coordinates but different screen coordinates using draw_primitive_* or a vertex buffer.

      • Sorry for the long delay, I didn’t realize i got a reply until I came back to the project today and was looking for this site again. I finally got it working by making 6 clipped triangle surfaces. I tried looking at videos on privatives but I couldn’t understand how to translate them into what i was looking for. I know using 6 surfaces isn’t optimized or ideal but its at least getting the result i want lol. Now i just have to find out how it would affect performance and how many points i can make before it does.

  3. This is really cool! Is there any way to get particle effects to draw normally in the circle? I can’t seem to get it to work.

  4. How would you inverse this effect? I.e. draw a solid color to the rectangular region, and cut out the sprite/shape at the mouse position?

  5. Quick question about the shader variant of the arbitrary clipping:
    Does it still mess with the alpha values? or is it just taking the surface’s pixels and then clipping the drawn stuff using it?

    • It multiplies drawn pixels’ colors and alpha by those from the mask-surface.

      For a sharp cutoff, you could use discard

  6. Thank you very much for this solution, does work on GMS2 IDE v2022.2.0.614 Runtime v2022.2.0.487. This saved my time and my project TuT

  7. What would be the method to inverse Rectangular clip (via surfaces)? So like, the entire room is back besides a cut out circle that follows the mouse around

    • Have a view-sized surface, clear it with color, draw a circle to it with bm_subtract blend mode, then draw it to the screen. Simple lighting systems do just this.

  8. I know this is old but it’s still very useful! I only realized how old it was when I saw the date of the comments here haha

    I just wanted to say that the “Rectangular clip for sprites” method does not work correctly if the sprite’s xscale or yscale is below zero

  9. Man your “simple” script for sprites really helped me out today, using it to draw sprites in a pseudo 3d racer, like over a hill but not in full veiw, was exactly the stuff I needed. You the man! You saved me so much headache I can’t believe how well it works for how dumb I am. I just plugged it in, barely understood how it worked and bam. It’s beautiful.

      • When I imported into GMS2 and pressed play, the console was filled with this message:
        “`
        Draw failed due to invalid input layout
        “`

        The first example doesn’t clip as expected; it draws the outline for a rectangle and an outline for the circle but that’s it.

        The problem occurs in obj_clip_rect_shd’s draw event, on line: draw_circle(mouse_x, mouse_y, mouse_r, false);

        I expected it was an issue with the shader; when I comment out that draw_circle line, I don’t see the error anymore.

        In rm_clip_rect_shd, I noticed a “Compatibility_Colour” layer is created with a depth of 2147483600, which I think is out of range in GameMaker Studio 2 so I set it to 999. This alone did not fix the problem.

        I deleted the Compatibility_Instances_Depth_0 layer, create a new instance layer, and dragged obj_clip_rect_shd into it. Still did not remove the console message.

        Finally, I edited shd_clip_rect and discovered that commenting out the line:

        v_vTexcoord = in_TextureCoord;

        resulted in no warning messages in the console and the shader worked.

  10. Wow neat stuff. I could definitely see using this in the near future. Most likely on a top down 3/4 project.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.