In some rare cases, you may need executable to 'vanish' (delete itself) upon shutdown.
The most trivial case for such behavior would be updating (when never version of game is downloaded, and there is no reason to keep old one).
Less common cases include removing game under specific conditions (note: this is not an appropriate way of stopping user from playing it) and single-use utilities (which user will never need to run again for sure)
However, as you may noticed, executable's cannot delete themselves while running.

So, some tricks are needed.
The easiest one is to have second executable to run afterwards and delete the first one.
To illustrate source of such 'clean-up' program, I have used language that provides smallest output file size, while still keeping it readable. In this case, it's C++.

#include "windows.h"
#include <tchar.h>
int APIENTRY _tWinMain(HINSTANCE ii, HINSTANCE pi, LPTSTR lpCmdLine, int ncs) {
	for (;;) {
		if (DeleteFile(lpCmdLine)) return 0;
		Sleep(100);
	}
}

(you can download compiled version of this code here)

Structure of code is quite simple, as you could notice - we include two required headers (windows.h for used functions and tchar.h to allow main function definition to be written this way); main function itself consists of a perfectly infinite loop (heard of these?) in which program attempts to delete file given by command line and quits on success, or waits 0.1 second and tries again.

Integration on GameMaker side is simple as well - all you have to do is to run program described above just before game ends with game's executable location as a single argument. Fortunately function parameter_string(0) can be called to retrieve mentioned location, so code can be minimized to

execute_program('DeleteThat.exe', parameter_string(0), false)

Obviously, this way you will have a secondary executable remaining upon success of all operations, which user will have to delete manually (or not delete at all, in case of use in updater).
To save user from potential wondering about purpose of small executable that appeared after game's end, it can be unpacked into game's temporary folder.

Download GMK

Example illustrates typical setup for such game: secondary executable is included inside of game, and is unpacked & launched at game end. To test it, compile game into executable file, run it (in this case it's just a placeholder window) and close it by pressing Escape or window button - executable will vanish in moments after process termination.