In this third-of-a-series post, I go over existing issues in GameMaker's data structure functions (some of which you might not have been aware of), and how to fix them.

Problems

Through series of coincidences, GameMaker data structures currently slightly resemble direct memory access in C++, with similar implications.

[If you want to reproduce these samples, run them in an empty-ish project on game start]

Wrong DS type access

Suppose you make a map and a list, and then you accidentally use

var a = ds_map_create();
a[?"hi"] = "oh no";
var b = ds_list_create();
show_debug_message(b[?"hi"]); // map function used on a list.

If you get lucky, you get an error. If you don't get lucky, you get "oh no". Or maybe undefined.

Did you know that running the game in debug mode reserves a couple of ds_map slots and can prevent you from getting the same sequence of errors when you are trying to reproduce them? It's good fun.

Non-data-structure access

Suppose you make a mistake and accidentally attempt data structure operations on an arbitrary number, or even a string containing a number (after 2.2.3 consistency changes). Since data structures are currently referenced by index, there's always a chance that, with some luck (or rather, absence thereof) you'll hit a valid data structure ID and thus get/set something you absolutely weren't meaning to.

Consider the following:

var a = ds_map_create();
var b = 0.3;
b[?"oh"] = "no";
var c = "0.7"; // new!
c[?"now"] = "this";
show_debug_message(json_encode(a));

So, if a was assigned index 0, you get { "oh": "no", "now": "this" }.

Some of the hardest-to-debug bugs that I've encountered originated from issues like this.

Access on destroyed structures

Did you know? For practical reasons (keeping memory usage at bay), GameMaker will reuse data structure indexes.

This is good for performance reasons, but not so good if you forget to "unassign" a variable after freeing the contained structure:

var a = ds_map_create();
ds_map_destroy(a);
var b = ds_map_create();
a[?"oh"] = "no";
show_debug_message(json_encode(b));

This will reliably produce { "oh": "no" }.

And, as an even more unpleasant side effect, attempting to destroy a a second time will indeed destroy the data structure index occupied by b, making for some confusing "data structure does not exist" errors.

Tracking down memory leaks

Admittedly not purely a GameMaker problem, but figuring out where are you forgetting to destroy data structures can be a little messy - while you can write wrapper scripts for _create and _destroy functions, these will not account for destroying nested structures (when marked via ds_*_mark_*).

Some language tools (like modern JS debuggers) offer tools for producing heap dumps (which show how much memory everything takes up, and where things were made from), but GML isn't at that point just yet.

To summarize

As you can see, use of indexes for data structures produces a separate set of issues and side effects.

The issue is acknowledged by YoYo Games, and "GML: data structures as a true datatype (not just an index)" item rests on considerations list, but it is not known just when this will be implemented (and, honestly, they have a lot of things to do, so it is understandable).

Solution

Although there had been a couple attempts of creative workarounds through the years (my JSON extension can also be considered as such), people often hesitated to use these due to inevitable performance impact of one or other kind.

For this reason, this new extension comes in two parts:

Quality Structures One

Offers a set of wrappers for almost every single data structure function while introducing a set of sanity checks.

The idea is as following:

a = ds_map_create(); // a is <index>
a = qs_map_create(); // a is [["qs::map"], <index>, <stack trace>]

So, instead of being just an index, you get a tiny array containing:

• A reference to a "marker" array, which identifies each data structure type
  (this allows to guard against accessing a wrong data structure type)
• The underlying data structure index
  (which is used for actual operations and unset upon destruction)
• A callstack (debug_get_callstack) for where this data structure was created/destroyed from (optional; allows to show context when accessing a destroyed data structure)

And then each wrapper script will check if it's the right thing and if it exists before doing anything. Add a custom JSON encoder, some helpers, and you suddenly have 2000 lines of code. Maybe that's why no one tried doing it this way before? Anyway,

Quality Structures Zero

This extension has identical function names to the above, but instead forwards them to each according built-in function directly - at zero overhead.

So, if performance impact from using QS1 becomes anyhow noticeable, you can create a separate "release" configuration in your GameMaker project, add it there, and set up files in each extension to only apply to their respective configuration.

Accesors

Since you cannot modify the behaviour of accessor operators, the extension also comes with a couple chained accessor functions. So, instead of doing

var map = json_decode(@'{"list":["hi!","hello!"]}');
var list = map[?"list"];
var hello = list[|1];

you would do

var map = qs_json_decode(@'{"list":["hi!","hello!"]}');
var hello = qs_get(map, "list", 1);

Examples

So, let's see what happens with each according example shown earlier with QS1 extension:

Mix up a data structure type? You now get an actual error:

var a = qs_map_create();
qs_set(a, "hi", "oh no");
var b = qs_list_create();
var c = qs_map_find_value(b, "hi"); // (a mishap)
//var c = qs_get(b, "hi"); // (also throws an error)
show_debug_message(c);

############################################################################################
FATAL FATAL ERROR in Room Creation Code for room room0

Expected a map, got qs::list (id 0), created from
  gml_Script_qs_list_create:5
  gml_Script_scr_test:3
  gml_Room_room0_Create:1
 at gml_Script_qs_impl_map_mismatch (line 2) - show_error("Expected a map, got " + qs_debug_dump(argument0), true);
############################################################################################
--------------------------------------------------------------------------------------------
stack frame is
gml_Script_qs_impl_map_mismatch (line 2)
called from - gml_Script_qs_map_find_value (line 9) - } else qs_impl_map_mismatch(argument0);
called from - gml_Script_qs_get (line 7) -               l_val = qs_map_find_value(l_val, l_key);
called from - gml_Script_scr_test (line 4) - show_debug_message(qs_get(b, "hi"));

Access a non-data structure? That's also a clean error:

var a = qs_map_create();
var b = 0.3;
qs_set(b, "oh", "no"); // throws error
var c = "0.7";
qs_set(c, "now", "this"); // also throws error
show_debug_message(qs_json_encode(a));

############################################################################################
FATAL FATAL ERROR in Room Creation Code for room room0

Expected a map, got 0.30 (real)
 at gml_Script_qs_impl_map_mismatch (line 2) - show_error("Expected a map, got " + qs_debug_dump(argument0), true);
############################################################################################
--------------------------------------------------------------------------------------------
stack frame is
gml_Script_qs_impl_map_mismatch (line 2)
called from - gml_Script_qs_map_set (line 7) - } else qs_impl_map_mismatch(argument0);
called from - gml_Script_qs_set (line 23) -        qs_map_set(l_val, l_key, l_new);
called from - gml_Script_scr_test (line 3) -        qs_set(b, "oh", "no"); // throws error

Access a destroyed data structure? That's also a readable error, even if the index had since been given out to a new data structure.

var a = qs_map_create();
qs_map_destroy(a);
var b = qs_map_create();
qs_set(a, "oh", "no");
show_debug_message(qs_json_encode(b));

############################################################################################
FATAL FATAL ERROR in Room Creation Code for room room0

This map is already destroyed from:
  gml_Script_qs_map_destroy:26
  gml_Script_scr_test:2
  gml_Room_room0_Create:1
 at gml_Script_qs_impl_map_amiss (line 16) - show_error(l_msg, true);
############################################################################################
--------------------------------------------------------------------------------------------
stack frame is
gml_Script_qs_impl_map_amiss (line 16)
called from - gml_Script_qs_map_set (line 6) -        } else qs_impl_map_amiss(argument0);
called from - gml_Script_qs_set (line 23) -        qs_map_set(l_val, l_key, l_new);
called from - gml_Script_scr_test (line 4) - qs_set(a, "oh", "no");

Want to see where you're leaking structures? Set qs_debug_active to true, pause the game in the debugger, and take a look at any of g_qs_active_* values (which are ds_maps containing DS ID -> creation origin pairs for currently active structures):

Conclusion

With some creative thinking and a bit of hard work, numerous existing inconveniences can be addressed without sacrificing much in return.

My resulting extension can be found on GM Marketplace and itch.io, as usual:

itch.io GM Marketplace

On a closing note, I leave you with an alternate icon+title that I have considered for this extension:

Related posts:

• Building a better localization system for GameMaker
• Some extensions are now GM8.1-compatible
• A summary of my GameMaker assets
• GameMaker: Loading bar + animated image in HTML5
• GameMaker: Window commands extension