This post about the issues with GameMaker's built-in JSON functions and an extension I made to address the issues.
Functions to deal with JSON were originally introduced in GameMaker somewhere around 2011 (when the current product was GameMaker HTML5).
Two main functions are pretty simple:
- json_decode takes a string containing JSON and returns a ds_map structure.
- json_encode takes a ds_map structure and returns a JSON string for it. Child structures must be marked accordingly.
And these would have been perfectly fine, but:
- GameMaker (currently) stores references to data structures as numeric IDs.
- GameMaker (still) lacks functions to determine whether a ds_map\ds_list item is marked as a map\list (this information is available internally).
What does that mean for you? Well - you can't validate the parsed JSON. Consider the following:
var q = json_decode('{ "a": [], "b": {}, "c": 1, "d": true }'); show_debug_message(q[?"a"]); show_debug_message(q[?"b"]); show_debug_message(q[?"c"]); show_debug_message(q[?"d"]);
The output of that is 0, 1, 1, 1.
So, if you were expecting a JSON object but got a numeric value (be that due to format changes, external errors, or an intentional malformed submission by a user), the game may crash (or do something unusual) unless you had inserted safeguards and checks at literally every corner.
Another shown problem is boolean values - APIs may occasionally require you to pass them an actual true / false as a parameter, but GameMaker's boolean type is numeric, so you end up with 1 / 0 and variously weird workarounds.
Solution
As of writing this post, "data structures as true datatype" item is on GameMaker Studio 2 roadmap and will likely make it into the software in near or semi-near future.
However, future is future, but things might need to be made right now, so I have developed an extension to tackle the problemin a timely matter.
The extension consists of a custom JSON decoder and encoder, and approaches the problems in a slightly more appropriate way:
- JSON arrays are mapped to GML arrays rather than ds_lists.
-
JSON objects are mapped to array-based structures.
(on JS-based platforms, stored as actual JS objects for performance) -
JSON boolean values are mapped to special reference values.
(on JS-based platforms, stored as actual JS true/false values)
The result is convenient:
- Since all values are either constant or reference-based, you no longer need to explicitly destroy the values produced by decode-function. No need to worry about memory leaks.
- Encode/decode functions are more straightforward - if you do tj_decode("[1, 2, 3]"), you get an actual array instead of a map with a ds_list in the "default" field. Similarly, you can call tj_encode on an array, you get a JSON array string "[...]" (which is something that you cannot do with json_encode at all).
-
You can check what any given value is - tj_is_array to check if something is an array, tj_is_object to check if something is a JSON object, and so on.
This means that you can reliably check if the data is correct, and dynamically iterate over the contents (see web demo on itch.io).
Example of use follows:
var i, q; // To decode a value, pass it to tj_decode, much like with json_decode q = tj_decode('[1, 2, 3]'); // JSON arrays are decoded into regular GML arrays: show_debug_message("An array: " + string(q)); // JSON objects are decoded into special array-based structures on native platforms, // and into actual JS Objects on JS-based platforms. This means that you don't have // destroy them after use - that is done automatically. q = tj_decode('{ "a": 1, "b": 2 }'); // tj_get returns a field of a TJSON object, much like ds_map_find_value. show_debug_message("q.a = " + string(tj_get(q, "a"))); // tj_set changes a field of a TJSON object. tj_set(q, "a", 3); // tj_size returns the number of fields that an object has. show_debug_message("q' size: " + string(tj_size(q))); // tj_keys returns an array containing fields of an object. var keys = tj_keys(q); show_debug_message("q' keys: " + string(keys)); // If you need to pick over all fields-values, you can use tj_keys+tj_get: for (i = 0; i < array_length_1d(keys); i++) { show_debug_message("q." + string(keys[i]) + " = " + string(tj_get(q, keys[i]))); } // To encode a value, pass it to tj_encode. This works for all supported types: show_debug_message("q encoded: " + tj_encode(q)); // If you provide a second parameter, the result will be multi-line and indented: show_debug_message("q indented: " + tj_encode(q, " ")); // To construct arrays/objects, you can use tj_array/tj_object: q = tj_object( "number", 4, "array", tj_array(1, 2, 3), ); show_debug_message("custom object: " + tj_encode(q, " ")); // JavaScript true/false false are converted into tj_true\tj_false // to be able to tell them apart. tj_bool\tj_is_bool are also available. q = tj_decode("true"); show_debug_message("js true == tj_true: " + string(q == tj_true)); show_debug_message("tj_false encoded: " + tj_encode(tj_false)); // show_message("All is well! Check the compile form / output log.");
Output being as following:
An array: { { 1,2,3 }, }
q.a = 1
q' size: 2
q' keys: { { a,b }, }
q.a = 3
q.b = 2
q encoded: {"a":3,"b":2}
q indented: {
"a": 3,
"b": 2
}
custom object: {
"number": 4,
"array": [
1,
2,
3
]
}
js true == tj_true: 1
tj_false encoded: false
As can be seen, is pretty straightforward to use.
The extension can be downloaded from Marketplace or itch.io.
In-depth documentation is included, and also available online.
Have fun!
Hey Vadim,
Thanks for making this awesome tool! You wrote this post back in 2017, and at the time you had hope that the “data structures as true datatype” item on GameMaker Studio 2 roadmap might become a better solution.
Interested to hear if GameMaker Studio 2 solved this yet and what your thoughts are on the current data processing situation.
2.3 beta introduces structs, which can more or less replace most of other data types. I have released a TJSON update that can encode/decode structs.
Does this work with GMS2?
It does
Greetings,
I just bought your extension and I would like to receive some hint on reading a Json like that:
[ { "ID":0, "NAME":"Simple Soul", "OBJ":"obj_weapon_soulBow", "COUNT":1 }, { "ID":1, "NAME":"Complex Soul", "OBJ":"obj_weapon_soulComplex", "COUNT":1 } ]The decoding plain and simple is not going as planned.
Many thanks in advance.
Hello! It’d be like
var items = tj_decode(source_string); // [{ ... }, { ... }] var count = array_length_1d(items); for (var i = 0; i < count; i++) { var item = items[i]; // { "ID": 0, "NAME": ... } var item_name = tj_get(item, "NAME"); var item_obj = asset_get_index(tj_get(item, "OBJ")); }Hi Yal,
This extension has been a great utility for my project, however I want to know if this is possible at all:
e.g.:
team.members.person1.role = “New Role”
I was not sure how to do the same thing above (updating a child level object’s field) using “tj_set()”.
Does “tj_set()” work on top-level only? is there a workaround to it?
Hi again, never mind the question, wow, I just found that it works, you only need to reference the child object you want to update.
So I made a function “data_deep_get()” to get to the tjson object node level I want, along the way checking things using the tjson functions to avoid errors, and finally returns a value.
tjson = tj_decode(jsonstr);
person_obj = data_deep_get(tjson,”team”,”members”,”person1″);
tj_set(person_obj, “role”, “new role”);
Then when I do:
show_debug_message(tjson);
It will show that the particular field’s new value has really been set.
Really, you did a great work on this extension, thank you very much.
Pingback: A summary of my GameMaker assets
Oh wow this is great!
I had been using frosty cat’s JSOnion for a bit, but it was slow and messy.
This on the otherhand seems much cleaner with the use of arrays! (I really do love arrays in GM)
JSOnion pays a pretty big price for GM8 compatibility – both encoding and decoding become exponentially slower as the size of input string/input structure increases (quickly reaching the point where it takes seconds instead of milliseconds to do the operation). Author had explained the issues a little here. It is unusual that it wasn’t written as a DLL originally – that would have solved multiple of it’s problems with data storage and performance in GM8.