Experimental Development: Building a Text Generator

Input language

Taking my cue from the construction of both Perchance and Tracery, I chose to use the following symbols the signify variables and options: [variable] and {option|option}. A [variable] might represent a call to another structure, or a randomly chosen item from a list. An {option|option} represents an either/or choice – the generator will display one of the options. This is more useful for quickly changing grammar on the surface, or adding ‘stings’ to the end of phrases{; like this, for example|.} And of course, it allows for nested grammars and variables, like {this one, [name].|{this|that [adjective]} version [location]}!

By choosing a familiar language I avoided the pitfalls of ‘reinventing the wheel’, and could take advantage of a ‘reverse engineering’ mindset in order to build this generator from scratch.

Input method

I decided on using the .json format, as it is lightweight and simple to write. A json consists of ‘tiers’ of information organised into {objects}, values and arrays[]. Here’s an excerptof a json with multiple objects, values and arrays:

The major upside to working with jsons is that they don’t require anyone to open Unity or search through any code in order to edit the generator’s content – if I were collaborating with someone on this generator, they could add in as much as they wanted, and as long as their entries followed the correct structure they would be absorbed by the generator with ease. I understand that there are Excel-to-json convertors that make this pipeline even easier…

Importing Jsons

Well, this was boring to learn. Apparently Unity’s JsonUtility doesn’t speak directly to Jsons that contain arrays, so you have to create a ‘wrapper’ class to import the file. For example, here’s my GenericJsonArray class:

Then you have to declare your regular Json class – in this case, GenericJson, each of which contains a single string ‘id’ and an array of strings called ‘choices.’

In the event of needing more complex nesting – for example, with each level’s concept, which contains multiple string arrays of ‘references’ and ‘aesthetics’ – I needed to define a whole other pair of classes. Below are my ConceptArray and Concept classes:

With the classes defined, a single line of code ‘deserializes’ the json (attached as a public TextAsset) into game-readable information. I kept all of this code in a single script called LoadJSON.cs.

Deserializing all the json containers for the game!

Phew! (Apparently external tools like NewtonSoft’s Json.Net can work more directly with jsons than Unity’s built-in tools; I found them to be just as difficult to use, but I think that speaks more to my lack of experience than to the quality of the tool.)

Filling the dictionary, setting constants and selecting structures

The first thing that TextSelector.cs does is fill a Dictionary with terms. This was my first time working with the Dictionary class, and I found it a really useful way of accessing variables – it’s essentially a container for pairs of information, organised into a ‘key’ and a ‘value’. A series of forloops enter the ‘id’ of almost every GenericJson (things, adjectives, inserts, features, verbs) into the key column, and a randomly selected ‘choice’ from that ‘id’ in the value column. That way, if a [variable] is encountered that matches one of the Dictionary’s keys, then it can immediately access the value at that index! (Dictionaries aren’t limited to pairs of strings – by using ints or floats one could tie most kinds of simple, game-pertinent information to another value…)

CreateDictionary()

Next, the script sets the constants for the level – level name, concept, and the features most prominent in the level. It can receive guidance from the level generation script in this, but mostly it’s just making random choices.

The subsidiary functions of SetConstants()

Finally, it chooses the appropriate structures through three different functions: SetStartText(), Populate() and SetEndText(). SetStartText() draws from the ‘introductions’ list in frames.json; Populate() assigns a randomly chosen structure from structures.json, and SetEndText() chooses from either a ‘generic’ signoff or a ‘spooky’ one, again stored in frames.json. (Why store the structures in different jsons? So the code just has to worry about returning [Random.Range(json.options.Count)] rather than knowing which index to pick! With this approach I can also add new content arrays into structures.json without worrying about pointing to the wrong index. In fact, in polishing for submission, I might even split frames.json into startstructures.json and endstructures.json for even more programming clarity…)

Parsing variables

But how do we know what a variable contains? It would be extremely performance-intensive to parse every character into lists of words and try to match each one to the Dictionary. Instead, I used regular expressions – or ‘regex’ – to parse each string for a number of matches, and String.Replace() to alter the structure in question.

Regex are sequences of characters used in string searching, and also in find-and-replace functions. They also look like gibberish – luckily, regex101.com allows you to test your regex on a string, as well as to consult a database of legal expressions.

My parsing function uses three different regex(es?) to process a string:

  • {options} – basically, this regex searches for anything between a pair of curly brackets, and includes the curly brackets themselves by using a ‘look behind’ and ‘look ahead’ as capture groups. The output is helpfully organised into two groups – {options} and options. If there are any matches, we increment the integer matchCount, save the first group (which will be replaced later) and process the second like so:
The beginning of ParseText(), using a regex to find a group of characters between two curly brackets
  • split|split – a new Regex is applied to every {options} result, looking for any series of characters on either side of a vertical slash. It will then choose one of these ‘splits’, and replace the entirety of {options} with that split. Importantly, in the case of {this option {another option|and another} | or this one} the function will ignore {this option {another option|and another} | or this one}, even though it’s possible for the ‘top level’ option to remove the ‘lower level’ option from consideration. This isn’t super-performant, but the Regex required to define anything more complex than this is basically magic to me. Since the function loops back on itself, it also avoids any errors caused by dangling {‘s or }’s. At this point we reset the matchCount to 0.
ParseText() splitting function
  • [variables] – with one round of simple {options} parsed, it’s time to move on to matching for [variables]. Luckily, the same capturing rules apply from my first Regex, returning [variable] as well as variable. If there are any matches, we increment the matchCount, then as long as the string variable is a match for one of the Dictionary’s keys, I can replace [variable] with that key’s value, which is a random choice from the ‘id’! (If there’s no Dictionary match, I have it output ‘NOVAR’ to signify a ‘broken’ or ‘dangling’ variable.)
ParseText() variable replacement code

If, by the end of the function, the matchCount has not reached zero, it loops back to the first {options} check, and continues looping until the matchCount rests at zero. Then the function returns the final version of the initial text so it can be displayed to the player!

Within the parsing function I have also secreted the RegenerateSeeds() function, which essentially causes every random variable in the Dictionary to ‘reroll’. I’m sure there’s a more performant way of doing this, but I couldn’t find a safe way of altering the Dictionary while it was being searched.

RegenerateSeeds()

Useful modifiers

For ease of writing (and my own sanity) I implemented the following extra modifiers as regex checks of their own:

  • [c.variable] – if the function encounters a ‘c.’ in a variable string, it will return the result but with the first letter in upper case, using C#’s String.ToUpper() method.
  • [r.variable] – by prefacing a variable with ‘r.’, I instruct the program to return the variable in duplicate, depending on the number of levels the player has visited. This results in unpunctuated, repetitive stream of consciousness, a technique that can quickly create a sense of acceleration, loss of control, or sensory overload.
  • {number-number} – if the ‘split’ regex encounters two numbers separated by a dash, it knows to return a random number between the first and second number.
  • {*|*} – if the ‘split’ regex encounters an asterix, it knows to replace it with [*], which I use to represent an empty string. This is handy if I need to quickly change the likelihood of a particular option being chosen, eg. {x|*|*} has a one in three chance to return ‘x’.

Possible tweaks

A method I have yet to code (and may run out of time in which to do so) is the ‘spookifier’, a way of tracking how long the player has gone without seeing a ‘spooky’ bit of text [reference procedural storytelling]. I imagine this could be achieved by adding a variable prefix – let’s call it [sp.] – which will only return the following variable if an integer is high enough, and on doing so resets the integer to 0. The integer would increment every time a string is fully parsed, as long as it wasn’t reset during that parse. This approach would yield a more consistently structured result than the current approach, which leaves ‘spooky’ interventions up to chance.