Test Suites

Every implementation of the Foundation Library should have its own test suite. This page documents how a test suite works, its goals and usage.

---

Table of Contents

• Introduction
• Test Suite Folder Location
• Test Files

---

Introduction

The library tests are designed to check the library integrity, both in terms of code behaviour and of linguistic correctness of library messages, especially for locales that require proper accordance of gender and number.

Since the generated tests transcripts (.a3t) are tracked by Git, any changes in the output will show up in the Git work areas as modified files, which allows to quickly track changes via status diffing.

Whenever the library sources are modified, the changes will immediately affect the test suite transcripts, allowing developers to check that the intended results are achieved, but also to detect any unforeseen changes (i.e. unexpected side effects in other areas of the library).

Creating a solid test suite, which covers extensively all library features, verbs, and edge cases, is fundamental for developing and maintaining the library code. Without proper tests coverage bugs might go by unnoticed, causing code entanglements that might be harder to fix if not caught immediately.

Thanks to the test suite, we were able to detect faulty logic in the original libraries code — e.g. CHECKs or conditional verb bodies that could never be executed. It's therefore very important that every library feature be tested, under every possible condition and variation, in order to ensure that all possible outcomes are tested, including intentional errors and failures.

Test Suite Folder Location

Library tests should be located in the tests/ subfolder of the locale-specific library implementation, e.g.

• alan_en/tests/ — Test suite of the English library.
• alan_es/tests/ — Test suite of the Spanish library.
• alan_it/tests/ — Test suite of the Italian library.
• alan_sv/tests/ — Test suite of the Swedish library.

The repository toolchain is automated via Rake which automatically builds test suite folders using file pattern rules.

New libraries need to ask the repository maintainers to enable their test suite folder in the Rake toolchain; this is a one-time-operation.

Once the test suite is enabled, Rake will automatically detect any new test files added to the tests/ folder, along with their dependencies, and automatically rebuild any tests whenever their dependencies have been updated.

The toolchain automatically assumes that the tests of any library implementation depend on the library sources found in the Foundation/ subfolder of that locale implementation:

• alan_en/ — English library:
  • tests/ — Test suite.
  • Foundation/ — Library sources.
• alan_es/ — Spanish library:
  • tests/ — Test suite.
  • Foundation/ — Library sources.

Since the repository is structured using consistent folders and files naming conventions, Rake is able to automatically infer the location of the library sources for any locale specific implementation, when calculating tests dependencies.

Test Files

Each test comprises a single adventure (.alan) to which one or more solution files (.a3s) are associated by sharing the same base name. E.g.:

file type	source	output	file type
source adventure	actors.alan	actors.a3c	compiled storyfile
solution	actors.a3s	actors.a3t	transcript
solution	actors-named.a3s	actors-named.a3t	transcript

The Rake toolchain automatically handles all tests files by associating each test adventure with its solution files according to the former's base name, via the pattern:

• <name>.alan → <name>*.a3s

Hence, there's no need to "register" any new tests that are added to the folder, Rake will pick them up at execution time.

IMPORTANT! — Because test files are distinguished via their base name, there can't be a source adventures whose name is is fully contained in the initial name part of another adventure. E.g. including the following adventures in the folder is illegal:

• actors.alan
• actors-named.alan

because the pattern matching mechanism will cause the solution files of the latter to be executed also on the former, since they'd always match the actors*.a3s pattern.

But the following is OK, since their base names differ:

• actors-unnamed.alan
• actors-named.alan