Aquila has been changed to permit testing of giface as well as tace. See Latest Changes
Aquila tests are controlled by a simple scripting language. The
language includes an
include command, and the main test
suite is organized as a master file that includes the individual test
files. The master file is at
How to put your test into the test suite
- Log in as the user that owns the test directories, i.e. the AceDB build user
- Place your script in
/nfs/disk100/acedb/aquila/config/script_main/and put any files it refers to into place, which will normally be in
.acefiles etc. (In your script, you can refer to this directory by the name
- Test your script by running Aquila against it. Please see the section on
running Aquila for full details.
Briefly: aquila -s scriptfile program_version_1 aquila -s scriptfile program_version_2 aquila program_version_1> program_version_2
- Add an
/nfs/disk100/acedb/aquila/config/script-- I suggest you put this at the end of the list unless there is an obvious reason to do otherwise.
- The output from the overnight Aquila run appears in ~acedb/aquila. There are plans to make it available on the web but this work hasn't been completed yet.
- The result format is described in How to interpret Aquila test results
You can view the collection of tests currently in use by looking in the latest test results which link through to a web copy of the test scripts used.
The scripting language
The Aquila language is a line-based format, in which each line contains a command and may contain a label.
Some commands form the start of a block of commands, and such blocks may be nested. The labels are used not as jump targets but in the generation of filenames for the stored results files on which the comparisons are done. (If there is no label set, an underscore is used in the file name at that point.) A label given for a command is used in the filenames throughout that command; the label of a block-starting command is used in all commands in that block. The label names for nested blocks are concatenated, with dot as the separator. (The last part of the filename is the name of the command that produced that file.)
Note that labels must be unique within a set of tests. This is because they are used to generate the names of the files used to store the results, and duplicate filenames would obviously cause problems. The next section should make that clear. The difficulty here is that as we've recently moved from a single gigantic script file to dozens of little ones, it's now much easier to accidentally end up with duplicate labels. I think that then the second set of tests is ignored.
Here is an example test script showing the labelling structure:
abnmqry: dbinit wspec.mini // angle bracket name queries tace parse: do parse $data/mini.angleb.ace abfind: do find M "<someth>" abfindw: do Query find M "<someth>" abfindesc: do find M \<someth\> abfindq: do find M <someth> abclear: do Query find M T & NOT T // clear active keyset abfind2: do find M "Wolf D." abfind3: do Query >?M "Wolf D." exit
This produces the followning files in the test directory
-rw-r--r-- 1 acedb acedb 0 Oct 11 10:28 abnmqry._ -rw-r--r-- 1 acedb acedb 88 Oct 11 10:29 abnmqry._.abclear.do -rw-r--r-- 1 acedb acedb 54 Oct 11 10:29 abnmqry._.abfind.do -rw-r--r-- 1 acedb acedb 54 Oct 11 10:29 abnmqry._.abfind2.do -rw-r--r-- 1 acedb acedb 88 Oct 11 10:29 abnmqry._.abfind3.do -rw-r--r-- 1 acedb acedb 54 Oct 11 10:29 abnmqry._.abfindesc.do -rw-r--r-- 1 acedb acedb 54 Oct 11 10:29 abnmqry._.abfindq.do -rw-r--r-- 1 acedb acedb 88 Oct 11 10:29 abnmqry._.abfindw.do -rw-r--r-- 1 acedb acedb 89 Oct 11 10:29 abnmqry._.exit -rw-r--r-- 1 acedb acedb 252 Oct 11 10:29 abnmqry._.parse.do -rw-r--r-- 1 acedb acedb 1049 Oct 11 10:29 abnmqry.dbinit
It is possible to omit the labels from the script. then aquila assumes _: as label, as is done for the "tace" line in the script above.
If two commands would produce the same file name, at least one of them must be labelled to avoid a conflict.
This starts a tace sessions, and contains a block of commands to be applied to that session. It ends with an "exit" command.
The "tace" command does not initialize the database -- you must used dbinit for that.
Note that recently we've altered aquila to permit testing of either tace or giface. We've disabled the -c command line option (gives a warning but is otherwise ignored) and now allow you to specify either tace or giface in the script.
For pragmatic reasons you still have to have tace available in your bin/ directory as it is still used for database initialisation even when you're testing giface.
This ends the block started by a "tace" aquila command. The data recorded is the time taken by that tace session.
Does a tace command from the script file. This must be used within a "tace" aquila command block, as must most of the following commands.
Tells the AceDB session to run a 'show' command, to output the current keyset.
Causes the AceDB session to execute a 'write' command.
Causes the AceDB session to execute a 'list' command.
Records the memory used by the AceDB session at the time.
Records the size of the current keyset. This is often regarded as enough, in terms of checking whether the result of an operation is still OK.
Prepares a fresh database for the test, with a given wspec directory which is copied to configure the new database. This is one of the commands that should be used outside the tace command block.
dbinit -f wspec deletes the database directory
rather than simply removing the ACEDB.wrm file to force
Sets a runtime environment variable for the AceDB session (or any subprocess it starts) to see. This is one of the commands that should be used outside the tace command block.
The result files are meant to be human-readable and platform- independent, here are some examples:
_._ contains: !! 26 objects read with 0 errors _._._.mem contains: sz 7659 KBytes rss 5222 KBytes _.exit contains: aborted 0 exit_status 0 exit_msg A bientot real_time 2 seconds cpu_time 2.18 seconds
However, the main point of the result files is to compare them against those from a reference version, rather than to examine them directly.
Handling commands with long output
The result of an aquila run is meant to be a fingerprint of a specific ace version on a specific platform on a specific machine. Since there will probably be a set of such fingerprints around, they should be small in size, but keep all what is necessary to see deviations between versions. This sections show what aquila provides to keep fingerprints small.
consider this script file:
tace 1: do find Sequence count 2: do -e table sometable.def mem exit
for examining the current keyset there are four commands -- to be used depending on the number of objects retrieved:
write and show -- to see full objects list -- to see only object names count -- to just count
if 'find Sequence' retrieves, say, 100,000 sequences, there is (usually) no point in listing all of them. just counting the number of objects is enough.
For the tablemaker output there is no correspondence, so aquila allows to limit the number of lines recorded. if sometable.def generated, say 100,000 output lines, do -e just records the 'ends' of the output, i.e. the first and the last line and a count of lines in between.
managing multiple ACEDB schemata
ACEDB is applied in a variety of environments using different database schemata. The schema of an ACEDB database is stored as a set of files in subdirectory called 'wspec'. These different wspecs are stored in the aquila/config/ directory, e.g
this is used in the following way in a script:
default: dbinit tace mem // check base memory requirement do classes mem // and after loading lexiques exit igd: dbinit wspec.igd tace load: do pparse longline.ace find: do find Sequence "gb:X561*" count exit ace4: dbinit wspec.ace4mieg sample: tace do pparse sample.ace mem exit similar: tace do -e Table tables/similar2.def mem exit
At execution time, a symbolic link from <ace_version>/wspec
to the specified config/wspec.x is made.
dbinit without a wspec parameter is equivalent to
Usually, it is enough to erase the file database/ACEDB.wrm to force
initialization of the database. So, by default aquila does just this.
If you suspect that this feature of ACEDB is bugged (as was in some ace33)
this erases all files in database/ at startup time.
Manipulating the environment:
the operation of some acedb commands are affected by environment variables. aquila can set those for the duration of one tace block, e.g.
setenv ACEDB_DUMP_ATTACH 1 t1: tace do find Sequence write exit t2: tace do find Sequence write exit
then t1 should dump with @queries, since ACEDB_DUMP_ATTACH is set; the setenv is effective until the next exit statement. thus t2 executes without ACEDB_DUMP_ATTACH.
Combining Multiple Tests
It is not good to pack tests together into one database. Each test should start at a fresh database, even if this triples the execution time of the test suite.
- What belongs to a complete test set?
the whole contents of the config directory, i.e.
- - the driving script
- - .ace, .ace.Z, .ace.gz data
- - .cmd command batch files
- - tables/*.def table defintions
- - wspec* schemata
- - .ace, .ace.Z, .ace.gz data
there are more examples in the distribution package, please see files config/script.example.*
Latest ChangesAquila can now be used to test both giface and tace binaries, and you can mix such tests, provided any one script is limited to testing one or other of the two. In our automated overnight regression testing, for example, most scripts specify tace as the binary, but the gff script specifies giface.
We have disabled the old -c flag (it now generates a warning but is otherwise ignored) which told aquila which binary to test, and altered aquila to allow it to determine the binary from the script on the fly. For pragmatic reasons, it still requires a tace binary in the bin/ directory in order to initialise the database, but once that's done, it uses the script commands to determine the binary to test.
And obviously if you run aquila to test both tace and giface, you're going to need both binaries in your bin/ directory.
To test giface instead of tace, simply specify giface on the second line of your script. So, for instance, in the first example from the top of this page, the new script would start like this:
abnmqry: dbinit wspec.mini // angle bracket name queries
parse: do parse $data/mini.angleb.ace
Back to top of page