Abstract:
This thesis addresses the problem of representing and learning qualitative
models of behaviour in complex virtual worlds. It presents a novel representation,
‘Q-Systems’, that integrates two existing representation frameworks:
qualitative process models and action description languages. QSystems
combines the expressive power of both frameworks to allow actions
and world dynamics to be modelled in a common way using a representation
based on non-deterministic and probabilistic finite state machines.
The representation supports learning and planning by using a
modular approach that partitions world behaviour into ‘systems’ of objects
with specific contexts and a related behaviour.
Q-Systems was developed and tested using an agent in a rich simulated
world that was created as part of the thesis. The simulation uses
a rigid body physics engine to produce complex realistic interactions between
objects. An action system and a qualitative vision system were also
developed to allow the agent to observe and act in the simulated world.
The thesis includes a proposed two stage learning process comprising
an initial stage in which ‘histories’ (contextually and temporally restricted
sequences of observations) are extracted from interactions with the simulation,
and a second stage in which the histories are generalised to create a
knowledge base of system models. An algorithm for generating histories
is presented and a number of heuristics are implemented and compared.
A system for learning generalised models is presented and it is used to
assess the suitability of Q-Systems with respect to learning in complex environments.
Planning with Q-Systems is demonstrated in an agent which reasons with generalised models to work out how to achieve goals in the simulated
world. A simple planning algorithm is described and a variety of
issues are explored. Planning with a single system is shown to be relatively
straightforward due to the modular nature of Q-Systems.
This thesis demonstrates that Q-Systems successfully integrate two different
representation frameworks and that they can be used in learning
and planning in complex environments. The initial results are promising,
but further investigation is required to fully understand the advantages
and disadvantages of the Q-System approach compared with existing
learning systems. This would involve the development of benchmark
problems (currently there are none for this particular domain).
