Controlling Information Density in Discourse Generation
Project A7
The goal of A7 is to develop a natural language generation (NLG) system which generates building instructions in Minecraft. The NLG system implements a rational speaker model which trades off succinctness against clarity of the instructions. In particular, it rationally chooses the level of abstraction at which the parts of the construction are explained: for a human listener with sufficient domain knowledge, the instruction “build a railing on the other side” will be very efficient, but a listener who does not know what a railing looks like may need a more verbose block-by-block explanation to complete the construction successfully.
In the previous phase, we have developed such an NLG system through an innovative combination of a hierarchical planner with a sentence generator. The hierarchical planner determines at which level of abstraction the individual building steps will be explained, whereas the sentence generator realizes the building instructions in natural language. These components are integrated tightly, in that the sentence generator supplies the cost function on which the planner relies. We have shown in crowdsourcing evaluations that this integrated NLG system guides human users effectively in the construction of buildings in Minecraft, and that the choice of abstraction level significantly impacts task completion times and user satisfaction.
The main theme of A7 in the third phase will be “Minecraft in flux”: The NLG system will adapt to its user over the course of each construction, tailoring its language use to the user’s implicit preferences in order to optimize the clarity-succinctness tradeoff. It will recompute the instruction plan on the fly in response to an improved understanding of the user’s needs, through greatly accelerated neurosymbolic planning techniques. Finally, we will complement the NLG system with a “simulated user”, which follows building instructions in Minecraft instead of generating them. This will allow us to generate the necessary training data for the neurosymbolic planner, while at the same time providing a model of a listener in Minecraft to go along with our rational speaker.
Over the course of the second phase, we have also identified collaboration opportunities with other projects, for whom Minecraft is an attractive domain to test their hypotheses in a behavioural experimental setting. We will specifically dedicate project time in phase III to collaborate with B3 and C3 on ellipsis and referring expressions.
In the previous phase, we have developed such an NLG system through an innovative combination of a hierarchical planner with a sentence generator. The hierarchical planner determines at which level of abstraction the individual building steps will be explained, whereas the sentence generator realizes the building instructions in natural language. These components are integrated tightly, in that the sentence generator supplies the cost function on which the planner relies. We have shown in crowdsourcing evaluations that this integrated NLG system guides human users effectively in the construction of buildings in Minecraft, and that the choice of abstraction level significantly impacts task completion times and user satisfaction.
The main theme of A7 in the third phase will be “Minecraft in flux”: The NLG system will adapt to its user over the course of each construction, tailoring its language use to the user’s implicit preferences in order to optimize the clarity-succinctness tradeoff. It will recompute the instruction plan on the fly in response to an improved understanding of the user’s needs, through greatly accelerated neurosymbolic planning techniques. Finally, we will complement the NLG system with a “simulated user”, which follows building instructions in Minecraft instead of generating them. This will allow us to generate the necessary training data for the neurosymbolic planner, while at the same time providing a model of a listener in Minecraft to go along with our rational speaker.
Over the course of the second phase, we have also identified collaboration opportunities with other projects, for whom Minecraft is an attractive domain to test their hypotheses in a behavioural experimental setting. We will specifically dedicate project time in phase III to collaborate with B3 and C3 on ellipsis and referring expressions.
Keywords: discourse generation
Publications A7
Other Area-A Projects
- Neurobehavioural Correlates of Surprisal in Online Comprehension A1
- The Role of Language Experience and Surprisal for Learning and Memory A5
- Expectancy-based mechanisms during language comprehension and their relation to memory formation and retrieval A6
- Adapting Text Generation to Individual Users A8
Successfully