Progress in the availability of efficient and accurate automatic speech recognisers (ASR) has meant that many computer-based tasks, such as editing text documents, can now be carried out without the use of keyboard or mouse, simply by using speech commands. This is true to a lesser extent for the area of mathematics, where entering equations by voice still poses a considerable problem. The ultimate aim of this miniproject was to improve how people can engage with electronic mathematical content. The contribution of the miniproject holders would be the design and implementation of a user-friendly application that allowed for the creation and manipulation/editing of mathematical content via speech, using commercially available ASR software as a front-end. The application would be run in a Windows environment together with the commercial ASR package Dragon NaturallySpeaking (DNS). It would be easy to use even by non-experts, and allow for both creation of new, and editing of existing, mathematical content. It would be available to the public as free ‘open-source’ software.
At the time of this miniproject being initiated, a prototype application called TalkMaths, using DNS as a front-end, was in development at Kingston University. TalkMaths relied on an unmaintained software library to communicate with DNS. This was suitable as a prototype at Kingston University, but to make TalkMaths compatible with future versions of DNS and to create a standalone application, which would be easier to distribute, it would be necessary to purchase a particular Software Development Kit (SDK). The project extended the existing TalkMaths functionality using this SDK, with the aim of creating a flexible application that was simple to install for users of DNS.
This application would be highly beneficial to students and staff that cannot access electronic mathematical content via traditional input devices (such as keyboard and/or mouse), for example, due to a wide variety of physical disabilities. It also aimed to allow the user to dictate and present mathematical text without having in-depth knowledge of any mathematical formatting language.
A range of research issues arise from this project, including designing efficient algorithms for parsing natural language input containing ambiguities, designing ‘standards’ for speaking mathematics, and investigating the usability of Speech User Interfaces and extending the accessibility of mathematics to disabled people.
The initial goal of removing the dependency of TalkMaths on the NatLink library, in order to improve the ease of installation and prevent potential future software compatibility problems was achieved. In fact, beyond just creating a desktop application as planned, a web application was created. This web application is available for free use, and cooperates with a speech front-end written using the DNS SDK. It can accept input either by speech or using the keyboard and mouse.