Numerous serious exergames advocate the use of engaging avatars to motivate a consistent exercise regimen. However, the process of specifying the prescribed exercise, implementing it as avatar animation, and developing an accurate feedback-providing mechanism is complex and requires a high level of expertise in game engines, control languages, and hardware devices. Furthermore, in the context of rehabilitation exergames, the requirements for accurate assessment and timely and precise feedback can be quite stringent. At the same time, the Kinect

$$^{TM}$$  motion-capture sensor offers a natural interface to game consoles, and its affordability and wide availability represents a huge opportunity for at-home exergames. In this paper, we describe our work towards a system that envisions to simplify the process of developing rehabilitation exergames with Kinect $$^{TM}$$ . The system relies on a language for specifying postures and movements between them, and includes an editor that enables rehabilitation therapists to specify the prescribed exercise, by editing a demonstration of the exercise. This exercise-specification grammar is used to drive the animation of an avatar and the provision of quality feedback, by comparing the player’s postures (as captured by the Kinect $$^{TM}$$ ) against those of the coaching avatar and the grammar.