Despite their basic attractiveness as an interaction paradigm for controlling intelligent environments, the design of spoken dialog systems for this purpose raises some usability challenges that require careful attention. This paper examines closely the communication failures that can occur in the control of one particular type of intelligent environment: a smart home system that provides control for multiple domestic devices through a state-of-the-art mixed-initiative spoken-dialog interface. The 24 participants completed several tasks with the INSPIRE system in a controlled experiment, and interaction failures were categorized with an error taxonomy that is related to more general error taxonomies but specialized to this class of systems. Despite efforts devoted to supporting natural, mixed-initiative dialog and to the prevention of communication failures, over one fourth of user utterances were problematic, often leading to stagnation or regression. The causes and consequences of these problems are discussed, along with their implications for the design of spoken dialog systems for intelligent environments.