The multiband paradigm

Introduction

1. The message may be impaired (by noise or reverberation) only in some frequency bands. When recognition is based on several independent decisions from different sub-bands, the decoding of the linguistic message may not be severely impaired, as long as the remaining clean sub-bands supply sufficiently reliable information.
2. Acoustic events, such as the transitions between more stationary segments of speech, do not necessarily occur at the same across the different frequency bands, which make the piecewise stationary assumption more fragile. The sub-band approach may have the potential advantage of relaxing the synchrony constraint inherent in current HMM systems.
3. A better use of the temporal information and, more generally, a better time/frequency compromise can be obtained by optimizing independently each sub-recognizer in terms of temporal resolution and acoustic context.
4. Different recognition strategies including the use of different acoustic parameters could be used in different frequency bands.

A first issue in the design of the system is to determine the number and position of the frequency bands and to choose the acoustical parameters. Once these are determined, the approach presented will fundamentally consist in the combination of the output of multiple recognizers. Each sub-recognizer has its own acoustic model and can generate its own temporal alignment inside the pre-defined lexical sub-units (words, syllable, phones, ...).

Of course, there is less information in a sub-band than in the whole band; the partial decisions may thus be less reliable. To avoid too much flexibility in choosing the time-warping path, it is necessary to re-introduce some constraints at a higher level. This is done by forcing synchrony of the different independent frequency band recognizers at some level, as shown on this figure.

We note that, while recombination at the state level quite easy, it is no longer straightforward at any higher sub-word unit level (simply using the standard one-pass dynamic programming approach). Rather, the system can either use an approach based on the 2-level dynamic time-warping programming, or else an adaptation of HMM decomposition [5] which can be used to do multi-dimensional time-warping and recombination of the frequency sub-bands.

A draft version of a theoretical discussion of the multi-band paradigm, and more generally the temporal integration of multi-stream inputs is available in :

References