Message passing-based inference in hierarchical autoregressive models

December 21, 2022

Albert Podusenko defended his thesis at the department of Electrical Engineering on December 20th.

The traditional design of hearing aid is inefficient as it requires people in the design process to design for a hearing impaired person's unique and context-dependent preferences. In contrast, a wearable synthetic intelligent agent could improve the quality of a hearing aid with on-the-spot suggestions for new hearing aid settings, rather than waiting for the intervention of a human designer. For his PhD research, Albert Podusenko looked at ways to automate the personalized design of hearing aid algorithms through in-the-field communication between a user and a portable intelligent agent.

To create such an agent, Albert Podusenko and his colleagues in the department of Electrical Engineering took inspiration from a theoretical neuroscience framework called the Free Energy Principle, which explains how living brains effectively control their environment by online Bayesian learning of a model of their environment.

According to this hypothesis, an agent (such as a brain) holds a generative probabilistic model for its sensory input signals. Translated to the context of a synthetic agent and an acoustic environment with a hearing aid (HA) and a HA patient, the agent's generative model should comprise a model for both environmental acoustic signals and user appraisals for hearing aid behavior.

These models ought to be learned under in-situ conditions through Bayesian inference, which offers a rigorous procedure for parameter estimation in probabilistic models.

Bayesian inference

Following the premise of the Free Energy Principle, the essence of Podusenko’s approach to automated HA design is that all engineering tasks can be formulated as a Bayesian inference on the generative probabilistic model.

In particular, his research focused on a specific family of models for environmental acoustical signals, namely Hierarchical Autoregressive Models. In principle, the flexibility of these models supports the description of complex non-stationary acoustic environments. Unfortunately, Bayesian parameter estimation in these models is not trivial, and inference solutions do not exist in closed-form. Therefore, Podusenko developed methods to automate Bayesian inference for both state and parameter updating in hierarchical autoregressive models.

Main findings

There are a number of key findings from the work of Podusenko and his colleagues. First, he explored different hierarchical autoregressive models such as continuous time-varying, switching, and coupled autoregressive models. He cast these models into a factor graph framework that provides a convenient visualization of the models. In addition, he shows that hierarchical models build on a network of special building blocks that can be re-used to increase the expressiveness of other dynamical models.

Second, Podusenko realized Bayesian inference using an efficient message passing-based algorithm on these probabilistic factor graphs. He obtained closed-form message passing update rules for hierarchical autoregressive models.

Third, closing in on the final application, he made use of the developed tools for efficient inference in hierarchical autoregressive models to build a synthetic agent that tunes hearing aid parameters under situated conditions. The developed agent solves the classification of acoustic context, infers optimal trial design, and executes the HA signal processing algorithm all by automated Bayesian inference.

In summary, Podusenko’s thesis provides a generic framework for hybrid, efficient, and automatable Bayesian inference on probabilistic graphical models representing hierarchical autoregressive models. All derivations for the inference procedures have been added to the open-source Julia package texttt{ReactiveMP.jl} that focuses on efficient and scalable Bayesian inference.

Title of PhD thesis: Message passing-based inference in hierarchical autoregressive models. Supervisors: Bert de Vries and Wouter Kouw.

Media contact

Barry Fitzgerald
(Science Information Officer)

Latest news

Keep following us