Papers

2025

1. ODIN

   U-Mamba2: Scaling State Space Models for Dental Anatomy Segmentation in CBCT

   Zhi Qin Tan, Xiatian Zhu, Owen Addison, and 1 more author

   In MICCAI Oral and Dental Image aNalysis (ODIN) Workshop, 2025

   Abs PDF Code

   Cone-Beam Computed Tomography (CBCT) is a widely used 3D imaging technique in dentistry, providing volumetric information about the anatomical structures of jaws and teeth. Accurate segmentation of these anatomies is critical for clinical applications such as diagnosis and surgical planning, but remains time-consuming and challenging. In this paper, we present U-Mamba2, a new neural network architecture designed for multi-anatomy CBCT segmentation in the context of the ToothFairy3 challenge. U-Mamba2 integrates the Mamba2 state space models into the U-Net architecture, enforcing stronger structural constraints for higher efficiency without compromising performance. In addition, we integrate interactive click prompts with cross-attention blocks, pre-train U-Mamba2 using self-supervised learning, and incorporate dental domain knowledge into the model design to address key challenges of dental anatomy segmentation in CBCT. Extensive experiments, including independent tests, demonstrate that U-Mamba2 is both effective and efficient, securing top 3 places in both tasks of the Toothfairy3 challenge. In Task 1, U-Mamba2 achieved a mean Dice of 0.792, HD95 of 93.19 with the held-out test data, with an average inference time of XX (TBC during the ODIN workshop). In Task 2, U-Mamba2 achieved the mean Dice of 0.852 and HD95 of 7.39 with the held-out test data. The code is publicly available at https://github.com/zhiqin1998/UMamba2.

2. ODIN

   U-Mamba2-SSL for Semi-Supervised Tooth and Pulp Segmentation in CBCT

   Zhi Qin Tan, Xiatian Zhu, Owen Addison, and 1 more author

   In MICCAI Oral and Dental Image aNalysis (ODIN) Workshop, 2025

   Abs PDF Code

   Accurate segmentation of teeth and pulp in Cone-Beam Computed Tomography (CBCT) is vital for clinical applications like treatment planning and diagnosis. However, this process requires extensive expertise and is exceptionally time-consuming, highlighting the critical need for automated algorithms that can effectively utilize unlabeled data. In this paper, we propose U-Mamba2-SSL, a novel semi-supervised learning framework that builds on the U-Mamba2 model and employs a multi-stage training strategy. The framework first pre-trains U-Mamba2 in a self-supervised manner using a disruptive autoencoder. It then leverages unlabeled data through consistency regularization, where we introduce input and feature perturbations to ensure stable model outputs. Finally, a pseudo-labeling strategy is implemented with a reduced loss weighting to minimize the impact of potential errors. U-Mamba2-SSL achieved an average score of 0.872 and a DSC of 0.969 on the validation dataset, demonstrating the superior performance of our approach. The code is available at https://github.com/zhiqin1998/UMamba2.

3. SP

   Differentiable Interacting Multiple Model Particle Filtering

   John-Joseph Brady, Yuhui Luo, Wenwu Wang, and 2 more authors

   Signal Processing, 2025

   Abs PDF Code

   We propose a sequential Monte Carlo algorithm for parameter learning when the studied model exhibits random discontinuous jumps in behaviour. To facilitate the learning of high dimensional parameter sets, such as those associated to neural networks, we adopt the emerging framework of differentiable particle filtering, wherein parameters are trained by gradient descent. We design a new differentiable interacting multiple model particle filter to be capable of learning the individual behavioural regimes and the model which controls the jumping simultaneously. In contrast to previous approaches, our algorithm allows control of the computational effort assigned per regime whilst using the probability of being in a given regime to guide sampling. Furthermore, we develop a new gradient estimator that has a lower variance than established approaches and remains fast to compute, for which we prove consistency. We establish new theoretical results of the presented algorithms and demonstrate superior numerical performance compared to the previous state-of-the-art algorithms.

4. arXiv

   Foundational Models for 3D Point Clouds: A Survey and Outlook

   Vishal Thengane, Xiatian Zhu, Salim Bouzerdoum, and 2 more authors

   arXiv preprint arXiv:2501.18594, 2025

   Abs PDF Code

   The 3D point cloud representation plays a crucial role in preserving the geometric fidelity of the physical world, enabling more accurate complex 3D environments. While humans naturally comprehend the intricate relationships between objects and variations through a multisensory system, artificial intelligence (AI) systems have yet to fully replicate this capacity. To bridge this gap, it becomes essential to incorporate multiple modalities. Models that can seamlessly integrate and reason across these modalities are known as foundation models (FMs). The development of FMs for 2D modalities, such as images and text, has seen significant progress, driven by the abundant availability of large-scale datasets. However, the 3D domain has lagged due to the scarcity of labelled data and high computational overheads. In response, recent research has begun to explore the potential of applying FMs to 3D tasks, overcoming these challenges by leveraging existing 2D knowledge. Additionally, language, with its capacity for abstract reasoning and description of the environment, offers a promising avenue for enhancing 3D understanding through large pre-trained language models (LLMs). Despite the rapid development and adoption of FMs for 3D vision tasks in recent years, there remains a gap in comprehensive and in-depth literature reviews. This article aims to address this gap by presenting a comprehensive overview of the state-of-the-art methods that utilize FMs for 3D visual understanding. We start by reviewing various strategies employed in the building of various 3D FMs. Then we categorize and summarize use of different FMs for tasks such as perception tasks. Finally, the article offers insights into future directions for research and development in this field. To help reader, we have curated list of relevant papers on the topic: this https URL.

5. PO

   Do monetary incentives encourage local communities to collect and upload mosquito sound data using smartphones? A case study in the Democratic Republic of the Congo

   Kieran Storer, Jane Messina, Eva Herreros-Moya, and 11 more authors

   PLoS One, 2025

   Abs PDF

   Malaria is one of the deadliest vector borne diseases affecting sub-Saharan Africa. A suite of systems are being used to monitor and manage malaria risk and disease incidence, with an increasing focus on technological interventions that allow private citizens to remotely record and upload data. However, data collected by citizen scientists must be standardised and consistent if it is to be used for scientific analysis. Studies that aim to improve data collection quality and quantity have often included incentivisation, providing citizen scientists with monetary or other benefits for their participation in data collection. We tested whether monetary incentives enhance participation and data collection in a study trialling an acoustic mosquito sensor. Working with the community in two health areas in the Democratic Republic of Congo, we measured data collection participation, completeness, and community responses. Our results showed mixed responses to the incentive, with more participants interested in the social status and monetary value of the technology used than the monetary incentive itself. The effect of incentives on data collection varied over the course of the trial, increasing participation in the start of the trial but with no effect in the latter half of the trial. Feedback from participants showed that opinions on technology, research objectives, and incentives varied between communities, and was associated with differences in data collection quantity and quality, suggesting that differences in community interest in data collection and the incentives may be more important than the incentive value itself. These results suggest that though there is an initial benefit, extrinsic motivations do not override differences in intrinsic motivations over time, and enhanced communication and dialogue with participants may improve citizen science participation and attitudes.

6. DM

   Predicting restoration failures in primary and permanent teeth – A machine learning approach

   Vitor Henrique Digmayer Romero, Eduardo Trota Chaves, Shankeeth Vinayahalingam, and 11 more authors

   Dental Materials, 2025

   PDF
7. DM

   An optical model for evaluating the influence of microstructures on zirconia translucency

   Linfeng He, Qijing Li, Du Chen, and 6 more authors

   Dental Materials, 2025

   PDF
8. IDJ

   AI-Driven Dental Caries Management Strategies: From Clinical Practice to Professional Education and Public Self Care

   Yutong Liang, Dongling Li, Dongmei Deng, and 6 more authors

   International Dental Journal , 2025

   Abs PDF

   Dental caries is one of the most prevalent chronic diseases among both children and adults, despite being largely preventable. This condition has significant negative impacts on human health and imposes a substantial economic burden. In recent years, scientists and dentists have increasingly started to utilize artificial intelligence (AI), particularly machine learning, to improve the efficiency of dental caries management. This study aims to provide an overview of the current knowledge about the AI-enabled approaches for dental caries management within the framework of personalized patient care. Generally, AI works as a promising tool that can be used by both dental professionals and patients. For dental professionals, it predicts the risk of dental caries by analyzing dental caries risk and protective factors, enabling to formulate personalized preventive measures. AI, especially those based on machine learning and deep learning, can also analyze images to detect signs of dental caries, assist in developing treatment plans, and help to make a risk assessment for pulp exposure during treatment. AI-powered tools can also be used to train dental students through simulations and virtual case studies, allowing them to practice and refine their clinical skills in a risk-free environment. Additionally, AI tracks brushing patterns and provides feedback to improve oral hygiene practices of the patients and the general population, thereby improving their understanding and compliance. This capability of AI can inform future research and the development of new strategies for dental caries management and control.

2024

1. TSP

   Normalizing Flow-based Differentiable Particle Filters

   Xiongjie Chen, and Yunpeng Li

   IEEE Transactions on Signal Processing (TSP), 2024

   Abs HTML PDF Code Slides

   Recently, there has been a surge of interest in incorporating neural networks into particle filters, e.g. differentiable particle filters, to perform joint sequential state estimation and model learning for non-linear non-Gaussian state-space models in complex environments. Existing differentiable particle filters are mostly constructed with vanilla neural networks that do not allow density estimation. As a result, they are either restricted to a bootstrap particle filtering framework or employ predefined distribution families (e.g. Gaussian distributions), limiting their performance in more complex real-world scenarios. In this paper we present a differentiable particle filtering framework that uses (conditional) normalizing flows to build its dynamic model, proposal distribution, and measurement model. This not only enables valid probability densities but also allows the proposed method to adaptively learn these modules in a flexible way, without being restricted to predefined distribution families. We derive the theoretical properties of the proposed filters and evaluate the proposed normalizing flow-based differentiable particle filters’ performance through a series of numerical experiments.

2. ECCV

   Bayesian Detector Combination for Object Detection with Crowdsourced Annotations

   Zhi Qin Tan, Olga Isupova, Gustavo Carneiro, and 2 more authors

   In European Conference on Computer Vision (ECCV), 2024

   Abs HTML PDF Code Slides

   Acquiring fine-grained object detection annotations in unconstrained images is time-consuming, expensive, and prone to noise, especially in crowdsourcing scenarios. Most prior object detection methods assume accurate annotations; A few recent works have studied object detection with noisy crowdsourced annotations, with evaluation on distinct synthetic crowdsourced datasets of varying setups under artificial assumptions. To address these algorithmic limitations and evaluation inconsistency, we first propose a novel Bayesian Detector Combination (BDC) framework to more effectively train object detectors with noisy crowdsourced annotations, with the unique ability of automatically inferring the annotators’ label qualities. Unlike previous approaches, BDC is model-agnostic, requires no prior knowledge of the annotators’ skill level, and seamlessly integrates with existing object detection models. Due to the scarcity of real-world crowdsourced datasets, we introduce large synthetic datasets by simulating varying crowdsourcing scenarios. This allows consistent evaluation of different models at scale. Extensive experiments on both real and synthetic crowdsourced datasets show that BDC outperforms existing state-of-the-art methods, demonstrating its superiority in leveraging crowdsourced data for object detection. Our code and data are available at: https://github.com/zhiqin1998/bdc.

3. FUSION

   Regime Learning for Differentiable Particle Filters

   John-Joseph Brady, Yuhui Luo, Wenwu Wang, and 2 more authors

   In International Conference on Information Fusion (FUSION), 2024

   Abs HTML PDF Code

   Differentiable particle filters are an emerging class of models that combine sequential Monte Carlo techniques with the flexibility of neural networks to perform state space inference. This paper concerns the case where the system may switch between a finite set of state-space models, i.e. regimes. No prior approaches effectively learn both the individual regimes and the switching process simultaneously. In this paper, we propose the neural network based regime learning differentiable particle filter (RLPF) to address this problem. We further design a training procedure for the RLPF and other related algorithms. We demonstrate competitive performance compared to the previous state-of-the-art algorithms on a pair of numerical experiments.

4. MIUA

   H-FCBFormer: Hierarchical Fully Convolutional Branch Transformer for Occlusal Contact Segmentation with Articulating Paper

   Ryan Banks, Bernat Rovira-Lastra, Jordi Martinez-Gomis, and 2 more authors

   In UK Conference on Medical Image Understanding and Analysis (MIUA), 2024

   Abs HTML PDF Code

   Occlusal contacts are the locations at which the occluding surfaces of the maxilla and the mandible posterior teeth meet. Occlusal contact detection is a vital tool for restoring the loss of masticatory function and is a mandatory assessment in the field of dentistry, with particular importance in prosthodontics and restorative dentistry. The most common method for occlusal contact detection is articulating paper. However, this method can indicate significant medically false positive and medically false negative contact areas, leaving the identification of true occlusal indications to clinicians. To address this, we propose a multiclass Vision Transformer and Fully Convolutional Network ensemble semantic segmentation model with a combination hierarchical loss function, which we name as Hierarchical Fully Convolutional Branch Transformer (H-FCBFormer). We also propose a method of generating medically true positive semantic segmentation masks derived from expert annotated articulating paper masks and gold standard masks. The proposed model outperforms other machine learning methods evaluated at detecting medically true positive contacts and performs better than dentists in terms of accurately identifying object-wise occlusal contact areas while taking significantly less time to identify them. Code is available at https://github.com/Banksylel/H-FCBFormer.

5. SAM

   Revisiting semi-supervised training objectives for differentiable particle filters

   Jiaxi Li, John-Joseph Brady, Xiongjie Chen, and 1 more author

   In IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM), 2024

   Abs PDF

   Differentiable particle filters combine the flexibility of neural networks with the probabilistic nature of sequential Monte Carlo methods. However, traditional approaches rely on the availability of labelled data, i.e., the ground truth latent state information, which is often difficult to obtain in real-world applications. This paper compares the effectiveness of two semisupervised training objectives for differentiable particle filters. We present results in two simulated environments where labelled data are scarce

6. ACML

   Folded Hamiltonian Monte Carlo for Bayesian Generative Adversarial Networks

   Narges Pourshahrokhi, Yunpeng Li, Samaneh Kouchaki, and 1 more author

   In Asian Conference on Machine Learning (ACML), 2024

   Abs HTML

   Probabilistic modelling on Generative Adversarial Networks (GANs) within the Bayesian framework has shown success in estimating the complex distribution in literature. In this paper, we develop a Bayesian formulation for unsupervised and semi-supervised GAN learning. Specifically, we propose Folded Hamiltonian Monte Carlo (F-HMC) methods within this framework to learn the distributions over the parameters of the generators and discriminators. We show that the F-HMC efficiently approximates multi-modal and high dimensional data when combined with Bayesian GANs. Its composition improves run time and test error in generating diverse samples. Experimental results with high-dimensional synthetic multi-modal data and natural image benchmarks, including CIFAR-10, SVHN and ImageNet, show that F-HMC outperforms the state-of-the-art methods in terms of test error, run times per epoch, inception score and Frechet Inception Distance scores.

2023

1. FoDS

   An overview of differentiable particle filters for data-adaptive sequential Bayesian inference

   Xiongjie Chen, and Yunpeng Li

   Foundations of Data Science, 2023

   Abs HTML PDF

   By approximating posterior distributions with weighted samples, particle filters (PFs) provide an efficient mechanism for solving non-linear sequential state estimation problems. While the effectiveness of particle filters has been recognised in various applications, their performance relies on the knowledge of dynamic models and measurement models, as well as the construction of effective proposal distributions. An emerging trend involves constructing components of particle filters using neural networks and optimising them by gradient descent, and such data-adaptive particle filtering approaches are often called differentiable particle filters. Due to the expressiveness of neural networks, differentiable particle filters are a promising computational tool for performing inference on sequential data in complex, high-dimensional tasks, such as vision-based robot localisation. In this paper, we review recent advances in differentiable particle filters and their applications. We place special emphasis on different design choices for key components of differentiable particle filters, including dynamic models, measurement models, proposal distributions, optimisation objectives, and differentiable resampling techniques.

2. Asilomar

   Learning Differentiable Particle Filter on the Fly

   Jiaxi Li, Xiongjie Chen, and Yunpeng Li

   In Asilomar Conference on Signals, Systems, and Computers, 2023

   Abs HTML PDF

   Differentiable particle filters are an emerging class of sequential Bayesian inference techniques that use neural networks to construct components in state space models. Existing approaches are mostly based on offline supervised training strategies. This leads to the delay of the model deployment and the obtained filters are susceptible to distribution shift of test-time data. In this paper, we propose an online learning framework for differentiable particle filters so that model parameters can be updated as data arrive. The technical constraint is that there is no known ground truth state information in the online inference setting. We address this by adopting an unsupervised loss to construct the online model updating procedure, which involves a sequence of filtering operations for online maximum likelihood-based parameter estimation. We empirically evaluate the effectiveness of the proposed method, and compare it with supervised learning methods in simulation settings including a multivariate linear Gaussian state-space model and a simulated object tracking experiment.

3. ICICS

   DRoT: A Decentralised Root of Trust for Trusted Networks

   Loganathan Parthipan, Liqun Chen, Christopher JP Newton, and 3 more authors

   In International Conference on Information and Communications Security (ICICS), 2023

   Abs HTML

   For many years, trusted computing research has focused on the trustworthiness of single computer platforms. For example, how can I decide whether I can trust my personal computer (A) or another computer (B), who communicates with A? In reality, both A and B are part of a computing network, in which there are many other computers, and these computers’ behaviour affects the trustworthiness of any communication between A and B. Obviously, the target of trusted computing is not only to build trusted devices but also trusted networks. Attestation is a mechanism initially designed to ascertain the trustworthiness of a single device. To check on the trustworthiness of a network, we need a network attestation mechanism. The basis of attestation is a root of trust, and research on building roots of trust for individual devices has been successful. One of the next challenges, the most important one, is to create a root of trust for network attestation. In this paper, we introduce our research on designing such a root of trust. This uses devices’ individual roots of trust and a decentralised ledger together with the techniques of “zero trust but verify”, which means that to start with, any entity in the system is not trusted until its functionality can be verified. Based on the verification results, the entities can establish trust. We aim to use such a root of trust to aggregate the attestation evidence and verification results from multiple devices in a network and to achieve trust in the network.

4. ICASSP

   Particle flow Gaussian sum particle filter

   Karthik Comandur, Yunpeng Li, and Santosh Nannuru

   In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

   Abs HTML PDF

   The particle flow Gaussian particle filter (PFGPF) uses an invertible particle flow to generate a proposal density. It approximates the predictive and posterior distributions as Gaussian densities. In this paper, we use a bank of PFGPF filters to construct a Particle flow Gaussian sum particle filter (PFGSPF), which approximates the prediction and posterior as Gaussian mixture model. This approximation is useful in complex estimation problems where a single Gaussian approximation is inadequate. We compare the performance of this proposed filter with the PFGPF and others in challenging numerical simulations.

5. ICASSP

   Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection

   Xiongjie Chen, Yunpeng Li, and Yongxin Yang

   In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023

   Abs HTML PDF

   Out-of-distribution (OOD) detection has recently received much attention from the machine learning community because it is important for deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modeling data distributions in feature spaces. We further incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble stochastic neural networks (BE-SNNs) and overcome the feature collapse problem. We compare the performance of the proposed BE-SNNs with the other state-of-the-art approaches and show that BE-SNNs yield superior performance on several OOD detection benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, Fashion-MNIST vs NotMNIST dataset, and the CIFAR10 vs SVHN dataset.

6. SSP

   Differentiable bootstrap particle filters for regime-switching models

   Wenhan Li, Xiongjie Chen, Wenwu Wang, and 2 more authors

   In IEEE Statistical Signal Processing Workshop (SSP), 2023

   Abs HTML PDF

   Differentiable particle filters are an emerging class of particle filtering methods that use neural networks to construct and learn parametric state-space models. In real-world applications, both the state dynamics and measurements can switch between a set of candidate models. For instance, in target tracking, vehicles can idle, move through traffic, or cruise on motorways, and measurements are collected in different geographical or weather conditions. This paper proposes a new differentiable particle filter for regime-switching state-space models. The method can learn a set of unknown candidate dynamic and measurement models and track the state posteriors. We evaluate the performance of the novel algorithm in relevant models, showing its great performance compared to other competitive algorithms.

2022

1. ICLR

   Augmented sliced Wasserstein distances

   Xiongjie Chen, Yongxin Yang, and Yunpeng Li

   In International Conference on Learning Representations (ICLR), 2022

   Abs PDF Code Slides

   While theoretically appealing, the application of the Wasserstein distance to large-scale machine learning problems has been hampered by its prohibitive computational cost. The sliced Wasserstein distance and its variants improve the computational efficiency through the random projection, yet they suffer from low accuracy if the number of projections is not sufficiently large, because the majority of projections result in trivially small values. In this work, we propose a new family of distance metrics, called augmented sliced Wasserstein distances (ASWDs), constructed by first mapping samples to higher-dimensional hypersurfaces parameterized by neural networks. It is derived from a key observation that (random) linear projections of samples residing on these hypersurfaces would translate to much more flexible nonlinear projections in the original sample space, so they can capture complex structures of the data distribution. We show that the hypersurfaces can be optimized by gradient ascent efficiently. We provide the condition under which the ASWD is a valid metric and show that this can be obtained by an injective neural network architecture. Numerical results demonstrate that the ASWD significantly outperforms other Wasserstein variants for both synthetic and real-world problems.

2. ECML

   Imitation learning with Sinkhorn distances

   George Papagiannis, and Yunpeng Li

   In European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), 2022

   Abs HTML PDF Code Slides

   Imitation learning algorithms have been interpreted as variants of divergence minimization problems. The ability to compare occupancy measures between experts and learners is crucial in their effectiveness in learning from demonstrations. In this paper, we present tractable solutions by formulating imitation learning as minimization of the Sinkhorn distance between occupancy measures. The formulation combines the valuable properties of optimal transport metrics in comparing non-overlapping distributions with a cosine distance cost defined in an adversarially learned feature space. This leads to a highly discriminative critic network and optimal transport plan that subsequently guide imitation learning. We evaluate the proposed approach using both the reward metric and the Sinkhorn distance metric on a number of MuJoCo experiments. For the implementation and reproducing results please refer to the following repository https://github.com/gpapagiannis/sinkhorn-imitation.

3. EUSIPCO

   Conditional measurement density estimation in sequential Monte Carlo via normalizing flow

   Xiongjie Chen, and Yunpeng Li

   In European Signal Processing Conference (EUSIPCO), 2022

   Abs HTML PDF

   Tuning of measurement models is challenging in real-world applications of sequential Monte Carlo methods. Re-cent advances in differentiable particle filters have led to various efforts to learn measurement models through neural networks. But existing approaches in the differentiable particle filter frame-work do not admit valid probability densities in constructing measurement models, leading to incorrect quantification of the measurement uncertainty given state information. We propose to learn expressive and valid probability densities in measurement models through conditional normalizing flows, to capture the complex likelihood of measurements given states. We show that the proposed approach leads to improved estimation performance and faster training convergence in a visual tracking experiment.

4. FUSION

   Particle flow Gaussian particle filter

   Karthik Comandur, Yunpeng Li, and Santosh Nannuru

   In International Conference on Information Fusion (FUSION), 2022

   Abs HTML PDF

   State estimation in non-linear models is performed by tracking the posterior distribution recursively. A plethora of algorithms have been proposed for this task. Among them, the Gaussian particle filter uses a weighted set of particles to construct a Gaussian approximation to the posterior. In this paper, we propose to use invertible particle flow methods, derived under the Gaussian boundary conditions for a flow equation, to generate a proposal distribution close to the posterior. The resultant particle flow Gaussian particle filter (PFGPF) algorithm retains the asymptotic properties of Gaussian particle filters, with the potential for improved state estimation performance in high-dimensional spaces. We compare the performance of PFGPF with the particle flow filters and particle flow particle filters in two challenging numerical simulation examples.

5. BMVC

   Towards unsupervised sketch-based image retrieval

   Conghui Hu, Yongxin Yang, Yunpeng Li, and 2 more authors

   In British Machine Vision Conference (BMVC), 2022

   Abs PDF

   The practical value of existing supervised sketch-based image retrieval (SBIR) algorithms is largely limited by the requirement for intensive data collection and labeling. In this paper, we present the first attempt at unsupervised SBIR to remove the labeling cost (both category annotations and sketch-photo pairings) that is conventionally needed for training. Existing single-domain unsupervised representation learning methods perform poorly in this application, due to the unique cross-domain (sketch and photo) nature of the problem. We therefore introduce a novel framework that simultaneously performs sketch-photo domain alignment and semantic-aware representation learning. Technically this is underpinned by introducing joint distribution optimal transport (JDOT) to align data from different domains, which we extend with trainable cluster prototypes and feature memory banks to further improve scalability and efficacy. Extensive experiments show that our framework achieves excellent performance in the new unsupervised setting, and performs comparably or better than state-of-the-art in the zero-shot setting.

2021

1. NeurIPS

   HumBugDB: A Large-scale Acoustic Mosquito Dataset

   Ivan Kiskin, Marianne Sinka, Adam D Cobb, and 13 more authors

   In Conference on Neural Information Processing Systems (NeurIPS), 2021

   Abs HTML PDF Code

   This paper presents the first large-scale multi-species dataset of acoustic recordings of mosquitoes tracked continuously in free flight. We present 20 hours of audio recordings that we have expertly labelled and tagged precisely in time. Significantly, 18 hours of recordings contain annotations from 36 different species. Mosquitoes are well-known carriers of diseases such as malaria, dengue and yellow fever. Collecting this dataset is motivated by the need to assist applications which utilise mosquito acoustics to conduct surveys to help predict outbreaks and inform intervention policy. The task of detecting mosquitoes from the sound of their wingbeats is challenging due to the difficulty in collecting recordings from realistic scenarios. To address this, as part of the HumBug project, we conducted global experiments to record mosquitoes ranging from those bred in culture cages to mosquitoes captured in the wild. Consequently, the audio recordings vary in signal-to-noise ratio and contain a broad range of indoor and outdoor background environments from Tanzania, Thailand, Kenya, the USA and the UK. In this paper we describe in detail how we collected, labelled and curated the data. The data is provided from a PostgreSQL database, which contains important metadata such as the capture method, age, feeding status and gender of the mosquitoes. Additionally, we provide code to extract features and train Bayesian convolutional neural networks for two key tasks: the identification of mosquitoes from their corresponding background environments, and the classification of detected mosquitoes into species. Our extensive dataset is both challenging to machine learning researchers focusing on acoustic identification, and critical to entomologists, geo-spatial modellers and other domain experts to understand mosquito behaviour, model their distribution, and manage the threat they pose to humans.

2. Globecom

   A survey of technologies for building trusted networks

   Loganathan Parthipan, Liqun Chen, David Gérault, and 4 more authors

   In IEEE Globecom Workshops (GC Wkshps), 2021

   Abs HTML

   In the current generation of networks, there has been a strong focus on security and integrity. In sixth-generation (6G) networks trust will also be an important requirement, but how do we build trust in a network? Many researchers have started to pay attention to this, but research in this field is still at an early stage. Taking our lead from the development of trusted computing for single devices we require a root of trust and a mechanism for reliably measuring and reporting on the state of the network. In this paper, we survey existing technologies that we feel can be used to achieve this. We explore trusted computing technologies that enable a single device to be trusted and suggest how they can be adapted to help build a trusted network. For reporting, we need a mechanism to immutably store measurements on the system. We consider that distributed ledger technologies could fulfil this role as they offer immutability, decentralised consensus, and transparency.

3. FUSION

   Differentiable particle filters through conditional normalizing flow

   Xiongjie Chen, Hao Wen, and Yunpeng Li

   In International Conference on Information Fusion (FUSION), 2021

   Abs HTML PDF

   Differentiable particle filters provide a flexible mechanism to adaptively train dynamic and measurement models by learning from observed data. However, most existing differentiable particle filters are within the bootstrap particle filtering framework and fail to incorporate the information from latest observations to construct better proposals. In this paper, we utilize conditional normalizing flows to construct proposal distributions for differentiable particle filters, enriching the distribution families that the proposal distributions can represent. In addition, normalizing flows are incorporated in the construction of the dynamic model, resulting in a more expressive dynamic model. We demonstrate the performance of the proposed conditional normalizing flow-based differentiable particle filters in a visual tracking task.

4. MEE

   HumBug–an acoustic mosquito monitoring tool for use on budget smartphones

   Marianne E Sinka, Davide Zilli, Yunpeng Li, and 8 more authors

   Methods in ecology and evolution, 2021

   Abs HTML

   1. Mosquito surveys are time-consuming, expensive and can provide a biased spatial sample of occurrence data—the data often representing the location of the surveys, not the occurrence of the mosquitoes. 2. We present the HumBug project, an acoustic system that can turn any Android smartphone into a mosquito sensor. Our sensor has the potential to significantly increase the quantity of mosquito occurrence data as well as access locations that are more difficult to survey by traditional means. 3. We describe our database of wild-captured mosquito fight tone audio data and outline our mosquito detection algorithms that these data train. We also present our MozzWear App, designed to work on budget smartphones, which, together with our HumBug Net (an adapted traditional bednet), facilitates data collection and allows the user to record and directly upload mosquito flight tones from any dwelling with a bednet in the field. 4. Our HumBug system has the potential to vastly increase our understanding of the distribution of mosquito species in space and time and greatly improve surveys needed to assess the success or failure of ongoing vector control measures. At a time when the WHO reports a plateauing in the decade-long decline in malaria mortality rates, this new technological solution for surveying mosquito vectors will provide a timely new resource.

2020

1. ICAIF

   Index tracking with differentiable asset selection

   Yu Zheng, Yunpeng Li, Qiuhua Xu, and 2 more authors

   In ACM International Conference on AI in Finance, 2020

   Abs HTML

   Partial index tracking aims to replicate the performance of a given benchmark index with a small number of its constituents. It can be formulated as a sparse regression problem, but remains challenging due to several practical constraints, especially the fixed number of assets in the portfolio. In this paper, we propose a differentiable relaxation for asset selection, such that we can construct a portfolio with exactly 𝐾 assets, where the objective function can be optimised efficiently via vanilla gradient descent. Our method is backtested with S&P 500 index data from 2002 to 2020. Empirical results demonstrate that our model achieves excellent tracking performance compared with some widely used approaches.

2. NCA

   Bioacoustic detection with wavelet-conditioned convolutional neural networks

   Ivan Kiskin, Davide Zilli, Yunpeng Li, and 3 more authors

   Neural Computing and Applications, 2020

   Abs HTML

   Many real-world time series analysis problems are characterized by low signal-to-noise ratios and compounded by scarce data. Solutions to these types of problems often rely on handcrafted features extracted in the time or frequency domain. Recent high-profile advances in deep learning have improved performance across many application domains; however, they typically rely on large data sets that may not always be available. This paper presents an application of deep learning for acoustic event detection in a challenging, data-scarce, real-world problem. We show that convolutional neural networks (CNNs), operating on wavelet transformations of audio recordings, demonstrate superior performance over conventional classifiers that utilize handcrafted features. Our key result is that wavelet transformations offer a clear benefit over the more commonly used short-time Fourier transform. Furthermore, we show that features, handcrafted for a particular dataset, do not generalize well to other datasets. Conversely, CNNs trained on generic features are able to achieve comparable results across multiple datasets, along with outperforming human labellers. We present our results on the application of both detecting the presence of mosquitoes and the classification of bird species.

2019

1. TSP

   Invertible particle-flow-based sequential MCMC with extension to Gaussian mixture noise models

   Yunpeng Li, Soumyasundar Pal, and Mark J. Coates

   IEEE Transactions on Signal Processing (TSP), 2019

   Abs HTML PDF

   Sequential state estimation in non-linear and non-Gaussian state spaces has a wide range of applications in statistics and signal processing. One of the most effective non-linear filtering approaches, particle filtering, suffers from weight degeneracy in high-dimensional filtering scenarios. Several avenues have been pursued to address high dimensionality. Among these, particle flow filters construct effective proposal distributions by using invertible flow to migrate particles continuously from the prior distribution to the posterior, and sequential Markov chain Monte Carlo (SMCMC) methods use a Metropolis-Hastings (MH) accept-reject approach to improve filtering performance. In this paper, we propose to combine the strengths of invertible particle flow and SMCMC by constructing a composite MH kernel within the SMCMC framework using invertible particle flow. In addition, we propose a Gaussian-mixture-model-based particle flow algorithm to construct effective MH kernels for multi-modal distributions. Simulation results show that for high-dimensional state estimation example problems, the proposed kernels significantly increase the acceptance rate with minimal additional computational overhead and improve estimation accuracy compared with state-of-the-art filtering algorithms.

2018

1. ML4D

   BCCNet: Bayesian classifier combination neural network

   Olga Isupova, Yunpeng Li, Danil Kuzin, and 3 more authors

   In NeurIPS Workshop on Machine Learning for the Developing World (ML4D), 2018

   Abs PDF Code

   Machine learning research for developing countries can demonstrate clear sustainable impact by delivering actionable and timely information to in-country government organisations (GOs) and NGOs in response to their critical information requirements. We co-create products with UK and in-country commercial, GO and NGO partners to ensure the machine learning algorithms address appropriate user needs whether for tactical decision making or evidence-based policy decisions. In one particular case, we developed and deployed a novel algorithm, BCCNet, to quickly process large quantities of unstructured data to prevent and respond to natural disasters. Crowdsourcing provides an efficient mechanism to generate labels from unstructured data to prime machine learning algorithms for large scale data analysis. However, these labels are often imperfect with qualities varying among different citizen scientists, which prohibits their direct use with many state-of-the-art machine learning techniques. We describe BCCNet, a framework that simultaneously aggregates biased and contradictory labels from the crowd and trains an automatic classifier to process new data. Our case studies, mosquito sound detection for malaria prevention and damage detection for disaster response, show the efficacy of our method in the challenging context of developing world applications.

2. DCASE

   Fast mosquito acoustic detection with field cup recordings: an initial investigation.

   Yunpeng Li, Ivan Kiskin, Marianne Sinka, and 7 more authors

   In Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE), 2018

   Abs PDF

   In terms of vectoring disease, mosquitoes are the world’s deadliest. A fast and efficient mosquito survey tool is crucial for vectored disease intervention programmes to reduce mosquito-induced deaths. Standard mosquito sampling techniques, such as human landing catches, are time consuming, expensive and can put the collectors at risk of diseases. Mosquito acoustic detection aims to provide a cost-effective automated detection tool, based on mosquitoes’ characteristic flight tones. We propose a simple, yet highly effective, classification pipeline based on the mel-frequency spectrum allied with convolutional neural networks. This detection pipeline is computationally efficient in not only detecting mosquitoes, but also in classifying species. Many previous assessments of mosquito acoustic detection techniques have relied only upon lab recordings of mosquito colonies. We illustrate in this paper our proposed algorithm’s performance over an extensive dataset, consisting of cup recordings of more than 1000 mosquito individuals from 6 species captured in field studies in Thailand.

2017

1. BSPC

   Microwave breast cancer detection via cost-sensitive ensemble classifiers: Phantom and patient investigation

   Yunpeng Li, Emily Porter, Adam Santorelli, and 2 more authors

   Biomedical Signal Processing and Control (BSPC), 2017

   Abs HTML PDF

   Microwave breast screening has been proposed as a complementary modality to the current standard of X-ray mammography. In this work, we design three ensemble classification structures that fuse information from multiple sensors to detect abnormalities in the breast. A principled Neyman–Pearson approach is developed to allow control of the trade-off between false positive rate and the false negative rate. We evaluate performance using data derived from measurements of heterogeneous breast phantoms. We also use data collected in a clinical trial that monitored 12 healthy patients monthly over an eight-month period. In order to assess the efficacy of the proposed algorithms we model scans of breasts with malignant lesions by artificially adding simulated tumour responses to existing scans of healthy volunteers. Tumour responses are constructed based on measured properties of breast tissues and real breast measurements, thus the simulation model takes into account the heterogeneity of the breast tissue. The algorithms we present take advantage of breast scans from other patients or tissue-mimicking breast phantoms to learn about breast content and what constitutes a “tumour-free” and “tumour-bearing” set of measurements. We demonstrate that the ensemble selection-based algorithm, which constructs an ensemble of the most informative classifiers, significantly outperforms other detection techniques for the clinical trial data set.

2. TSP

   Particle filtering with invertible particle flow

   Yunpeng Li, and Mark Coates

   IEEE Transactions on Signal Processing (TSP), 2017

   Abs HTML PDF

   A key challenge when designing particle filters in high-dimensional state spaces is the construction of a proposal distribution that is close to the posterior distribution. Recent advances in particle flow filters provide a promising avenue to avoid weight degeneracy; particles drawn from the prior distribution are migrated in the state space to the posterior distribution by solving partial differential equations. Numerous particle flow filters have been proposed based on different assumptions concerning the flow dynamics. Approximations are needed in the implementation of all of these filters; as a result, the particles do not exactly match a sample drawn from the desired posterior distribution. Past efforts to correct the discrepancies involve expensive calculations of importance weights. In this paper, we present new filters which incorporate deterministic particle flows into an encompassing particle filter framework. The valuable theoretical guarantees concerning particle filter performance still apply, but we can exploit the attractive performance of the particle flow methods. The filters we describe involve a computationally efficient weight update step, arising because the embedded particle flows we design possess an invertible mapping property. We evaluate the proposed particle flow particle filters’ performance through numerical simulations of a challenging multitarget multisensor tracking scenario and complex high-dimensional filtering examples.

3. ML4Audio

   Cost-sensitive detection with variational autoencoders for environmental acoustic sensing

   Yunpeng Li, Ivan Kiskin, Davide Zilli, and 4 more authors

   In NeurIPS Workshop on Machine Learning for Audio Signal Processing (ML4Audio), 2017

   Abs PDF

   Environmental acoustic sensing involves the retrieval and processing of audio signals to better understand our surroundings. While large-scale acoustic data make manual analysis infeasible, they provide a suitable playground for machine learning approaches. Most existing machine learning techniques developed for environmental acoustic sensing do not provide flexible control of the trade-off between the false positive rate and the false negative rate. This paper presents a cost-sensitive classification paradigm, in which the hyper-parameters of classifiers and the structure of variational autoencoders are selected in a principled Neyman-Pearson framework. We examine the performance of the proposed approach using a dataset from the HumBug project which aims to detect the presence of mosquitoes using sound collected by simple embedded devices.

4. ML4D

   Mosquito detection with low-cost smartphones: data acquisition for malaria research

   Yunpeng Li, Davide Zilli, Henry Chan, and 4 more authors

   In NeurIPS Workshop on Machine Learning for the Developing World (ML4D), 2017

   Abs PDF

   Mosquitoes are the only vector for malaria, causing hundreds of thousands of deaths in the developing world each year. Not only is the prevention of mosquito bites of paramount importance to the reduction of malaria transmission cases, but understanding in more forensic detail the interplay between malaria, mosquito vectors, vegetation, standing water and human populations is crucial to the deployment of more effective interventions. Typically the presence and detection of mosquitoes is quantified through insect traps and human operations. If we are to gather timely, large-scale data to improve this situation, we need to automate the process of mosquito detection and classification as much as possible. In this paper, we present a prototype mobile sensing system that acts as both a portable early warning device and an automatic acoustic data acquisition pipeline to help fuel scientific inquiry and policy. The machine learning algorithm that powers the mobile system achieves excellent off-line multi-species detection performance while remaining computationally efficient. Further, we have conducted preliminary live mosquito detection tests using low-cost mobile phones and achieved promising results. The deployment of this system for field usage in Southeast Asia and Africa is planned in the near future. In order to accelerate processing of field recordings and labelling of collected data, we employ a citizen science platform in conjunction with automated methods, the former implemented using the Zooniverse platform, allowing crowdsourcing on a grand scale.

5. SPIE

   Particle flow superpositional GLMB filter

   Augustin-Alexandru Saucan, Yunpeng Li, and Mark Coates

   In SPIE Conference on Signal Processing, Sensor/Information Fusion, and Target Recognition, 2017

   Abs HTML PDF

   In this paper we propose a Superpositional Marginalized δ-GLMB (SMδ-GLMB) filter for multi-target tracking and we provide bootstrap and particle flow particle filter implementations. Particle filter implementations of the marginalized δ-GLMB filter are computationally demanding. As a first contribution we show that for the specific case of superpositional observation models, a reduced complexity update step can be achieved by employing a superpositional change of variables. The resulting SMδ-GLMB filter can be readily implemented using the unscented Kalman filter or particle filtering methods. As a second contribution, we employ particle flow to produce a measurement-driven importance distribution that serves as a proposal in the SMδ-GLMB particle filter. In high-dimensional state systems or for highly- informative observations the generic particle filter often suffers from weight degeneracy or otherwise requires a prohibitively large number of particles. Particle flow avoids particle weight degeneracy by guiding particles to regions where the posterior is significant. Numerical simulations showcase the reduced complexity and improved performance of the bootstrap SMδ-GLMB filter with respect to the bootstrap Mδ-GLMB filter. The particle flow SMδ-GLMB filter further improves the accuracy of track estimates for highly informative measurements.

6. ICASSP

   Particle flow SMC delta-GLMB filter

   Augustin-Alexandru Saucan, Yunpeng Li, and Mark Coates

   In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017

   Abs HTML PDF

   In this paper we derive a particle flow particle filter implementation of the δ-Generalized Labeled Multi-Bernoulli (δ-GLMB) filter. The bootstrap particle filter δ-GLMB suffers from weight degeneracy for high-dimensional state systems or low measurement noise. In order to avoid weight degeneracy, we employ particle flow to produce a measurement-driven importance distribution that serves as a proposal in the δ-GLMB particle filter. Flow-induced proposals are developed for both types of targets encountered in the δ-GLMB filter, i.e., persistent and birth targets. Numerical simulations reflect the improved performance of the proposed filter with respect to classical bootstrap implementations.

7. ICASSP

   Sequential MCMC with invertible particle flow

   Yunpeng Li, and Mark Coates

   In International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017

   Abs HTML PDF

   Particle filters are among the most effective filtering algorithms for nonlinear and non-Gaussian models. When the state dimension is high, they are known to suffer from weight degeneracy. Sequential Markov chain Monte Carlo (SMCMC) methods have been proposed as an alternative sequential inference technique that can perform better in high dimensional state spaces. In this paper, we propose to construct a composite Metropolis-Hastings (MH) kernel within the SMCMC framework using invertible particle flow. Simulation results show that the proposed kernel significantly increases the acceptance rate and improves estimation accuracy compared with state-of-the-art filtering algorithms, in high dimensional simulation examples.