Subash Khanal

I am a PhD candidate in Computer Science at the Washington University in St. Louis , working in Multimodal Vision Research Laboratory led by Dr. Nathan Jacobs.

I have a MS in Electrical Engineering from the University of Kentucky in Lexington. During my masters, I worked with Dr. Michael T. Johnson focusing on speech recognition and signal processing.

Email  /  CV  /  Google Scholar  /  Linkedin  /  Github

Research

I’m interested in applying deep learning to address various Computer Vision (CV) challenges, with a focus on multimodal data. Currently, I’m developing methods using multimodal machine learning to integrate satellite imagery, audio, and text for creating global-scale soundscape maps.

Publications

We develop a probabilistic, multi-scale, and metadata-aware embedding space that connects audio, text, and overhead imagery. This enables the creation of dynamic, multi-scale soundscape maps for any geographic region, along with uncertainty estimates for the mapping.

We learn a tri-modal embedding space between audio, text and overhead imagery. This enables us to create soundscape maps over any geographic region, using either audio or textual queries.

We train a contrastive learning framework, Sat2Cap on a novel large scale dataset. This enables us to create maps using free-form textual descriptions.

This work presents GeoSynth, a diffusion-based model for synthesizing satellite images with global style and image-driven layout control.

This work presents a flexible framework, with vector embedding and metric learning variants, that supports both species distribution mapping with fine-grained visual classification.

This work presents a general framework that can be used to create an embedding space with any number of modalities by using satellite images as the binding element.

This work introduces the task of mixed-view panorama synthesis, where the goal is to synthesize a novel panorama given a small set of input panoramas and a satellite image of the area.

We introduced a novel approach for species distribution modeling that uses a large-language model to generate a representation of species. This provides flexibility to generate range maps at different levels of the taxonomic hierarchy and for unseen species.

Add an extra special token (explainable token) into Vision Transformer (ViT), and train it to select the most important patches in the input image.

Training ViT on 3D-to-2D converted multi-modal PET images achieves better Alzheimer's disease prediction.

Machine learning on different types of genetic data helps to identify candidate genes for Alzheimer's disease progression.

Learning a hierarchical, probabilistic embedding space allows one to achieve uncertainty estimate of feature distributions coming from sources with variable bands of information.

This paper compares the difference in articulatory patterns between native (L1) and non-native (L2) Mandarin speakers of English, for the purpose of providing an understanding of mispronunciation behaviors of L2 learners.

Articulatory features improve the performance of Automatic Speech Recognition (ASR) based Mispronunciation Detection and Diagnosis (MDD) systems.

The focus of this work was to analyse articulatory patterns of mispronunciation and design of ASR based MDD system.

This website is modified from source code of John Barron's website.