top of page
This site was designed with the
.com
website builder. Create your website today.Start Now

Thrust 1: American Sign Language (ASL) Recognition using mm-Wave RADAR

This project aims at designing RF sensing systems, and related algorithms, for sign language recognition to improve the interaction of smart environments with individuals who are Deaf or who use sign language as their preferred method for communication.

Human Subject Study:

  • Deaf/ Hard of Hearing people

  • ASL learners

​

Linguistic Analysis:

  • -Effect of Kinematics and fluency of signing

  • Fractal complexity to measure information flow

  • one-handed vs Two handed sign identification

  • Signing Speed comparison between fluent vs imitation singers

  • Symmetric vs asymmetric sign Identifications

​

Tools: MATLAB, Python, ELAN

​

Micro-Doppler Generation using Time-Frequency Analysis:

  • Range profile from raw RF data

  • Clutter and Noise removal using MTI filter

  • Short-time-Fourier -Transform

  • Spectrograms

​

  • RF Sensors Exploited

  • Texas Instruments 77GHz Automotive Radar: 

  • AWR 1642, 2243 Boost, 2243 Cascade imaging Radar

  • Anchotek 24 GHz FMCW, Xethru 10GHz UWB CW

  • Machine Learning Algorithms

  • Feature space plots: PCA, t-SNE

  • Classifier: SVM, Random Forest, K-NN, Decision Tree

  • Deep Learning

  • Classifier: CNN VGG Net, ResNET, AlexNet, Convolutional Auto Encoder

  • Synthetic Data Generation using GANs:

  • DCGAN, ACGAN, WGAN, Physics-inspired Multi-Branch GAN (MBGAN)

  • Imitation to fluent sign transformation: Pix2Pix, CycleGAN, Travel GAN

  • Frameworks used: Keras, Tensor flow

Related Publications

• MM Rahman, E Malaia, AC Gurbuz, DJ Griffin, SZ Gurbuz ,’Effect of Kinematics and Fluency in Adversarial Synthetic Data Generation for ASL Recognition’. Transaction of Aerospace and Electronic system (TAES) 

 

• SZ Gurbuz, MM Rahman, E Kurtoglu,et.al., ‘Multi-Frequency RF Sensor Fusion for Word-Level Fluent ASL Recognition’. IEEE Sensors Journal 2021

 

• SZ Gurbuz, AC Gurbuz, EA Malaia, DJ Griffin, CS Crawford, MM Rahman, et.al., American sign language recognition using RF sensing. IEEE Sensors Journal 21 (3), 3763-3775

​

 • M. Mahbub Rahman, Sevgi Gurbuz, Evie Malaia, ‘Dynamic parameters of signing differences between signers and novice learners.’, Theoretical Issues in Sign Language Research conference (TISLR). (Submitted)

 

• MM Rahman, E Kurtoglu, R Mdrafi, AC Gurbuz et.al., Word-Level ASL Recognition and Trigger Sign Detection with RF Sensors. ICASSP 2021-2021

• MM Rahman, R Mdrafi, AC Gurbuz, et.al., ‘Word-level sign language recognition using linguistic adaptation of 77 GHz FMCW radar data.’ 2021 IEEE Radar Conference (RadarConf21)

 

• MM Rahman, SZ Gurbuz, ‘Multi-frequency RF sensor data adaptation for motion recognition with multi-modal deep learning.’ 2021 IEEE RadarConf20201

 

• SZ Gurbuz, AC Gurbuz, EA Malaia, et.al., ASL Recognition Based on Kinematics Derived from a Multi-Frequency RF Sensor Network. 2020 IEEE Sensors, 1-4

 

• SZ Gurbuz, MM Rahman, E Kurtoglu,et.al., ‘Cross-frequency training with adversarial learning for radar micro-Doppler signature ’. Radar Sensor Technology XXIV 11408, 114080A

 

• SZ Gurbuz, AC Gurbuz, EA Malaia, DJ Griffin, C Crawford, MM Rahman, et.al., ‘A linguistic perspective on radar micro-doppler analysis of American sign language’. 2020 IEEE International Radar Conference

Thrust 2: Physics-Aware  Machine Learning

Radar based indoor monitoring, Fall risk assessments, telehealth, Human Activity recognitions Always face the challenges of low support samples while applying Deep learning based classifiers. A common approach to address this limitation is to use Generative adversarial Networks (GANs) to synthesize RF signatures from few real samples collected from human participants. But GANs generates signatures which are kinematic-ally inaccurate. We try to address this issue with the fundamental research in neural network architectures and training methodologies that incorporate physical sensor and target models into GAN architecture so that it generates signatures within accurate kinematic bounds. This in turns helps to boost recognition performance by improving classification accuracy.

Physics aware GAN

  •      Modification of Discriminators: 

  •      Modification of Loss Functions:

  •     Perceptual loss: 

Related Publications

  • MM Rahman, SZ Gurbuz, MG Amin, "Physics aware Generative Adversarial Network for Radar-based Human Activity Recognition." IEEE Transaction of Aerospace and Electronics Systems (TAES). 

​

  • MM Rahman, SZ Gurbuz, MG Amin, ‘Physics-Aware Design of Multi-Branch GAN for Human RF Micro-Doppler Signature Synthesis.’ 2021 IEEE Radar Conference (RadarConf21).

Thrust 3: RF Skeleton for Gait Parameter Estimation

Main Goal of the Research:

The primary objective of this research is to explore the capabilities of radio frequency (RF) sensors in assessing human gait parameters for fall risk assessment and in recognizing human daily activities for remote health monitoring using radar technology.

Key Contributions:

  1. Activity Recognition Using Radar:​

    • Developed supervised and  self-supervised DNN frameworks that achieved a 93% accuracy rate in recognizing 15 ambulatory human activities.

  2. Gait Parameter Estimation for Fall Risk Assessment:

    • Conducted experiments with 15 healthy participants in a Biomechanics lab, measuring gait variability using RF under visual perturbations from a virtual reality headset. Vicon motion capture was utilized as a validation tool.

    • Proposed a methodology to extract crucial gait parameters like step time (ST), step velocity (SV), step length (SL), and their variabilities from the torso velocity recorded by both RF sensors and motion capture systems. 

  3. Significant Outcomes:

    • The research conclusively demonstrated that RF sensors offer a reliable means to estimate gait variability. Furthermore, they are effective in detecting changes in gait patterns, which can serve as indicators for potential fall risks.

​

Thrust 4: Radar Remote Sensing

From 2018 Fall to 2019 summer, I worked as Research assistant at UA Remote sensing centre. My focus was on Signal and data processing for Snow, ice and soil moisture measurements. My contribution during that time was as follows:

​

  • Developed signal processing algorithm for calculation of radar backscatter coefficient for soil moisture estimation using FMCW radar.

  • Developed algorithms for identification of snow and ice layers from airborne FMCW radar data, e.g. pulse compression, stretch processing, 2D SAR fk-migration algorithm with narrow-beam motion compensation, delay-and-sum beamforming, and layer tracking.

  • Developed a Synthetic Aperture Radar (SAR) simulation environment for validation of 3D tomographic SAR imaging algorithms.

​

Soil Moisture Estimation: 

Related Publications

  • R Aksu, MM Rahman, SZ Gurbuz , ‘3D scene reconstruction from multi-sensor EO-SAR data’, Algorithms for Synthetic Aperture Radar Imagery XXVII 11393, 113930B

  • ​

  • D Taylor, S Yan, C O'Neill, P Gogineni, S Gurbuz, et.al., ‘Airborne Dual-Band Microwave Radar System for Snow Thickness Measurement’, IGARSS 2020-2020

bottom of page