I am a first-year Ph.D. student in Computer Engineering at Virgnia Tech. My research interests include deep learning, computer vision, and machine learning.

Prior to join Virginia Tech, I received my M.S. degree in Electrical and Computer Engineering from University of Michigan, Ann Arbor. I am fortunate to work with Prof. Matthew Johnson-Roberson, Prof. Jeffrey A. Fessler, and Prof. Yuni K Dewaraja on different projects. I obtained my B.S. degree in Telecommunication Engineering from Beijing Institute of Technology.

I have done research and projects in a wide variety of areas. During my time at Virginia Tech, I worked on projects related to the image to image translation and human object detection. During my master's degree at the University of Michigan, I worked with Prof. Jeffrey A. Fessler on an image denoising project. Recommended by Prof. Jeffrey A. Fessler, I then worked with Prof. Yuni K Dewaraja on medical image processing. After that, I have worked with Prof. Matthew Johnson-Roberson on object detection in the driving scene.

Research - Semantic Image to Image Translation

Semantic Image to Image Translation

Existing methods have shown great success in generating images with diverse attributes. But can we generate images with specific attributes?

We propose a novel Semantic Generative Adversarial Network to generate images with attributes specified explicitly. The inputs are an edge image along with sentences, where the sentences serve as the guidance for generator. The output is a synthetic image with specific attributes described by the sentences. Ou approach is to concatenate the desired attribute embedding with image features learned in the generator and apply a discriminator to distinguish whether the generated image is real or fake.

This method is tested on Pix2pix and achieved lower Mean Average Error than Pix2pix. We further demonstrated that this simple yet effective approach could manipulate the specific attributes of generated images. Our code is available at https://github.com/e-271/semantic_image_translation

Research - Human Object Interaction

Human-Object Interaction Detection (Ongoing Work)

This work focuses on tackling the challenging problem of human-object interaction (HOI) detection. This project is currently in review.

Research - Image Denoising

Multi-channel Weighted Schatten p-Norm Minimization for Color Image Denoising

This work addresses the classical yet fundamental problem, color image denoising, for image quality enhancement in computer vision. The Multi-channel Weighted Nuclear Norm Minimization (MC-WNNM) method has already achieved a good performance in color image denoising. However, the nuclear norm tends to over-shrink the singular values and it is just a special case of Schatten p-norm.

The proposed model, namely Multi-channel Weighted Schatten p-Norm Minimization (MC-WSNM), is a more flexible method to address the color image denoising problem. Instead of using the Nuclear norm, weighted Schatten p-norm regularization is applied to guarantee a more accurate recovery of the information and assign different weights to different singular values.

The picture shown above is a comparation between the denoising results of our MC-WSNM and MC-WNNM method. Our results show that the proposed method has achieved both a higher PSNR and 10+ times faster speed in denoising than MC-WNNM method

Research - Object Detection

Learning from real world images for object detection in autonomous vehicles

For autonomous vehicles, it is easy to acquire a large number of images with cameras, but labeling them can be expensive. While current object detector approaches have enabled great advances in detecting cars and pedestrians, they may not transfer well when applied in real-world scenes. In this work, we want to answer the question: can we leverage information within the dataset to adjust object detectors to real-world scenes?

This work proposes to generate annotations on unlabeled data using a model trained on large amounts of labeled data, and then retrain the model using the corrected annotations to improve the performance. We identify the prediction errors using the temporal and stereo information.

Research - Medical Image Processing

Y-90 PET radiomics modeling analysis for response prediction of liver cancer

The work focuses on predicting lesion response in radiobolization of liver cancer patients with lesion characteristics extracted from medical images. We applied LASSO regression to select features that can be applied to predict lesion response and evaluated the robustness of the selected features.


Esther Robb, Jiarui Xu, Wen-Sheng Chu, Abhishek Kumar, Jia-Bin Huang. Anonymous Submission. Neural Information Processing Systems (NeurIPS), 2020

Chen Gao, Jiarui Xu, Yuliang Zou, Jia-Bin Huang. Anonymous Submission. European Conference on Computer Vision (ECCV), 2020

Lise Wei, Jiarui Xu, Can Cui, Issam El Naqa, Yuni K. Dewaraja. “Y-90 PET radiomics modeling analysis for response prediction within hours of radioembolization in liver cancer patients”. Annual Congress of the European Association of Nuclear Medicine, 2018

Contact Me

Feel free to contact me at jiaruixu [at] vt [dot] edu