compressive holographic video

Compressive holographic video (Published in Optics Express)

The motivation of this project is to overcome the spatiotemporal tradeoffs of modern cameras. Data is redundant. This is a ubiquitous phenomenon people have witnessed. On the other hand, sampling a full resolution image (with redundancy) takes longer time than sampling fewer pixels for a camera. Thus, a natural question is if we can save spatial resolution for higher imaging speed.

My most recent project works on 4D imaging. That’s to compress both spatial and temporal information into a single image.

Compressive holography (3D slicing from a 2D holographic image)

Subsampled compressive holography

Recovering 4D information, e.g. dropping glitters, from a single image.

Glint impression and diffuse coarseness

Looking into special surface effects: diffuse coarseness and glint impression

(Published in Coloration Technology, Volume 132, Issue 2, Pages 153–161, April 2016 , DOI: 10.1111/cote.12203)

glint sample

How do human understand and distinguish sparking (glint) and graininess (coarseness).

We conducted a psychophysical experiment to model these two effects.

glint geometry

And if you compare the visual result with instrumental measurement, you will find one dimension is not enough to represent human response.

glint dimensions

We used multidimensional scaling to find the feasible dimensionality (2 dimensions are better and not redundant).

glint mds

and then, if we represent the data into 2 dimensions, what is actually the principle axis, what is the adjunct axis. And what are the meanings of them?

Interested in this types of research topic? Please refer to my paper here.

Image stitching with tone mapping

This is a course project from EECS 332 Intro to Computer Vision, Northwestern University. (2016 winter)

Conventional image stitching method comes as the following four steps:

  1. Scale-Invariant Feature Transform (SIFT)
  2. Feature matching
  3. Random sample consensus (RANSAC)
  4. Center-weighted blending.

image-stitch_ori

image-stitch_ori1

But this is only local blending. It is quite obvious if two images have different white balance. The images above are two pictures I took within a minute and the white balance is changing just as I moved my phone. This color inconsistency happens a lot for outdoor scenes, the camera may arise larger noise and different white balance. We came up with the idea to use the overlapped region for color blending globally.

  • Method: K mean clustering
  • Reason: pixel locations cannot match precisely. Pixel-wise adjustment is not a good choice. Alternatively, we find mutual color regions to match pixel values.

image-stitch_3

Overlap region

image-stitch_kmean

k-mean clustering (k = 2)

The k mean clustering is a broadly used method for image segmentation. It can help find the principle clusters.

image-stitch-4

Stitching results: up–local stitching; middle–fusing two images; lower–from right to left

The previous case solves the problem of dim environment. What about sunny days?

image-stitch5

When the sun is in the scene, our camera will adjust the exposure time. This will lead to an inconsistency from irradiance to pixel values.

picture1

picture1

image stitching results.

We take the idea by high dynamic range (HDR) tone mapping.

picture1

 

picture1

Final tone-mapped result.

Correlated color temperature simulations

This is a previous project (undergraduate, 2012 fall) on the simulation of real-life environment.

The simulation is a 1:1 simulation of a living room setup.

cct-scene

The goal is to simulate the effect of changing correlated color temperature (CCT). The results are shown below,

3000 K

living room 3k

4000 K

living room 4k

5000 K

cct-5k

6500 K

cct-6k5