Image Processing GEometric Operations

Patrick Zimmer

| | | |

Image Processing Geometric Operations

Projects / Academic / Image Processing Geometric Operations

Task Details

1 Image Acquisition

As in the first practical, acquire and store suitable grey-level images of a hand (for example, your own) and a face (more amusing if it's someone else's). By "suitable", I mean under appropriate lighting and imaging conditions such that the hand and face fill a reasonably large proportion of the field of view, that detail within each image can be seen, such as facial features, and that the images are not too dark (ie, above the noise level) without being too bright (ie, reaching the maximum signal level of the camera). In view of the exercises to be carried out, you might grab several examples of each image so that you can choose the one best suited to each exercise. You may use the images acquired for the first practical if you think they are suitable.

If your images don't work well, try the jpeg example image of me (bernard0.jpg) and the image of Charlie (charliegrey.tif) in my public_html directory. The former is a colour image and will need converting to a grey-level image. Alternatively, try changing each colour channel separately or just have fun using the colour editor in xv.

2 Geometric Transformations

Select an area of detail on one of your hand and one of your face images and blow it up by a factor of two in each linear dimension. Do this using nearest-neighbour, bilinear and bicubic interpolation, using suitable functions from IDL (eg "congrid"). Describe what differences you would expect to see in the different methods of interpolation. Are such differences visible in your example? Comment on their presence or absence. What happens if you blow up the image by a further factor of two or more in each linear dimension?

3 Image Warping

Subject one of your face images to a 45 degree shear on the x-axis by defining a suitable linear (affine) transformation of the image co-ordinates. Comment on whether you think it is necessary to use interpolation in order to compute the destination image in this case. Discuss what would have to be done for a 30 degree shear. Investigate whether there are any IDL functions that you can use to help carry out such image shears.

4 Image Rotation by Shearing

Use a combination of several 45 degree shears to carry out a 90 degree rotation. Think carefully about how many shears are needed. Use xv or functions provided in IDL to check your results. Comment on whether you think this is a good way of carrying out a 90 degree rotation? Comment also on whether you think it is a good way of carrying out image rotations in general.

5 Manual Warping

Define a set of control points on one of your face images and use them to warp it in whatever way you please. There are IDL functions which can be used for this, but check how they work. In particular, do they only work when you have enough control points to make the transformation determined or overdetermined? Will they work or not if too few control points are used? Describe, as far as you can, what is happening during the transformation.

Project Work

Contents Page

1. Introduction

1 2. Methods 2

2.1 Geometric Transformations 2

2.2 Image Warping by Shearing 2

2.3 Image Rotation by Shearing 3

2.4 Manual Warping 4

3. Results 5

3.1 Geometric Transformations 5

3.2 Image Warping by Shearing 8

3.3 Image Rotation by Shearing 9

3.4 Manual Warping 10

4. Discussion and Conclusions 12

4.1 Geometric transformations 12

4.2 Image Warping by Shearing 12

4.3 Image Rotation by Shearing 13

4.4 Manual Warping 13

5. Appendix 15

5.1 Geometric transformations 15

5.2 Image Warping by Shearing 16

5.3 Image Rotation by Shearing 17

5.4 Manual Warping 17

1. Introduction

The aim of this practical is to investigate a number of geometric and spatial transform methods. Some geometric transforms are written in code while spatial transforms are all performed by IDL functions. Initial transforms involve resizing and shearing an image to understand the concepts involved in pixel location mapping.. Finally more complicated warping operations are performed using IDL functions. The results are used to try and understand the combination of methods used by the functions.

Images

The images of a hand and a face used for the investigated transforms, are shown below (Fig. 1.1a and 1.1b)

Fig. 1.1a Face (371 x 292) Fig. 1.1b Hand (410 x 576) 2. Methods 2.1 Geometric Transformations

Areas of detail on the face and hand were selected and the elements copied to a new array using code written in the previous practical.

The selected area was subjected to the IDL resizing operations, REBIN and CONGRID to increase the image size by a factor of 2. REBIN uses bilinear interpolation as standard but the SAMPLE keyword can be set to achieve nearest-neighbour interpolation. Using the CONGRID function, the CUBIC keyword can be set to produce cubic convolution interpolation with an interpolation parameter between 0 and -1. Hence results of resizing, using nearest-neighbour, bilinear and cubic interpolation were produced. When using the CONGRID function with the CUBIC keyword, the parameter, p was set to -1 (default), -0.5 (recommended) and -0.1.

The operations were performed on the centre on the face and an area of interest, the eye, as well as on a region of the hand image. The operations on the eye were repeated specifying new image dimensions, four times larger than the original size. To highlight the most interesting features, the bilinear, nearest neighbour and cubic transformations with p=-0.1 were repeated, using a scaling factor of 8.

2.2 Image Warping by Shearing

A shear on the x-axis is produced using the mapping:

x = x' + a'₂ y'

y = y'

separated as row and column operations.

Since:

a'₂ = (x - x')/ y'

a'₂ is the ratio of change in x to change in y, so a'₂ = tan f, where f is the angle of shear.

If f = 0 ie. no shear, a'₂ = 1 and so x = x'.

If f = 90 ^o ie. 90 ^o rotation, x = y' and so y = x', however.

In the case of a 45 degree shear a'₂ = 1, so

x = x' + y'

y = y'

Two for loops traverse the image array and the original elements are copied to the calculated locations in a new array. The new array is larger since the width of the image is increased by its height. ( x_max = x'_max + y'_max ).

In the case of a 45 degree shear no interpolation is needed since pixels are square. The transformed coordinate, x will always be an integer since a'₂ = 1.

2.3 Image Rotation by Shearing

A 90 degree rotation of an image can be performed using a series of 45 degree shear operations as shown in Fig. 2.1.

The dimensions of the face image were changed to 100 x 150, using an external editor. The original image is almost square and so is a bad example to view the exchange of width and height.

The initial shear (Fig. 2.1 - 2) is on the x-axis and uses the standard coordinate mapping:

x = x' + y'

y = y'

The array required for the sheared image must have dimensions to contain the new width, consisting of the sum of original height and width (H_new = H, W_new = W + H).

The second shear (Fig. 2.1 - 3) is on the y-axis and uses the mapping:

x = x'

y = y' - x'

to continue the appearance of clockwise rotation. In the code the mapping:

x = x'

y = y' - x' + W

is used, where W is the width of the original image. This is to keep the new image coordinates positive and align them with the bottom of the new array. The new array requires dimensions to contain a height equal to the original width, and a width equal to the sum of original height and width. (H_new = W, W_new = W + H).

The final shear (Fig. 2.1 - 4) is on the x-axis and uses the original mapping:

x = x' + y'

y = y'

In the code, the mapping:

x = x' + y' - W

y = y'

is used. This is to align the image in the left of the new array. The new array dimensions need to hold a width equal to the original height and a height equal to the original width. (H_new = W, W_new = H).

Fig. 2.1 Using a combination of 45 degree shear operations to produce a rotation

2.4 Manual Warping

The WARP_TRI function returns a warped image based on user defined input and output control points. The input control points are warped by shifting them to the output points. The control points must define the output area so the minimum number is three. It was decided to define all four corners of the image as control points, mapped without change, to display it fully. Control points within the image can then be chosen to test the effects of warping. Warped images were saved using 1 and 2 additional points and shifting them vertically, horizontally and diagonally. The effect of using the QUINTIC keyword was also tested by repeating some of the initial warps with the keyword set.

Code was also written to produce crosses centered on the input and output control points. Knowing where the control points are makes it easier to relate them to the changes produced. Input control points are larger than output and the first set are black while the second are white.

3. Results

3.1 Geometric Transformations

Fig. 3.1.1 a-f Face enlargement by a factor of 2 using IDL functions REBIN and CONGRID with the following methods of interpolation.

Left to Right, First Row then Second:

a. Original Image, b. Bilinear Interpolation, c. Nearest-Neighbor Interpolation,

d. Cubic Interpolation (interpolation parameter, p = -1)

e. Cubic Interpolation (p = -0.5), f. Cubic Interpolation (,p = -0.1)

Fig. 3.1.2 a-f Eye enlargement by a factor of 2 using IDL functions REBIN and CONGRID with the following methods of interpolation.

Left to Right, First Row then Second:

a. Original Image, b. Bilinear Interpolation, c. Nearest-Neighbor Interpolation,

d. Cubic Interpolation (interpolation parameter, p = -1)

e. Cubic Interpolation (p = -0.5), f. Cubic Interpolation (,p = -0.1)

Fig. 3.1.3 a-f Fingertip enlargement by a factor of 2 using IDL functions REBIN and CONGRID with the following methods of interpolation.

Left to Right, First Row then Second:

a. Original Image, b. Bilinear Interpolation, c. Nearest-Neighbor Interpolation,

d. Cubic Interpolation (interpolation parameter, p = -1)

e. Cubic Interpolation (p = -0.5), f. Cubic Interpolation (,p = -0.1)

Fig. 3.1.4 a-f Eye enlargement by a factor of 4 using IDL functions REBIN and CONGRID with the following methods of interpolation.

Left to Right, First Row then Second:

a. Original Image, b. Bilinear Interpolation, c. Nearest-Neighbor Interpolation,

d. Cubic Interpolation (interpolation parameter, p = -1)

e. Cubic Interpolation (p = -0.5), f. Cubic Interpolation (,p = -0.1)

Fig. 3.1.5 a-d Eye enlargement by a factor of 8 using IDL functions REBIN and CONGRID with the following methods of interpolation.

Left to Right, First Row then Second:

a. Original Image, b. Bilinear Interpolation, c. Nearest Neighbor Interpolation,

d. Cubic Interpolation (interpolation parameter, p = -0.1)

3.2 Image Warping by Shearing

Fig. 3.2 Result of a 45 degree shear on the x-axis

3.3 Image Rotation by Shearing

Fig. 3.3 a-d Using a series of shears to produce a rotation

Left to Right, First Row then Second:

a. Result of First Shear, b. Original Image, c. Result of Second Shear,

d. Result of Final Shear

3.4 Manual Warping

Black and white distinguishes the two different control points. Input control points are larger than output. The input and output coordinates of the control points used, are shown above each image.

Fig. 3.4.1 a - d Warped images changing one control point

Fig. 3.4.1a (200,200) -> (150,200) Fig. 3.4.1b (200,200) -> (200,150)

Fig. 3.4.1c (200,200) -> (150,150) Fig. 3.4.1d (200,200) -> (100,100)

Fig. 3.4.2 a - d Warped images changing two control points

Fig. 3.4.2a (200,200) -> (150,200) Fig. 3.4.2b (200,200) -> (150,200)

(200,150) -> (200,200) (200,150) -> (250,150)

Fig. 3.4.2c (200,200) -> (150,200) Fig. 3.4.2d (200,200) -> (200,100)

(200,150) -> (150,150) (200,150) -> (200,220)

Fig. 3.1.4 a - b Warped images using quintic interpolation

Fig. 3.4.3a (200,200) -> (150,200) Fig. 3.4.3b (200,200) -> (150,150)

(200,150) -> (250,200) (200,150) -> (250,200)

4. Discussion and Conclusions

4.1 Geometric transformations

Using a large area of the picture makes it difficult to observe the difference between the scaling using bilinear (Fig. 3.1.1b) and nearest-neighbor interpolation (Fig.3.1.1c). Fig. 3.1.2b and c show that the pixels size when using nearest-neighbor interpolation appears larger than when using bilinear interpolation. This difference is more noticeable at high contrast object borders eg. the edge of the iris. This explains why it is hard to distinguish the pixel size between the hand areas in Fig. 3.1.3b and c, since the fingertip is far less defined than the eye. Fig. 3.1.4c makes the effect more visible and Fig. 3.1.5c clearly shows that the size of the pixels is increasing with the scaling factor, when using nearest neighbor interpolation. This effect was expected since square groups of output pixel points will all map to within the same input pixel (the nearest neighbor) so will acquire the same value.

When using bilinear interpolation each output pixel will use a unique set of four input points for mapping so the output image should have smoother changes in intensity. This appears, from comparison of Fig. 3.15b with the other methods (Fig. 3.15c and d), to be the case.

Due to the lack of familiarity with the method of cubic interpolation, it is difficult to predict the effect of using this method. It would be expected that the result would have a smaller pixel size than that when using nearest neighbor interpolation since cubic interpolation is more sophisticated.

Fig. 3.1.1d and e show some distortion when using cubic interpolation with p=-1 and p=-0.5 around the eyes and teeth. These areas can be seen to have the greatest contrast between black and white at an object boundary (eg a tooth).

Fig. 3.1.2 - d and e show more clearly the distortion, which decreases as the interpolation parameter is raised in value. Using p=-0.1 produces no distortion but careful inspection of Fig. 3.1.4f shows that the image seems more pixilated than when using p=-1 or -0.5, as shown in Fig. 3.1.4d and e. In this sense there seems to be a trade off between distortion and pixel size but without studying the method of cubic interpolation in more detail, there is not much more that can be concluded.

If a 30 degree shear were required, a'₂ = tan 30 = 3 ^-1/2 = 5.774, so x coordinate values would not be integers. The center of an output pixel would then not correspond to the center of any input pixel so its value would have to be interpolated from the pixel values around the input point. The IDL INTERPOLATE function would be required which returns an interpolated value given the data array and the required coordinates.

4.2 Image Warping by Shearing

The code produces a 45 ^o shear along the x-axis as required (Fig. 3.2). The shear could also have been performed using a matrix transformation but this would involve more code. Involved, would be making arrays of x and y coordinates on which to operate, calculating the new coordinates and then mapping the old pixel values to the new coordinates.

4.3 Image Rotation by Shearing

Fig. 3.3b shows the original image, which is sheared to produce Fig. 3.3a, Fig.3.3c and finally Fig. 3.3d. The final image was observed to be the same as a 90 ^o clockwise rotation produced using the IDL function ROTATE (it was decided not to print the identical image). This function performs clockwise matrix rotations in multiples of 90^o.

Using shear operations to produce a 90 degree rotation is very computationally inefficient since the same effect would be produced by interchanging the horizontal and vertical coordinates.

Three shear operations would be needed for any general rotation, involving three separate operations on each pixel. Using a matrix transformation would require only one operation on each pixel so is far more computationally efficient.

4.4 Manual Warping

IDL is provided with two main warping functions aimed at image-warping, WARP_TRI and POLY_2D. POLY_2D requires a set of polynomial coefficients, which are used to control the effects of the warp. These coefficients are usually produced with the POLYWARP function, which produces the polynomial by fitting a set of input coordinates as a function of a set of output coordinates. The WARP_TRI function differs by fitting straight lines between the control coordinates instead of a polynomial.

It was decided to use the WARP_TRI function since the results would be easier to understand and explain. Following is a simple explanation of how the function works:

The output control points define a grid, which is triangulated using the IDL TRIANGULATE function. The TRIGRID function is used to find the input points that correspond to output pixels. These points are interpolated using the INTERPOLATE function to find the value of the required output pixel.

To define an area, a minimum of three control points is required.

Fig. 4.4a and b show the triangular input and output surfaces. The greatest unwanted distortion is the abrupt change in image alignment at the edges of the triangles as seen in Fig. 4.4c. This effect is avoided by using the QUINTIC keyword, which specifies the use of smooth quintic interpolation (Fig. 4.4d). The triangular output surfaces are then not visible which produces a more effective result.

Fig. 3.4.1 a and b show that the distortion caused by triangulation is much less visible when the direction of the control point change is not perpendicular to distinct lines in the image. The most defined lines in the image are formed by the hair so the downward warp in Fig. 3.4.1b shows no sign of triangulation. Fig. 3.4.1c and d show that the effect is worse as the control point is shifted further from its original position.

More control points can be used to manipulate the image with more precision. Fig. 3.4.2a shows how defining unchanged control points can prevent certain areas of the image from being affected by other control point changes. The area around the second control point is unchanged when compared with Fig. 3.4.1a.

Both Fig. 3.4.2a and b show that using control points close together leads to image distortion unless both control points warp in the same direction, as in Fig. 3.4.2c. Using more control points (Fig. 3.4.2c) to perform the same warp could produce much better effects if they are chosen correctly, since the triangles formed would be smaller.

When control points warp in directions that cross or oppose each other some parts of the image are mirrored and displayed in a distorted manner as shown in Fig. 3.4.2d.

Using quintic interpolation prevents the triangular distortion and produces smooth changes in the warped image (Fig. 3.4.3a and b). The TRIGRID function now uses a non-linear (quintic) mapping method, but the details of this type of interpolation are not in the IDL help file.

In general warping is very versatile but requires thought to produce the required result. A shear on the x-axis for example could have been performed by defining the four corners of the image as input control points and shifting the upper corners along the x-axis as required.

Fig. 4.4 Warp using WARP_TRI function shifting one control point from (200,200) to (100,200)

Left to Right, First Row then Second:

Original Image triangulated from input control point
Warped Image triangulated from output control point
Warped image (Control points: white-input, black-output)
Warped image using quintic interpolation

5. Code Appendix

All code is written for and tested in IDL Version 5.2.1 Win32.

Common Methods

The following code to read a tiff image into an array and invert the array horizontally (to produce an upright displayed image) precedes all the following code sections and will hence be excluded. The QUERY command to obtain the image dimensions is used in most sections of code so is included here instead of constantly repeating it. The image file destination is not specified since it is irrelevant.

dest= ';image file destination'

imag = READ_TIFF(dest)

image=REVERSE(ROTATE(imag,2)) ;invert horizontally

info=QUERY_TIFF(dest,inf)

;Obtain image dimentions

Wi=inf.DIMENSIONS[0]

He=inf.DIMENSIONS[1]

Output images were saved using the following command, as required:

WRITE_BMP, 'C:/output.bmp', TVRD()

Which saves the current contents of the display window to a bitmap file.

5.1 Geometric transformations

The appropriate dimensions were specified and the IDL functions REBIN and CONGRID used to perform the various methods of array enlargement. The enlargement factor, n is set as required before compiling the program. Code used to copy the selected area to a new array is taken from the previous practical.

;select face

;hix=280

;hiy=220

;lox=150

;loy=90

;select eye

hix=280

hiy=220

lox=220

loy=190

;select finger

;hix=120

;hiy=120

;lox=110

;loy=110

Wi=hix-lox

He=hiy-loy

;Creating new integer array with selected area

area=INTARR(Wi,He)

;Copy selected area to new array

FOR A = lox, hix DO BEGIN

FOR B = loy, hiy DO area[[A-lox],[B-loy]]=image[[A],[B]]

ENDFOR

n=2 ;enlargement factor - set as required

bi=REBIN(area,n*Wi,n*He) ;bilinear interpolation

nn=REBIN(area,n*Wi,n*He, /SAMPLE) ;nearest neighbor interpolation

cu1=CONGRID(area,n*Wi,n*He, /CUBIC) ;cubic interpolation

cu2=CONGRID(area,n*Wi,n*He, CUBIC=-0.5) ;cubic interpolation

cu3=CONGRID(area,n*Wi,n*He, CUBIC=-0.1) ;cubic interpolation

;Display images side by side

tv, area, 0

tv, bi, 1

tv, nn, 2

tv, cu1, 3

tv, cu2, 4

tv, cu3, 5

END

5.2 Image Warping by Shearing

The arrays created for the sheared images are filled with value 255 to identify the array area in the display window, which is black as standard (and to save ink).

;Create array for sheared image

shear=MAKE_ARRAY(He+Wi,He, /INTEGER, VALUE = 255)

FOR I = 0, Wi-1 DO BEGIN

FOR J = 0, He-1 DO shear[[I+J],[J]]=image[[I],[J]]

ENDFOR

tv, shear

END

5.3 Image Rotation by Shearing

The three shear operations as described in section 2.3 are implemented in the following code. The results of the first and second shear are displayed in vertical alignment to show the exchange of width and height.

shear=MAKE_ARRAY(He+Wi,He, /INTEGER, VALUE = 255)

shear1=MAKE_ARRAY(He+Wi,Wi, /INTEGER, VALUE = 255)

shear2=MAKE_ARRAY(He,Wi, /INTEGER, VALUE = 255)

FOR I = 0, Wi DO BEGIN

FOR J = 0, He DO shear[[I+J],[J]]=image[[I],[J]]

ENDFOR

FOR I = 0, (Wi+He) DO BEGIN

FOR J = 0, He DO shear1[[I],[J+Wi-I]]=shear[[I],[J]]

ENDFOR

FOR I = 0, (Wi+He) DO BEGIN

FOR J = 0, Wi DO shear2[[I-Wi+J],[J]]=shear1[[I],[J]]

ENDFOR

;Display images with shear and shear1 vertically aligned

tv, shear, 0

tv, image, 3

tv, shear1, 4

tv, shear2, 8

5.4 Manual Warping

The IDL WARP_TRI function is used with arrays of input and output control points to define the warp. The last two control points in the arrays are used to produce the changes, and set as required.

Crosses are produced by making horizontal and vertical lines of an extreme intensity centred on the control points.

:Define all four corners of the image as unchanged control points

;Define two changeable control points which are set as required

X1 = [0,0, Wi,Wi,200,200]

Y1 = [0,He,0, He,200,150]

X0 = [0,0, Wi,Wi,150,250]

Y0 = [0,He,0, He,200,150]

;Warp the image

warp=WARP_TRI(x0,y0,x1,y1,image)

;The following line is used instead, to set the QUINTIC keyword

;warp=WARP_TRI(x0,y0,x1,y1,image, /QUINTIC)

;Create crosses centred on the control points