Computer Vision: Current Trends and Future Possibilities

Extracting some useful information from images is considered as Computer Vision. These images can be of any form from Visual to Infrared to X-rays in the whole electromagnetic spectrum. The basic idea is to duplicate human visual perception in images to extract the same information.

Some major applications of Computer vision consists tasks including Object detection, Object tracking, segmentation, Image inpainting and 3d modelling from images.

A lot of research work is carried out all over the globe in all of the above mentioned fields. So in this article we are going to discuss some of the very interesting yet strange fields for new comers.

Image Inpainting:

Image inpainting is a process to recover some useful information from deteriorated images or give some artistic look to images by removing unwanted objects still maintaining smooth background. Such a task is a daily part of human life as imagine a person at some distinct place or removing a particular object from a scene but in computers, this is trivial. 

Method:

First a binary map is created on the basis of which part of the image is to be removed. Now that part of the image is filled in a manner to minimize energy. This is often done using a very simple operator called Laplacian operator. This is essentially second order derivative of image. Second order derivatives are used because its direction is similar to the direction of edges rather than perpendicular to it as in the case of first order derivative. Thus we find to minimize this function as this would perfectly reflect the second order derivative.

Image Segmentation:

Segmentation is a process of image processing to segment out one or more objects from an image. The goal is to find out a boundary of pixels that can perfectly differentiate between two objects based on colour or shape or both. Applications for image segmentation includes Object detection, Face detection and in medical imaging. Tumour detection, Surgery planning and diagnosis of anatomical structures are some of the major applications of segmentation in medical imaging.

Motion Analysis:

Motion analysis is in the simplest case to find out a moving object from a sequence of images. This work can be extended to find the direction of movement, velocity and displacement calculation and object tracking. The basic idea is find out a static region (background) and a moving region with substantial displacement. One very popular method for this is finding the Optical flow. Motion analysis is extremely important in Surveillance and video object tracking. Tracking with a moving camera increases the complexity a lot due to the relative motion between camera and the object. Tracking with multiple cameras with overlapping or non-overlapping regions are current research issues in Object tracking.

Hardware Implementation of Image Processing Algorithms

Digital image processing is done to improve the quality of images. The images are processed and enhanced in order to obtain the required information from it. It is an ever expanding field with a number of applications in medicine, space, art, meteorology, etc. Hardware implementation of digital image processing techniques is very commonly done using Verilog-HDL or VHDL gives a logical explanation of any circuit which can be further developed and tested. The main advantage of using HDLs is that any logical input can be simulated on FPGA (Hardware implementation).Application specific hardware implementation offers much greater speed than a software implementation.

Advancements in VLSI technology have made this hardware implementation an attractive, feasible and less time consuming task to undergo.  FPGA is one of the best technologies available for hardware design  because their structure is able to exploit spatial and temporal parallelism .FPGAs are ideal in many embedded systems applications because of their small size, low power consumption ,a number of input-output  ports and logic blocks. Also FPGAs are reprogrammable chips, hence are very versatile to be used.

Algorithms can be implemented in Verilog HDL using Xilinx ISE, MATLAB and MODEL SIM. Hardware implementation of Verilog codes provides us with the ability to verify the logical codes simulated using software. Hardware implementation helps in the co-simulation of the already simulated logic. It helps the designer to reformulate the algorithm.

It provides us an overlook of the way the respective logical circuit will work when brought to real life applications. Also, an idea of the market need of the circuit and how it will actually work, is provided by hardware implementation. The use of reconfigurable hardware to implement algorithms for image processing minimizes the time-to-market cost, provides prototyping and debugging.

Therefore, the reconfigurable devices like FPGA seem to be the ideal choice for implementation of image processing algorithms. SiliconMentor is a team with expertise in the fields both software implementation and hardware prototyping. The team focuses to provide a shared platform for the researchers and innovators by providing guidance in the specified areas.

HOW TO SELECT ARCHITECTURE FOR IMAGE PROCESSING ON FPGA

Implementation of real time image processing on serial general purpose processors is hard to achieve. This is due to the limited resources, general purpose architecture and large data set presented by image. For example if we have to perform single operation on every pixel of the 640x480 gray scale frame from input video source at 30 frames per second, it will require the serial processor to perform 1.84 million operations per second excluding the operations required for reading from and writing data to buffers. This demands a very high throughput serial processor (GPP). 

The above stated problem of image processing at real time can be countered by using FPGAs by using their inherent parallelism of hardware which is an advantage over the fetch decode architecture of processor. Along with solutions comes the complexity of implementation. The software algorithms developed for general purpose processors cannot be directly implemented on FPGA and need to be converted to take advantage of parallelism and meet the constraints implied by hardware. There are popular ways to implement software image processing codes in high level languages (HLL) to FPGA.

• Using HLL to HDL compilers
• Manual conversion

The cases where the hardware implementation is functionally equivalent to software implementation, the mapping is easy and compilers can be used efficiently. Sometimes standard algorithm made for software is not compatible for hardware due to reasons like, the implementation requires too many resources or accesses memory in a way which the hardware cannot support. In such cases we need to re write the algorithm in hardware keeping in mind the limitations and constrains of hardware.

The major constraints implied by hardware implementation are:

•  Timing
• Bandwidth
• Resource utilization and conflict 

The affect of these constraints is closely dependent on the processing model adopted for implementation of the application. The general processing models used are: 

• Stream
• Offline
• Hybrid processing

In stream processing the design samples incoming data like raster scan and perform as much operations as possible. In this mode the role of memory is very important and the processing speed is dependent on bandwidth of memory.

Offline processing doesn’t imply much constraints and it is most suitable for direct mapping of software based algorithm. Hybrid processing is mixture of stream and offline processing and the timing constraints re relaxed as the image is sampled at slower rate.

HISTOGRAM EQUALIZATION AND ITS ROLE IN IMAGE PROCESSING

In the era of image processing, scientific analysis, digital photography, remote sensing and in visualization, medical image analysis, surveillance system; image enhancement plays a vital role. By enhancement of image noise can be reduced and it can remove artifacts. A special feature of image enhancement is that it can hold all the details of image after enhancement.

As we know contrast enhancement of an image is done by making light colors lighter and dark colors darker at the same time. And this process is done by setting all color components below a specified lower bound to zero, and all color components above a specified upper bound to the maximum intensity. Here various methods of histogram equalization is addressed. Histogram graphically shows the distribution of pixels among grey scale values. Dynamic range of an image can be improved by equalization method. Histogram equalization is an efficient and useful technique.

The intensities will be equally distributed in output image after the process of histogram equalization.

There are some reasons that led to the need of enhancement:

•          bad quality of the used imaging device,
•          lack of expertise of the operator
•         The adverse external conditions or environment condition at the time of capture.

       IMAGE ENHANCEMENT

Processing of images to extract some specific features of an image is called image enhancement. The main motive of image enhancement is to improve the original image for some specific applications. It sharpens or improves image features such as boundaries, or contrast to make a better graphic display, and better analysis

Image enhancement has two categories:

1) Spatial domain method

2) Frequency domain method

Spatial domain method is based on direct manipulation of pixels in an image. Frequency domain method is based on modifying FT of an image.

Enhancement of an image is done by sharpening, noise removal and brightness increment. Unfortunately there is no general theory for determining what ‘good’ image enhancement, when it comes to human perception. If it looks good .it is good!

Operation of image enhancement is shown by given block diagram

 

Operation of Image Enhancement

Histogram equalization is distribution of particular type of data. It plays a vital role in image processing. By histogram equalization we can improve contrast and appearance of an image. Entire spectrum of pixels (0-255) will be stretches by histogram equalization. A histogram that covers all possible values which is used by gray scale is determined as a good histogram. A good histogram tends to have good contrast and the details of an image that may be easily observed.

In particular, the method can lead to better views of bone structure in x-ray images, and to better detail in photographs that are over or under-exposed.

Here some advantage and disadvantage of this method

Advantage: A key advantage of the method is that it is a fairly straight forward technique and an invertible operator. So in theory, if the histogram equalization function is known, then the original histogram can be recovered.

Disadvantage: A disadvantage of the method is that it is indiscriminate. It may increase the contrast of background noise, while decreasing the usable signal.

Histogram equalization is a specific case of the more general class of histogram remapping Methods. These methods seek to adjust the image to make it easier to analyze or improve the visual quality.

As the methods of histogram equalization are histogram expansion, cumulative distributive equalization, par sectioning, odd sectioning, and local area histogram equalization.

Process of histogram equalization using cumulative distribution function have described in given figure.

 

                                                  Process of histogram equalization