Computer vision

Computer vision is the study and application of methods which allow computers to "understand" image content or content of multidimensional data in general. The term "understand" means here that specific information is being extracted from the image data for a specific purpose: either for presenting it to a human operator (e. g., if cancerous cells have been detected in a microscopy image), or for controlling some process (e. g., an industry robot or an autonomous vehicle). The image data that is fed into a computer vision system is often a digital gray-scale or colour image, but can also be in the form of two or more such images (e. g., from a stereo camera pair), a video sequence, or a 3D volume (e. g., from a tomography device). In most practical computer vision applications, the computers are pre-programmed to solve a particular task, but methods based on learning are now becoming increasingly common.

Computer vision can also be described as the complement (but not necessary the opposite) of biological vision. In biological vision and visual perception real vision systems of humans and various animals are studied, resulting in models of how these systems are implemented in terms of neural processing at various levels. Computer vision, on the other hand, studies and describes technical vision system which are implemented in software or hardware, in computers or in digital signal processors. There is some interdisciplinary work between biological and computer vision but, in general, the field of computer vision studies processing of visual data as a purely technical problem.

State of the art


The field of computer vision can be characterized as immature and diverse. Even though earlier work exists, it was not until the late 1970's that a more focused study of the field started when computers could manage the processing of large data sets such as images. However, these studies usually originated from various other fields, and consequently there is no standard formulation of the "computer vision problem". Also, and to an even larger extent, there is no standard formulation of how computer vision problems should be solved. Instead, there exists an abundance of methods for solving various well-defined computer vision tasks, where the methods often are very task specific and seldom can be generalized over a wide range of applications. Many of the methods and applications are still in the state of basic research, but more and more methods have found their way into commercial products, where they often constitute a part of a larger system which can solve complex tasks (e.g., in the area of medical images, or quality control and measurements in industrial processes).

Computer vision is by some seen as a subfield of artificial intelligence where image data is being fed into a system as an alternative to text based input for controlling the behaviour of a system. Some of the learning methods which are used in computer vision are based on learning techniques developed within artificial intelligence.

Since a camera can be seen as a light sensor, there are various methods in computer vision based on correspondences between a physical phenomenon related to light and images of that phenomenon. For example, it is possible to extract information about motion in fluids and about waves by analyzing images of these phenomena. Also, a subfield within computer vision deals with the physical process which given a scene of objects, light sources, and camera lenses forms the image in a camera. Consequently, computer vision can also be seen as an extension of physics.

A third field which plays an important role is neurobiology, specifically the study of the biological vision system. Over the last century, there has been an extensive study of eyes, neurons, and the brain structures devoted to processing of visual stimuli in both humans and various animals. This has led to a coarse, yet complicated, description of how "real" vision systems operate in order to solve certain vision related tasks. These results have led to a subfield within computer vision where artificial systems are designed to mimic the processing and behaviour of biological systems, at different levels of complexity. Also, some of the learning-based methods developed within computer vision have their background in biology.

Yet another field related to computer vision is signal processing. Many existing methods for processing of one-variable signals, typically temporal signals, can be extended in a natural way to processing of two-variable signals or multi-variable signals in computer vision. However, because of the specific nature of images there are many methods developed within computer vision which have no counterpart in the processing of one-variable signals. A distinct character of these methods is the fact that they are non-linear which, together with the multi-dimensionality of the signal, defines a subfield in signal processing as a part of computer vision.

Beside the above mentioned views on computer vision, many of the related research topics can also be studied from a purely mathematical point of view. For example, many methods in computer vision are based on statistics, optimization or geometry. Finally, a significant part of the field is devoted to the implementation aspect of computer vision; how existing methods can be realized in various combinations of software and hardware, or how these methods can be modified in order to gain processing speed without losing too much performance.

Related fields
Computer vision, Image processing, Image analysis, Robot vision and Machine vision are closely related fields. If you look inside text books which have either of these names in the title there is a significant overlap in terms of what techniques and applications they cover. This implies that the basic techniques that are used and developed in these fields are more or less identical, something which can be interpreted as there is only one field with different names.

On the other hand, it appears to be necessary for research groups, scientific journals, conferences and companies to present or market themselves as belonging specifically to one of these fields and, hence, various characterizations which distinguish each of the fields from the others have been presented. The following characterizations appear relevant but should not be taken as universally accepted.

Image processing and Image analysis tend to focus on 2D images, how to transform one image to another, e.g., by pixel-wise operations such as contrast enhancement, local operations such as edge extraction or noise removal, or geometrical transformations such as rotating the image. This characterization implies that image processing/analysis does not produce nor require assumptions about what a specific image is an image of.

Computer vision tends to focus on the 3D scene projected onto one or several images, e.g., how to reconstruct structure or other information about the 3D scene from one or several images. Computer vision often relies on more or less complex assumptions about the scene depicted in an image.

Machine vision tends to focus on applications, mainly in industry, e.g., vision based autonomous robots and systems for vision based inspection or measurement. This implies that image sensor technologies and control theory often are integrated with the processing of image data to control a robot and that real-time processing is emphasized by means of efficient implementations in hardware and software.

There is also a field called Imaging which primarily focus on the process of producing images, but sometimes also deals with processing and analysis of images. For example, Medical imaging contains lots of work on the analysis of image data in medical applications.

Finally, pattern recognition is a field which uses various methods to extract information from signals in general, mainly based on statistical approaches. A significant part of this field is devoted to applying these methods to image data.

A consequence of this state of affairs is that you can be working in a lab related to one of these fields, apply methods from a second field to solve a problem in a third field and present the result at a conference related to a fourth field!

Examples of applications for computer vision
Another way to describe computer vision is in terms of applications areas. One of the most prominent application fields is medical computer vision or medical image processing. This area is characterized by the extraction of information from image data for the purpose of making a medical diagnosis of a patient. Typically image data is in the form of microscopy images, X-ray images, angiography images, ultrasonic images, and tomography images. An example of information which can be extracted from such image data is detection of tumours, arteriosclerosis or other malign changes. It can also be measurements of organ dimensions, blood flow, etc. This application area also supports medical research by providing new information, e.g., about the structure of the brain, or about the quality of medical treatments.

A second application area in computer vision is in industry. Here, information is extracted for the purpose of supporting a manufacturing process. One example is quality control where details or final products are being automatically inspected in order to find defects. Another example is measurement of position and orientation of details to be picked up by a robot arm. See the article on machine vision for more details on this area.

Military applications are probably one of the largest areas for computer vision, even though only a small part of this work is open to the public. The obvious examples are detection of enemy soldiers or vehicles and guidance of missiles to a designated target. More advanced systems for missile guidance send the missile to an area rather than a specific target, and target selection is made when the missile reaches the area based on locally acquired image data. Modern military concepts, such as "battlefield awareness," imply that various sensors, including image sensors, provide a rich set of information about a combat scene which can be used to support strategic decisions. In this case, automatic processing of the data is used to reduce complexity and to fuse information from multiple sensors to increase reliability.

One of the newer application areas is autonomous vehicles, which include submersibles, land-based vehicles (small robots with wheels, cars or trucks), and aerial vehicles. An unmanned aerial vehicle is often denoted UAV. The level of autonomy ranges from fully autonomous (unmanned) vehicles to vehicles where computer vision based systems support a driver or a pilot in various situations. Fully autonomous vehicles typically use computer vision for navigation, i.e. for knowing where it is, or for producing a map of its environment (SLAM) and for detecting obstacles. It can also be used for detecting certain task specific events, e. g., a UAV looking for forest fires. Examples of supporting system are obstacle warning systems in cars and systems for autonomous landing of aircraft. Several car manufacturers have demonstrated systems for autonomous driving of cars, but this technology has still not reached a level where it can be put on the market. There are ample examples of military autonomous vehicles ranging from advanced missiles to UAVs for recon missions or missile guidance. Space exploration is already being made with autonomous vehicles using computer vision, e. g., NASA's Mars Exploration Rover.

Other application areas include the creation of visual effects for cinema and broadcast, e.g., camera tracking or matchmoving, and surveillance.

Object recognition
Detecting the presence of known objects or living beings in an image, possibly together with estimating the pose of these objects.

Examples:
 * Searching in digital images for specific content (content-based image retrieval)
 * Recognizing human faces and their location in images.
 * Estimation of the three-dimensional pose of humans and their limbs
 * Detection of objects which are passing through a manufacturing process, e.g., on a conveyor belt, and estimation of their pose so that a robot arm can pick up the objects from the belt.

Optical character recognition
OCR (optical character recognition) takes pictures of printed or handwritten text and converts it into computer readable text such as ASCII or Unicode. In the past images were acquired with a computer scanner, however more recently some software can also read text from pictures taken with a digital camera.

Tracking
Tracking known objects through an image sequence.

Examples:
 * Tracking a single person walking through a shopping center.
 * Tracking of vehicles moving along a road.

Scene interpretation
Creating a model from an image/video.

Examples:
 * Creating a model of the surrounding terrain from images, which are being taken by a robot-mounted camera.
 * Anticipating the pattern of the image to determine size and density to estimate the volume using tomography like device. The cloud recognition is one the government project using this method.

Egomotion
The goal of egomotion computation is to describe the motion of an object with respect to an external reference system, by analyzing data acquired by sensors on-board on the object. i.e. the camera itself.

Examples:
 * Given two images of a scene, determine the 3d rigid motion of the camera between the two views.

Computer vision systems
A typical computer vision system can be divided in the following subsystems:

Image acquisition
The image or image sequence is acquired with an imaging system (camera,radar,lidar,tomography system). Often the imaging system has to be calibrated before being used.

Preprocessing
In the preprocessing step, the image is being treated with "low-level"-operations. The aim of this step is to do noise reduction on the image (i.e. to dissociate the signal from the noise) and to reduce the overall amount of data. This is typically being done by employing different (digital)image processing methods such as:
 * Downsampling the image.
 * Applying digital filters
 * convolutions, computing a scale space representation
 * Correlations or linear shift invariant filters
 * Sobel operator
 * Computing the x- and y-gradient (possibly also the time-gradient).
 * Segmenting the image.
 * Pixelwise thresholding.
 * Performing an eigentransform on the image
 * Fourier transform
 * Doing motion estimation for local regions of the image (also known as optical flow estimation).
 * Estimating disparity in stereo images.
 * Multiresolution analysis

Feature extraction
The aim of feature extraction is to further reduce the data to a set of features, which ought to be invariant to disturbances such as lighting conditions, camera position, noise and distortion. Examples of feature extraction are:
 * Performing edge detection or estimation of local orientation.
 * Extracting corner features.
 * Detecting blob features.
 * Extracting spin images from depth maps.
 * Extracting geons or other three-dimensional primitives, such as superquadrics.
 * Acquiring contour lines and maybe curvature zero crossings.
 * Generating features with the Scale-invariant feature transform.

Registration
The aim of the registration step is to establish correspondence between the features in the acquired set and the features of known objects in a model-database and/or the features of the preceding image. The registration step has to bring up a final hypothesis. To name a few methods:
 * Least squares estimation
 * Hough transform in many variations
 * Geometric hashing
 * Particle filtering
 * RANdom SAmple Consensus

General resources

 * The Computer Vision Homepage
 * Keith Price's Annotated Computer Vision Bibliography
 * HIPR2 image processing teaching package
 * USC Iris computer vision conference list
 * Machine Vision Glossary
 * Machine Vision Glossary

Computer vision laboratories

 * ETH Zürich Computer Vision Laboratory
 * MMVL MediaWiki
 * Kingston University's Digital Imaging Research Centre (DIRC)
 * Computer Vision: Probabilistic and Statistical Inference Group @ University of Toronto

Tutorials

 * CVonline: The Evolving, Distributed, Non-Proprietary, On-Line Compendium of Computer Vision
 * Tutorial to Image Processing
 * Introduction to computer vision (464KB pdf file)

Papers

 * Machine Perception of Three-Dimensional Solids - the paper mentioned by Joseph Mundy in the video

Bildverstehen Visión artificial fa:بینایی رایانه‌ای Vision artificielle コンピュータビジョン datorseende คอมพิวเตอร์วิทัศน์ 计算机视觉