In recent years, 3D sensing technology has been continuously improved, component modules have become increasingly compact, prices of high-quality CMOS image sensors and powerful data processors have plummeted, and the application of machine vision has also shown a blowout trend.

New markets for machine vision systems include:
Factory automation and robotics: Visual inspection, Diagnostics, Assembly, Mobile robots, Digital fabrication, Service robots, Rescue robots

Intelligent transportation systems: Traffic monitoring, Autonomous vehicles, Driver safety assistance

Security and Law Enforcement: Security surveillance, Camera networks, Remote sensing, Underwater and Harsh environment operations

Life sciences: Agriculture, Forestry, Fisheries, Civil/ Construction engineering, Business, Sports, Fashion, Home furnishing, etc.

Multimedia: Database archiving/retrieval, Documentation, Culture/Heritage, Virtual reality (VR)/Mixed reality (MR)/Augmented reality (AR), Entertainment

Biomedicine: Tomography, Endoscopy, Computer-Assisted Diagnosis, Computer-Assisted Surgery, Computational Anatomy, Bioinformatics, Nursing

Human-computer interaction: Facial/ Gesture/ Behavior/ Gait/ Gaze analysis, Biometric Identification, Wearable computing, First-person vision system

The key to successfully developing and implementing machine vision solutions is to choose a trusted technology partner to build the necessary hardware components and software frameworks to deliver vision algorithms, camera interface standards, advanced analytics, artificial intelligence, and machine learning. For any business, there are things can do and things can not to do. We have summarized 10 considerations to note when developing modern machine vision solutions.

1. AI development starts with quality data: For machine vision systems to function properly, they need to acquire, process, analyze, and understand images, all of which is done by AI. To achieve this understanding, it first needs to compile information, which is training data for the AI to learn from. The better the quality of the training data, the better the quality of the AI. This also improves the performance of the machine vision system. Poor quality or insufficient training data will hinder the success of AI and machine vision applications. Without proper training data, even the best programmed AI will not perform as expected.

2. Adding features and losing sight of the original intention: Start every project with a set of realistic expectations and goals. The human brain is able to process data from all five senses simultaneously and act on that data instantly—true multitasking. Machines are often programmed to do a single task well, but AI can struggle when it comes to learning and performing multiple tasks. During the initial planning phase, focus on the key features that will determine success or failure. The initial version of an AI application that performs a wide variety of tasks can result in tasks not being performed correctly and unsatisfactory initial results.

3. The Language of Vision: Successful machine vision applications require not only powerful hardware, but also superb programming skills. Programming can take the form of AI frameworks and coding languages. AI frameworks can simplify and accelerate the development of AI applications, including machine learning, deep learning, neural networks, and natural language processing (NLP). AI frameworks can serve as development templates for AI systems. This approach is much easier to develop, deploy, and manage than developing AI applications from scratch. There are several programming languages that can be used with AI, each with unique advantages. These include Caffe, Python, Pytorch, TensorFlow, C++, Lisp, Java, R, Prolog, and Model Zoo

4. Choosing the suitable hardware host: There are many options when choosing the hardware to run your machine vision AI application. Field programmable gate arrays (FPGAs), graphics processing units (GPUs), and microcontrollers (MCUs) all have their strengths.

FPGA: FPGAs are very powerful processing units that can be configured to meet the needs of almost any application. Custom FPGA architectures can be created to handle specific applications. Achieve higher performance, lower cost, and better energy efficiency compared to other options such as GPU and CPU.

GPU: GPUs are specialized processors that are primarily used to process images and videos. Compared to CPUs, they have simpler processing units but carry more cores. GPUs are therefore ideal for applications that require processing large amounts of data in parallel, such as image pixels or video codecs. However, GPUs also have some limitations: they consume a lot of energy and are less flexible than CPUs because they are programmed using languages such as CUDA and OpenCL.

CPUs: CPUs have a limited number of cores and can‘t quickly process the large amounts of data required for AI. Therefore, CPUs are only suitable for small models with small effective batch sizes. The advantages of CPUs are that they are easy to program, low cost, and widely support various programming frameworks.

5. Image Sensors and Illumination: Great advances have been made in front-side illuminated (FSI) and back-side illuminated (BSI) technology for CMOS sensors, allowing for higher resolution images even at low light levels. Proper illumination is also an important consideration. The basis of all illumination performance can be reduced to three main image sensor characteristics: quantum efficiency (QE), dark current, and saturation capacity. Quantum efficiency is the ratio of charge produced by a device for a specific number of incident photons. Since QE varies at different wavelengths, it is best plotted as a function of wavelength. From this, an accurate measurement of the device's sensitivity can be achieved. When mounted inside a camera, the maximum QE of the camera should be less than the sensor due to external optical and electronic effects.

Dark current and saturation capacity are also important considerations for machine vision system design. Dark current measures the change in the number of electrons generated by heat within a CMOS imager, and the noise that may be introduced at the same time. Saturation capacity indicates the number of electrons that can be stored by a single pixel. Although these parameters are not usually stated in camera manufacturers' data sheets, they can be used together with QE measurements to calculate the maximum signal-to-noise ratio (S/N), absolute sensitivity, and dynamic range of the application.