Carridademo Program User Documentation

Revision: 1.0
Date: 2017-08-30
Contact: alpr.support@vision-components.com
Copyright: 2017 Vision Components GmbH, Germany
Author: VC Support

Table of Contents

1   Introduction

Carrida is a software library for Automated License Plate Recognition (ALPR) in images. The purpose of Carrida is to detect and read vehicle license plates and provide them as text strings to your application. The main function of Carrida is very similar to optical character recognition, but it is tuned and optimized to the special structure and syntax of license plates text. Typical applications of the Carrida engine include:

  • Parking lot management (register license plates as they enter/exit a parking lot)
  • Vehicle ticketing/management
  • Automatic gate opening
  • Vehicle counting
  • Law enforcement applications
  • Traffic planning (count vehicles, map traffic to geographic regions)
  • Traffic flow monitoring

For those applications, Carrida provides a simple programming interface to extract license plate strings from images.

Carrida offers the following features and benefits:

  • It is easy to configure
  • Any number of license plates per image can be detected
  • Support for State/Country recognition
  • Reading of license plates with two rows
  • Reading of license plates with stacked characters
  • Robustness against image varying quality (illumination, contrast, noise, rotation)
  • Computation of confidences for detected plates as well as for individual characters
  • Exact position of detected plates as well as all individual characters within the image is delivered
  • Thread safe implementation

A list of abbreviations used throughout this document:

ARM
Stands for processors with the ARM instruction set.
ALPR
Automatic License Plate Reader.
dB
Decibel, in the Carrida documents used to describe a relative sensitivity (gain) value in cameras.
FOV
Field of view.
HW
Hardware.
LP
License Plate.
ms
Millisecond, one 1/1000 th of a second.
PC
Personal Computer.
px
Pixel. The smallest unit of resolution in an image. For example, VGA images are made up of 640x480 pixels.
ROI
Region of Interest, typically a marked area in an image.
SDK
Software Development Kit.
SW
Software.
x86
Stands for Processor Architectures with the Intel instruction set.
VC
Vision Components GmbH.
OSD
On-screen display
Image symbols used in this document
Symbol Meaning
Note Sign Points to especially important aspects of the manual or the software.
Hint Sign The Light bulb highlights hints and ideas that may be helpful.
Warning Sign This warning sign alerts of possible pitfalls to avoid. Please pay careful attention to sections marked with this sign.

2   Getting started with Carridademo

When starting the program for the first time Quick help window will appear, as shown on the image below. It provides useful information about the program and quick tips on how to use it.

./images/Picture5.PNG

If you wish not to see Quick help each time on start-up uncheck ,Show on start-up’.

Illustrated on the image below is the graphical user interface for Carridademo. The Menu bar (1) contains general sub-menus: File, Tools, Window and Help, detailed in Chapter 3.

./images/Picture20.PNG

Carridademo user interface

Source bar (2) is used for defining input sources.

Input can be selected as:

  • folder with images,
  • video file,
  • live stream and
  • USB camera(s).

Specifics about different source types are elaborated in Chapter 4.

The Log Window (3) that appears at start-up contains the logging history during the time the program is running. When closed it can re-open it in Windows sub-menu located on the Menu bar (1).

./images/Picture20_b.PNG

Carridademo user interface

On the right side of the program window the Settings file is located. This is where classifiers are defined.


Parameter settings (4) are located on the right side of the program window under the Settings file. They are divided in three sub-categories Basic, Stream and Advanced settings, detailed in Chapter 5.

On the Results card (5) last result image can be viewed by clicking on ,View image’. The ,History’ button will open a new window with an extended list of results (look at the image below).

./images/Picture23_c2.PNG

List of results images.

Highlighted on the image above is the ,Open output folder’ button. It opens the folder with result images defined during installation.

Clicking on any of the entries on the list will open the image on the OSD, and information about the reading will display on the right side beside the picture.

Hint

Hint Sign The results in History are limited to the amount set in File-Settings-Number of last results to keep in the program which is by default set to 100.


On the image below two green vertical lines are positioned on the lower left side of the OSD (6). These lines are to illustrate the size of characters in pixels.

The engine will recognize only those license plates with characters bigger than the min size (left green bar) and smaller than max size (right green bar).

./images/Picture20_c.PNG

Carridademo user interface

Min is the Minimal letter height, and Max being the Maximal letter height which are part of Basic settings (4). Use these bars as a guideline for more appropriate size estimation in that these parameters can affect processing speed and reading results.


The bottom part of the screen is the Play/Record bar (7). When video or image files are selected this bar will serve for enabling/disabling image processing, playing/pausing the source material.

Shown in the following table are steps how to start reproducing source material, license plate detection and producing results.

Only when in Recording mode the images are being automatically saved in the output folder.

When a stream is suspended, it will be indicated with a red "Paused" sign on the bottom of the screan.


The white/blue bar next to the Play/Pause button indicates the percentage of the stream that is currently processed (image below).

./images/Picture15_a.PNG

How to read information displayed above the bar:
p7 Average processing time for the last 100 readings.
p8 Percentage of image magnification.
p9 Size of the area in which the ALPR will search for plates.
p10 Position of the mouse on the image in pixels

2.1   Starting image processing

The green rectangle bordering the image (see figure below) represents the region of interest (ROI) for ALPR detection, meaning that the program will search for plates only inside of this rectangular area.

./images/Picture21.PNG

Resizing the region of interest (ROI) for ALPR.

Before any image processing starts make sure the ROI size covers the path on which the plates will move through the screen.

ROI can be re-sized by:

  • clicking on any of the four corners and dragging them to the wanted position on the screen, or
  • click on any of the lines and carry the border to the desired position.

Images size on the OSD can be manipulated. Look at the following steps how to zoom in/out an image:

  • Press and hold ALT and scroll in and out with the mouse.


    65a

  • Press + or - buttons on the keypad.


    65b

  • When zoomed in you can pan the image by pressing ALT and dragging the image with the mouse while pressing left mouse button.


    65c

  • Double-clicking on the image it will resize it to the full screen. To exit full screen view press Escape (Esc) or double-click on the image.


Hint

Hint Sign Use Min and Max sidebars to define the character size. Enlarge the image and use the sidebars as guidelines to set the size more accurately (see picture below).


ii



Minimal and maximal letter height can be modified in Basic settings. When the size is optimized processing speed increases as well as the reading certainty.


After setting the ROI and defining the min and max letter height do the following:

n1 p11 Select an input source: Video file, Image folder, USB camera or Live stream. Two folders with video and image files are provided with the demo program.
n2 p12

Click on ,Play’ to start stream processing (if it did not start automatically).

Press ,Record’ to start saving results on a hard drive.

To pause image streaming process click on ,Pause’. p4

To start saving results on a hard drive press ,Record’. It will change color to red when in recording mode.

Press the same button to pause/stop recording.

p5
n3 p13 Go to History on the Results card to view processed images. Image files are saved in the defined output folder, and a direct link can be found in History window.

After everything is set up properly as soon as a vehicle is detected license plate processing will start. The image below shows an example of one output produced by the ALPR.


i23

This is a result image with ,Overlay confidence’ and ,Overlay states’ turned on.

At position 1 state information is displayed, followed by the percentage of recognition certainty (2).

On position 3 vehicle plate is displayed, followed by the percentage of recognition certainty (4).

4   Source bar

Source material can be imported from USB camera(s), Video file, Folder with images or from a Live stream.

./images/Picture17_.PNG

Hint

Hint Sign With Carridademo two samples are provided, a video file, and an image folder. They are intended for testing the program and different parameter settings.



When selecting any of the sources they will have common general elements, illustrated on the image below.

ag 1 Region of interest (ROI)

The green rectangular border represents region of interest for ALPR. Just inside these borders ALPR will detect plates.

Resize it by clicking and dragging any corner, or click on any of the lines and "carry" the border to a desired position.

2 History History will contain a limited list of results with images. The size can be defined in File-Settings-Number of last results.
Live

Image streaming will be displayed on the Live card in real time without optional processing is implemented (image rotation, deinterlacing, etc.).

In situations where ALPR results can be displayed with a delay, for example offline streaming mode, Live image will display the input stream without any delays.

3 Start ALPR detection

On the Play/Record bar pressing ,Play’ will start image streaming.

Press the ,Record’ button for ALPR detection to start. Processed images will than be saved to the output folder.

Examples and detail explanation can be found in Chapter 2.1, Start image processing.

4 Refresh and Folder buttons

When using image folders and video files an additional ,Folder’ button will appear. Use it to open new files or folder of the same source.

Use the ,Refresh’ button to reinitialize files/folder in use.

4.1   USB Camera

When selecting USB Camera as the source, image streaming from the camera should appear on the OSD.

To start image processing click on the ,Record’ button (1).

Processed images will than be saved in the specified output folder.

./images/Picture27.PNG

USB camera source.

When input stream is suspended it will be indicated with a red "Paused" sign on the bottom of the OSD.

Note that in some cases ALPR results will be available with a delay (for example in offline streaming mode, see stream presets further in this document). Live image without any delays can be viewed on the Live card (2) if ,Show live stream’ (3) is enabled.


cam If more than one camera is connected the additional window may appear as shown on the image. Select a specific camera from the drop down list of connected devices and choose one for the main source.

4.2   Video file and Image folder

Select one of these sources to reproduce single video file or a folder with one or more images.

When Video file is selected the program will only recognize video formats, not images and vice versa. If the selected source format conflict with the data chosen an error message will appear.

To change the path for prior selected source click on the Folder icon (1). And to reload the same folder/video click on the Refresh icon .

./images/Picture63.PNG

On the Play/Record bar (2) image streaming can be started/paused and recording processed images can be activated. When the streaming is suspended a red "Paused" sign will appear. Further use of the buttons is explained in the table below.

Click on ,Play’ to start stream processing (if it did not start automatically). sf
To pause image streaming process click on ,Pause’. p4

To start saving results on a hard drive press ,Record’. It will change color to red when in recording mode.

Press the same button to pause/stop recording.

p5

Only when the Record button is activated (red) results will be produced to the destination folder.


On the Results card (3) basic information about the latest available ALPR result is displayed. The last result image can be accessed through ,View image’ and the list of results can be found in ,History...’.

4.3   Live stream

For Live stream you need to specify path to the stream (rtsp or http) in the pop up window.

How to start image processing and view results read in Chapter 2.

5   ALPR parameter settings

Generally all ALPR relevant parameters are stored in the ALPR settings file (1).

./images/Picture28.PNG

The default Carrida intallation includes only so called world classifier.

What is a classifier?

Classifiers are files which contain state/country specific information. This information can for example be the size an font of letters and numbers, plate dimensions, format etc. They are divided in regions (Europe, USA, Thailand...) and each region contains different country's.

Using the suitable classifier will improve accuracy of plate recognition according to the country/state parameter as they support ALPR with features that make recognition easier and more reliable.

The "demo" file is set as a default classifier. This is a default world classifier suited to read license plates containing Latin letters (A-Z) and Arabic numerals (0-9). This classifier does not contain state recognition or region specific information. For that reason ALPR generates state recognition result as "WORLD" with "0%" recognition certainty.

To increase reading accuracy and enable state recognition an appropriate classifier must be applied.


After selecting an appropriate setting file, its parameters will be loaded and displayed in the corresponding fields (Basic, Stream and Advanced settings). All parameters can be changed "on fly", meaning every change will have immediate effect. To make changes persistent, click on the ,Save’ icon.

Parameter settings are divided into three groups:

  • Basic
  • Stream
  • Advanced settings.

In the bottom right side of each parameter group, help button (2) is located. When pressing on this button, a window will appear with detailed description of each parameter.

5.1   Basic settings

Basic settings offer the possibility to change most frequently used parameters. Optimizing the minimal and maximal letter height is important before starting image processing, and is located in basic settings. Setting the character size accurately will improve both recognition quality and processing time.

./images/Picture51.PNG

Depending on the source material being processed a corresponding mode must be selected. Modes are set to have different processing speed and certainty of recognition. When for example live image stream is set as the source, and processing speed is important then fast mode is optimal. When processing speed is not that important, in case of video files, then best mode is optimal, as it has the best possible reading quality.

More information on basic settings are in the continuation of this Chapter.

5.1.1   Mode

The Carrida engine can operate in several performance modes, which are differentiated by recognition speed and reading confidence.

You can choose between the following modes:

  Mode Description
mo Freeflow Fastest possible processing speed with lower recognition quality as in other modes.
Fast Fast processing speed with slightly better recognition quality than in freeflow.
Normal A good compromise between speed and reading performance, recommended to be used in most situations.
Best The best possible reading quality. Can be up to two times slower than in the fast mode. At the same time, reading confidence will be about 3-4% better.

As the best compromise between processing speed and reading accuracy we recommend to operate in Normal mode.

When working in an environment where processing time is not critical or (e.g. processing of video files) or when using powerful hardware, use Best mode to reach the best possible reading accuracy.

5.1.2   Minimum plate confidence (%)

The minimum confidence level of license plates to be accepted. License plates having lower confidences will be rejected.

Example:If plate confidence level is set to 70 the program will produce only results recognized with confidence of 70% or higher, all other detections will be ignored.

5.1.3   Minimum/Maximum letter height (in pixels)

Define the minimal and maximal height of characters that ALPR will recognize. The minimum value cannot not be less than 8 pixels, for a maximum value there are no limits.

These are one of the most important parameters - setting them accurately will improve both recognition quality and processing time.

Use the Min and Max sidebars on the bottom left side of the image to control the correctness of the min/max values.

./images/Picture30.PNG

Zoom in (+/-) and use Min and Max bars as guidelines to assess the average size of characters.

Use the zoom buttons to enlarge the image and set min and max height more precise. Optimizing the size will increase processing speed and improve reading accuracy.

5.1.4   Deinterlace

Deinterlacing is a process of converting interlaced video (such as analog signals or 1080i format HDTV signals, etc.) into a non-interlaced form.

If enabled, deinterlacing will be applied to the input image before further processing.

5.1.5   Scale width/height

When these parameters are adjusted input images will be resized by given ratios in percentage before processing.

New values have to be provided in percent of the original size (100 = original size).

This procedure can be found useful when license plates are visually distorted due to not ideal camera positions, especially for steep top views.

Also, scale parameters can be used to reduce image resolution for faster processing. For example when the image is bigger than needed it can be reduced in size and the program will not have to process as many pixels.

5.1.6   Rotate/Flip image

Images can be rotated before processing begins. In case of rotation all further settings will apply on rotated images. Output license plate coordinates and output images will be rotated as well.

Possible rotations:

  • None - no rotation
  • Rotate right - 90 degrees clockwise.
  • Rotate left - 90 degrees counter clockwise.
  • Rotate image 180 degrees (mirror it both horizontally and vertically).
  • Flip image horizontally.
  • Flip image vertically.
./images/Picture52.PNG
./images/Picture53.PNG

5.1.7   Rotation angle

Rotate the angle of input images in one degree steps so cars appear in images horizontally. This is not a requirement, but it can improve recognition quality slightly.

Example:

How the rotation angle tool can help when the camera does not have the ideal angle to the vehicle:


./images/Picture62_ab.PNG

5.2   Stream settings

Stream parameters are related to handling of video streams - i.e., situations where a number plate is visible in more than one successive frame. Mostly, this concerns the way how multiple license plate readings are merged into one reading result per vehicle.

Parameters can be set by individual preference, but there are several streaming modes available that set individual parameters to some meaningful predefined values.

./images/Picture14.PNG

Stream parameters are set according to the type of source material. Meaning if video file is selected as the source, streaming mode will automatically be set to offline video. This option can be disabled in File-Settings un-checking ,Automatically adjust stream preset depending on source type’ .

5.2.1   Apply preset

In order to make some basic setups easier to handle, the demo program provides several presets, which can be used as a good starting point for further experiments.

The stream presets provide processing settings for various application scenarios, detailed in the table below.

    Usage Output
pr Single images Streaming parameters are configured in such a way that the video handling function is switched off. Each input image is considered not to be related to adjacent images. Every single detected license plate. Recommended for folder with images.
Parking mode This mode is designed for access and parking control systems. Each vehicle number plate is required to be detected at least three times. The output result is available immediately afterwards. Unique license plates. Recommended for live stream and USB cameras.
Offline video In this mode results are available with a delay of about 25 frames. The reading quality is the best in this mode as the engine has more frames to analyze before producing results. It is not suited for real time application where any delay is unwanted. The main purpose of this mode is to process video files. Unique license plates. Recommended for video files.
Free flow This mode is designed for freeflow real-time systems. It is configured in such a way that it tries to catch all passing vehicles. The result may become available with a delay up to 15 frames. Unique license plates. Recommended for live stream and USB cameras.

By default ,Automatically adjust stream preset depending on source type’ is enabled. Meaning that every time when a new source is selected ALPR will switch the preset mode accordingly (e.g. live stream preset may be changed to freeflow). Each individual parameter can be set explicitly (thus overriding its default value). In that case the automatic adjustment may not be desired, and can be disabled in File-Settings.

5.2.2   Minimal number of readings

Refers to the minimal number of required readings per vehicle before its license plate will be produced to output. It can be used to suppress occasional wrong readings.

This parameter can be set in range from 1 to 10 readings.

If set too high, the engine may produce less output images, as the vehicles may pass too quickly to read them the desired number of times.

Example:

Readings for the last seven successive frames: ABC123 x ABC123 x x x ABC123 (x being a frame without reading results).

If Minimal readings = 5, no images will be produces. But if e.g. Minimal readings = 2, output image will be produced as there are more than 2 readings detected.

5.2.3   Buffer length

A buffer represents memory storage used to temporarily store data while it is being moved from one place to another.

When buffer length is set to a higher value than 1 results will become available with a delay.

The number of delayed frames is equal to the defined buffer length [0...100].

The advantage is that the engine has more frames to analyze before producing the output results which gives time to correct possible reading errors by merging multiple reading results.

Hint

Hint Sign A suggestion is to set this parameter to a lower value for real-time applications (Live stream, USB camera) where any delay is critical, and to increase this parameter for offline applications (Video files).

5.2.4   Drop size

Ideally, a license plate should be detected in each frame during which the vehicle moves across the screen. In practice, however, there can come to some time (frame) gaps in readings.

Drop size parameter [0...100] specifies the biggest allowed gap in frames in which to keep the license plate string in the processing history. After reaching this gap, the vehicle with a relevant license plate string is removed from the memory. New readings would trigger new detection.

Opposite to Buffer length it does not produce any delays in results.

Example:Readings for the last seven successive frames: ABC123 x ABC123 x x x ABC123 (x being a frame without reading results). If DropSize = 5, single result will be produced. But if e.g. DropSize = 2, two reading results will be produced, as there is a gap of three frames without reading results.

5.2.5   Set horizontal/vertical border distance factor

The intention of these parameters is to filter (reject) license plates that are located too close to image borders, as there is a probability exists that a license plate is not fully visible yet.

These parameters are relative to the average height of the characters on the plate. ALPR will ignore plates which appear closer to the ROI than Horizontal border multiplied by detected plate height/.

Example:Horizontal border is set to 3.0, and the plates appear in e.g. average size of 25 pixels, the license plate must have a minimum distance to the left or right side of ROI 3x25=75 pixels.

5.2.6   Maximal matching distance

Describes the maximal Levensthein distance* between two ANPR reading results achieved in two different frames. It is used by the program to make a decision if these readings belong to the same vehicle or not.

Setting this parameter to a higher value increases tolerance of the system to reading errors. Normally, there is no need to change this parameter.

The Levenshtein distance = Number of character deletions + Insertions + Replacements to transform one string into another. For example, the Levenshtein distance between "HR737AM" and "HR231AN" is 3, since it takes exactly three edits to change one into the other.

5.2.7   Process every frame

If this parameter is checked, each upcoming frame is guaranteed to be processed. Here you need to keep in mind that if image queues for all Carrida instances are full, the engine will be blocked until one of the queues will be able to accept a new image. If not checked, an input image will be rejected from processing in case all queues are full.

The recommendation would be to set Mode to false (leave unchecked) for processing Live streams and to set it to true (check) for processing of offline Video files or Image folders, so that no frame will be missed.

5.3   Advanced settings

./images/Picture8.PNG

5.3.1   Minimal character confidence

The minimal classification precision value for each symbol on the plate to be accepted by the recognition software. The range is [0..100]. Normally, there is no need to change this parameter.

Example:./images/Picture56.PNG

On the first image ,Min character confidence’ is set to 30%, and all of the letters have been processed as output.

Second example sets ,Min char confidence’ at 70%, and the result for the same plates will be different as number "8" has recognition certainty at 55%.

5.3.2   Maximal plate angle

This parameter allows to change the largest expected license plate angle relative to x-Axis. It is defined in degrees, and can be set up to 45°.

5.3.3   Maximal number of plates in frame

This parameter refers to the maximal number of license plates that the engine will return per image.

Example:./images/Picture57.PNG

On the first image the limitation for ,*,Max plates in frame’* is set to 1 and just one result has been produced.

In case of multiple plates in frame plates with higher confidence will be selected over the ones with lower confidence.

Hint

Hint Sign If set to any, all recognized plates will be returned.

5.3.4   Minimal/maximal number of symbols on the plate

The smallest and the biggest number of characters (letters and numbers) on the license plate that the software is allowed to recognize. Normally, there is no need to change this parameter.

Example:./images/Picture58.PNG

On the first image ,Min number of symbols’ is set to 8, the program does not accept plates with less characters and the result is not generated. On the second image ,Min number of symbols’ is set to 7 and the same plate gets processed as now it meets the requirements.

5.3.5   Minimal contrast

Defines the minimal contrast to differentiate the foreground (characters) from background (license plate itself). It is defined in pixels and can be set between 8 and 100. By default it is set to 12 px in best mode, and 50 px for other modes.

Hint

Hint Sign Reducing this parameter can increase both reading performance and processing time.

5.3.6   State recognition

Disabling state recognition switches off the state recognition process for classifiers with built-in state recognition.

Hint

Hint Sign Processing time will be reduced, but the reading performance can degrade slightly.

5.3.7   Filter noise

When checked image noise will be filtered before processing begins.

Usage of this parameter can significantly improve run-time performance for noisy images.

5.3.8   Remove pure letter/number plates

If checked ,Remove pure letters’ the program filters license plates that consist only of letters, and vice versa. If ,Remove pure numbers’ is checked the program will not process plates with only numbers.

Example:

Shown on the first image is a vehicle with a license plate formed of only letters. The second image shows a vehicle with the license plate made of pure numbers.

./images/Picture59.PNG

If you wish not to process images with pure letters check ,Remove pure letters’.

5.3.9   Search inverted

If checked, white-on-black (inverted) plates will be also searched for in case when no black-on-white plates are found.

Example:./images/Picture60.PNG

If ,Search inverted’ is disabled the program will not detect inverted license plates, as illustrated on the first image. On the second image this parameter is enabled and the ALPR processed the plates and generated output.

Hint

Hint Sign Setting this parameter to true will increase the processing time per image.

5.3.10   Always search inverted

If checked, inverted license plates will be always searched for, independent of whether there are no inverted plates found.

Hint

Hint Sign Setting this parameter to true will increase processing time per image.