PRELIMINARY VC MIPI Camera Module Operating Manual

PRELIMINARY Hardware specifications of VC MIPI Camera Modules

Revision: 0.3
Date: 2019-03-15
Contact: support@vision-comp.com
Copyright: 1996-2019 Vision Components GmbH Ettlingen, Germany
Author: VC Support, mailto:support@vision-comp.com
./images/mipi_sensor_diagonal.png
Foreword and Disclaimer

This documentation has been prepared with most possible care. However Vision Components GmbH does not take any liability for possible errors. In the interest of progress, Vision Components GmbH reserves the right to perform technical changes without further notice.

Please notify support@vision-components.com if you become aware of any errors in this manual or if a certain topic requires more detailed documentation.

This manual is intended for information of Vision Component’s customers only. Any publication of this document or parts thereof requires written permission by Vision Components GmbH.

Image symbols used in this document
Symbol Meaning
Note Sign The Light bulb highlights hints and ideas that may be helpful for a development.
Warning Sign This warning sign alerts of possible pitfalls to avoid. Please pay careful attention to sections marked with this sign.
Example Sign This is a sign for an example.

Trademarks

Linux, Debian, the Tux logo, Vivado, Xilinx and Zynq, ARM, Cortex, Windows XP, Total Commander, Tera Term, Motorola, HALCON, Vision Components are registered Trademarks. All trademarks are the property of their respective owners.

ESD sensitivity

Warning

Warning Sign The components are very sensitive to electrostatic discharge (ESD)! Please take all the precautions necessary to avoid ESD!

ESD

ESD Sign The electronic components and circuits are sensitive to ElectroStatic Discharge (ESD). When handling any circuit board assemblies, it is necessary that ESD safety precautions be observed.

ESD safe best practices include, but are not limited to:

This note is not an exhaustive information about the protection against electrostatic discharge (ESD).

Table of Contents

1   General Information

1.1   Hardware Compatibility

MIPI-Modules can be connected to a number of different computer boards like the

  • Raspberry Pi 1, 2, 3, 3B, 3B+ (with a 22-to-15 FPC cable)
  • Raspberry Compute Module - CMIO Rev. 3.0 (with a 22pin FPC cable)
  • Raspberry Zero v. 1.3 (with a 22pin FPC cable)
  • Tinker board (with a 22-to-15 FPC cable)
  • 96boards computer boards (with 22pin FPC cable and the 96 adaptor board)
  • Nvidia TX2 development board (with 22pin FPC cable and the TX2 adaptor board)

Note

Note Sign

VC performs compatibility tests for a number of boards on a regular basis and publishes the results. However, no guarantee for the sensor hardware compatibility for any third party board can be given.

1.2   Technical Specification

Technical Data: VC MIPI IMX290, VC MIPI IMX296, VC MIPI OV9281
Component / Feature Module Name  
VC MIPI IMX290 VC MIPI IMX296 VC MIPI OV9281  
CMOS Sensor 1/2.8" Sony IMX290, monochrome, 2.0 Mpixel 1/2.9" Sony IMX296, Monochrome, 1.56 Mpixel 1/4" Omnivision OV9281, Monochrome, 1.02 Mpixel  
Active pixels 1937(H) x 1097(V) (full HD) 1440(H) x 1080(V) 1280(H) x 800(V)  
Pixel size 2.90(H) x 2.90(V) µm 3.45(H) x 3.45(V) µm 3(H) x 3(V) µm  
Active sensor size 5.617(H) x 3.181(V) mm 4.968(H) x 3.726(V) mm 3.840(H) x 2.400(V) mm  
Shutter type Rolling Shutter Global shutter Global shutter  
ADC Resolution 10 Bit / 12 Bit 10 Bit 8 bit / 10 bit  
CSI-2 output 2 lanes / 4 lanes (not 15pin connector) / RAW 10 or RAW 12 1 lane / RAW 10 2 lane / RAW 8/10  
CSI-2 speed 891 Mbps/lane max. 1.2 Gbps/lane max. 800 Mbps/lane max.  
Max fps (full resolution) 120 fps at 10 Bit / 60 fps at 12 Bit (4 lanes) 60.3 fps 120 fps / 60 fps externally triggered  
Output modes streaming mode streaming mode, fast trigger mode streaming mode, trigger mode (must be > 10 fps)  
Shutter resolution 1 horizontal unit in streaming mode: 1 horizontal unit, in fast trigger mode: 200nsec in streaming mode: 1 horizontal unit, in trigger mode: 1 horizontal unit  
Gain 0–30dB analog gain + 0–42dB digital gain 0–24dB analog gain + 0–24dB digital gain analog and digital gain  
S/N ratio n/a n/a 38dB  
dynamic range n/a n/a 68dB  
HDR Yes: DOL HDR (digital overlap HDR) No No  
CRA 6.14 degrees 1.8 degrees 9 degrees  
ROI support Yes Yes Yes, but no timing advantage  
Binning support No Yes: 2x2 binning (simultaneous ROI-mode not supported) Yes, 2x2 binning  
Reverse modes for horizontal / vertical: normal or inverted readout for vertical or horizontal: normal or inverted readout for vertical or horizontal: normal or inverted readout  
Trigger input No Yes in fast trigger mode Yes in trigger mode  
Flash output Yes Yes Yes  
Power consumption 200mA(@3.3V) typical, 300mA(@3.3V) max. when active 200mA(@3.3V) typical, 300mA(@3.3V) max. when active 80mA(@3.3V) typical, 100mA(@3.3V) max. when active  
Operating temperature -30 to +75 deg C, non-condensing -30 to +75 deg C, non-condensing -30 ... +85 deg C, non-condensing  
Storage temperature -30 to +80 deg C -40 to +85 deg C -40 ... +85 deg C  
Performance guarantee temperature -10 to +60 deg C -10 to +60 deg C 0 ... +50 deg C  
Supply voltage 3.3V +/- 3% at input of module (not including voltage drop on cable) 3.3V +/- 3% at input of module (not including voltage drop on cable) 3.3V +/- 3% at input of module (not including voltage drop on cable)  
Technical Data: VC MIPI IMX274 C, VC MIPI IMX326 C, VC MIPI IMX327 C
Component / Feature Module Name  
VC MIPI IMX327 C VC MIPI IMX326 C VC MIPI IMX274 C
CMOS Sensor 1/2.8" Sony IMX327 (better ver. of IMX290), color, 2.0 Mpixel 1/2.9" Sony IMX326, color, 6.8 Mpixel 1/2.5" Sony IMX274, color, 8.6 Mpixel
Active pixels 1937(H) x 1097(V) (full HD) 3096(H) x 2196(V) 3864(H) x 2196(V)
Pixel size 2.90(H) x 2.90(V) µm 1.62(H) x 1.62(V) µm 1.62(H) x 1.62(V) µm
Active sensor size 5.617(H) x 3.181(V) mm 5.015(H) x 3.558(V) mm 6.260(H) x 3.558(V) mm
Shutter type Rolling Shutter Rolling Shutter / Global Reset Shutter Rolling Shutter / Global Reset Shutter
ADC Resolution 10 Bit / 12 Bit 10 Bit / 12 Bit 10 Bit / 12 Bit
CSI-2 output 2 lanes / 4 lanes (not 15pin connector) / RAW 10 or RAW 12 4 lanes / RAW 10 or RAW 12 no support for 15pin connector 4 lanes / RAW 10 or RAW 12 no support for 15pin connector
CSI-2 speed 891 Mbps/lane max. 1.440 Gbps/lane max. 1.440 Gbps/lane max.
Max fps (full resolution) 120 fps at 10 Bit / 60 fps at 12 Bit (4 lanes) 60 fps at 10 Bit / 30 fps at 12 Bit 60 fps at 10 Bit / 30 fps at 12 Bit
Output modes streaming mode streaming mode / global reset shutter mode streaming mode / global reset shutter mode
Shutter resolution 1 horizontal unit 1 horizontal unit 1 horizontal unit
Gain 0–30dB analog gain + 0–42dB digital gain 0–27dB analog gain + 0–36dB digital gain 0–27dB analog gain + 0–36dB digital gain
S/N ratio n/a n/a n/a
dynamic range n/a n/a n/a
HDR Yes: DOL HDR (digital overlap HDR) Yes: DOL HDR (digital overlap HDR) Yes: DOL HDR (digital overlap HDR)
CRA 6.14 degrees 3.9 degrees 4.5 degrees
ROI support Yes Yes Yes
Binning support No Yes: 2x2 binning Yes: 2x2 binning
Reverse modes for horizontal/vertical: normal or inverted readout for vertical: normal or inverted readout for vertical: normal or inverted readout
Trigger input No No No
Flash output Yes Yes Yes
Power consumption 200mA(@3.3V) typical, 300mA(@3.3V) max. when active 200mA(@3.3V) typical, 300mA(@3.3V) max. when active 200mA(@3.3V) typical, 300mA(@3.3V) max. when active
Operating temperature -30 to +85 deg C, non-condensing -30 to +75 deg C, non-condensing -30 to +75 deg C, non-condensing
Storage temperature -40 to +85 deg C -30 to +80 deg C -30 to +80 deg C
Performance guarantee temperature -10 to +60 deg C -10 to +60 deg C -10 to +60 deg C
Supply voltage 3.3V +/- 3% at input of module (not including voltage drop on cable) 3.3V +/- 3% at input of module (not including voltage drop on cable) 3.3V +/- 3% at input of module (not including voltage drop on cable)
Technical Data: VC MIPI IMX412 C
Component / Feature Module Name  
VC MIPI IMX412 C    
CMOS Sensor 1/2.3" Sony IMX412, color, 12.3 Mpixel    
Active pixels 4056(H) x 3040(V)    
Pixel size 1.55(H) x 1.55(V) µm    
Active sensor size 6.287(H) x 4.712(V) mm    
Shutter type Rolling Shutter    
ADC Resolution 10 Bit / 12 Bit    
CSI-2 output 2 lanes / 4 lanes (not 15pin connector) / RAW 10 or RAW 12    
CSI-2 speed 2.1 Gbps/lane max.    
Max fps (full resolution) 60 fps at 10 Bit / 40 fps at 12 Bit (4 lanes)    
Output modes streaming mode    
Shutter resolution 1 horizontal unit    
Gain 0–27dB analog gain + 0–24dB digital gain    
S/N ratio n/a    
dynamic range n/a    
HDR Yes: DOL HDR (digital overlap HDR)    
CRA 12 degrees    
ROI support Yes    
Binning support Yes: 2x2 binning    
Reverse modes for horizontal/vertical: normal or inverted readout    
Trigger input No    
Flash output No    
Power consumption 200mA(@3.3V) typical, 300mA(@3.3V) max. when active    
Operating temperature -30 to +75 deg C, non-condensing    
Storage temperature -30 to +80 deg C    
Performance guarantee temperature -10 to +60 deg C    
Supply voltage 3.3V +/- 3% at input of module (not including voltage drop on cable)    

1.3   Electrical and Mechanical Prerequisites

./images/mipi_sensor_frontal.png

Frontal view of MIPI Camera Sensor

Before connecting, make sure that the pinout of the computer board fits the pinout of the MIPI module, especially the power supply (3V3) and GND pins. In 2014 a number of boards for the Compute Module IO (CMIO) have been sold with reverse pinout which may damage the MIPI Module and the Compute Module when connected. If in doubt contact the manufacturer or the dealer of the computer board. Vision Components can not take resposibility for incorrect pin-outs, changes and modifications of third party manufacturers.

Always use the original Vision Components FPC cable for connecting the MIPI Modules. FPC cables are available with 15 pin or 22 pin header on one side. The MIPI module side always has 22 pins. The original FPC cables connect the signal traces on one side and have a GND shield on the other side. Flat cables with only the signal wires without shielding are not suitable for this purpose.

Note

Note Sign Please be aware that using the FPC cable with 15 pin on one side limits the number of MIPI data lanes to 2 (instead of 4 for the FPC cable with 22 pins on both sides). This may lead to lower frame rates than specified.

Make sure to connect the correct side of the cable with the MIPI Module. The correct side for the MIPI Module is marked "sensor".

Due to the high transmission rate of about 1.5 Gbit/sec per lane the cable length is limited to 200mm. Using a repeater board the cable length can be doubled to 400mm.

1.4   Pin Cable Connector for Vision Components MIPI CSI camera modules

Data signals are compliant to MIPI CSI 2. Not all of the 4 lanes are supported for all sensor modules, e.g.:

  • Raspberry Pi Compute Module IO V3.0: CAM0 connector supports 2 lanes only; CAM1 connector supports 4 lanes
  • Raspberry Pi Zero V1.3 and Raspberry Pi 3B+ support 2 lanes only

1.4.1   Flexible Printed Circuit (FPC) Cable

./images/fpc_pin1_info.png

Position variations of each Pin1 side and 3.3V Pin

./images/mipi_patchcable_connectors.png

Patch Cable Connectors

Warning

Warning Sign At the time of writing the schematics of the models 'Raspberry Pi CMIO V3.0' and the 'Raspberry Pi Zero V1.3' have the pin numbering of the MIPI sensor connector in a wrong, inverse order, so pin 1 would be pin 22, pin 2 would be pin 21, …

  • Always check cable numbering when connecting the sensor boards to hardware not from Vision Components!
  • Vision Components is not liable to any damages by connecting to incompatible hardware!
  • Always use original Vision Components FPC cable!
  • FPC cable can only be used in one direction!
FPC Cable Signals
Signal Sensor module Processor board (22 Pin) Processor board (15 Pin)
Type   regular inverse regular
GND 1 22 1 1
CSI_DAT0_N 2 21 2 2
CSI_DAT0_P 3 20 3 3
GND 4 19 4 4
CSI_DAT1_N 5 18 5 5
CSI_DAT1_P 6 17 6 6
GND 7 16 7 7
CSI_CLK_N 8 15 8 8
CSI_CLK_P 9 14 9 9
GND 10 13 10 10
CSI_DAT2_N 11 12 11
CSI_DAT2_P 12 11 12
GND 13 10 13
CSI_DAT3_N 14 9 14
CSI_DAT3_P 15 8 15
GND 16 7 16
trigger_to_sensor [1] [2] 17 6 17 11
flash_from_sensor [1] [2] 18 5 18 12
GND 19 4 19
I2C_SCL [2] 20 3 20 13
I2C_SDA [2] 21 2 21 14
Vcc3V3 22 1 22 15
[1](1, 2) not supported for all sensor modules.
[2](1, 2, 3, 4) signals have 3.3V LVTTL/LVCMOS level.

1.4.1.1   FPC Cable End to MIPI Camera Sensor

./images/mipi_sensor_backside.png

Pin 1 of MIPI Module FPC Cable

./images/mipi_sensor_backside_with_FPC.png

Correct Insertion of FPC Cable, Sensor Side

1.4.1.2   FPC Cable End to Processor Board

./images/mipi_raspberry_3b_Plus.png

Correct Insertion of FPC Cable, Raspberry Pi 3B+

./images/mipi_raspberry_3_cmio.png

Correct Insertion of FPC Cable, Raspberry Pi CMIO 3.0

./images/mipi_raspberry_zero_1.3.png

Correct Insertion of FPC Cable, Raspberry Pi zero 1.3

1.5   MIPI Sensor Module Circuit

./images/mipi_circuit.png

Overview of the MIPI Sensor Module Circuit

2   Accessories

2.1   MIPI camera module order numbers

VC MIPI camera modules
Order Number Product ID Service description
EK003222 VC MIPI IMX296 monochrome
EK003313 VC MIPI OV9281 monochrome
n/a VC MIPI IMX274 C color
n/a VC MIPI IMX290 monochrome
n/a VC MIPI IMX326 C color
n/a VC MIPI IMX327 C color
n/a VC MIPI IMX412 C color

2.2   Order numbers of MIPI camera module Accessories

Accessories for VC MIPI camera modules
Order Number Product / Service description
EK003260 Flexible Printed Circuit (FPC) Cable: 22 to 22 Pin
EK003261 Flexible Printed Circuit (FPC) Cable: 22 to 15 Pin
VK000057 S-Mount Lens Holder (Plastic )
n/a S-Mount Lens Holder (Aluminum)
n/a CS-/C-Mount Lens Holder
VK001257 Lens, S-Mount Color (IR Filter), f=3.6
VK000021 Lens, S-Mount Monochrome, f=8.0
EK000625 IR Filter Glass, d=16mm
EK003348 Adap96 MIPI Adapter for '96' Boards
./images/VK000057_h180.png

S-Mount Lens Holder (Plastic )

./images/VK000400_h180.png

CS-/C-Mount Lens Holder

3   Raspberry Pi 3B+ Reference Software Setup

Before attaching the MIPI sensor, be sure that the RaspberryPi system works correctly without the new camera sensor.

These are the steps to get the sensor working:

  1. Download and install the Operating System.
  2. Install (or compile and install) the kernel device tree overlay fitting to your sensor type.
  3. Install (or compile and install) the sensor driver as kernel module.
  4. Change the system setup to use the newly installed software.
  5. Run the test code to get an image.

3.2   Driver Acquisition and Setup

You can download the binaries with sources from the website:

www.mipi-modules.com

The following text passages contain a placeholder symbol named 'PLACEHOLDER' which should be read as being replaced by a string depending on your sensor's name as listed at the table:

PLACEHOLDER values depending on the Sensor
Sensor Name PLACEHOLDER replacement text
VC MIPI IMX296 vc_mipi_imx296
VC MIPI OV9281 vc_mipi_ov9281
VC MIPI IMX274 C vc_mipi_imx274c
VC MIPI IMX290 vc_mipi_imx290
VC MIPI IMX326 C vc_mipi_imx326c
VC MIPI IMX327 C vc_mipi_imx327c
VC MIPI IMX412 C vc_mipi_imx412c

3.2.1   Compiling the device tree overlay

You can try the binary blob of the device tree first if it matches your kernel; see step 3 of the compile instructions to know where to put it.

However here are the steps to compile the device tree by yourself:

The dowload includes the device tree source file named

vc_mipi_imx296-overlay.dts
  1. Install the 'device-tree-compiler' package via 'sudo apt-get install device-tree-compiler'

  2. Compile the dtbo binary kernel device tree overlay by using the following command:

    dtc -@ -I dts -O dtb -o PLACEHOLDER.dtbo PLACEHOLDER-overlay.dts

  3. Copy the binary 'PLACEHOLDER.dtbo' to '/boot/overlays/PLACEHOLDER.dtbo'.

3.2.2   Compiling the sensor module driver (kernel module)

You can try the binary blob of the sensor module driver first if it matches your kernel; see step 7 of the compile instructions to know where to put it.

However here are the steps to compile the device tree by yourself:

The dowload includes the device tree source files named

  1. Update the 'raspberrypi-kernel' package and your system via 'sudo apt-get update && apt-get upgrade'

  2. Reboot.

  3. Install the 'raspberrypi-kernel-headers' package via 'sudo apt-get install raspberrypi-kernel-headers'

  4. Test if the version of the running kernel matches the version of the kernel headers, the following command should show the directory for compiling the sensor module kernel module driver:

    ls "/usr/src/linux-headers-$(uname -r)"

  5. If the test was successful, extract the sensor module kernel driver, e.g. '/usr/src/', and change to the directory.

  6. The source directory contains a Makefile to compile the driver. Do so by calling:

    make clean all

  7. Calling make the following way will install the kernel module in your current kernel module path '/lib/modules/.../kernel/drivers/media/i2c/':

    make install

3.3   Raspberry Pi System Setup

Raspberry Pi first boots a real-time operating system (RTOS) on the GPU. This RTOS looks into the file '/boot/config.txt'. It loads a default Kernel device tree and patches it by overlaying the device tree parts listed by the 'dtoverlay' entries at the file '/boot/config.txt'.

Next, with this generated customized kernel device tree, the linux kernel will be booted.

To check the behaviour of the RTOS one can look at the output by the following command

sudo vcdbg log msg

3.3.1   Reserving Contiguous Memory for the Image Captures

In contrast to the normally used non-contiguous memory the capture hardware needs a contiguous memory region to transfer pixel data to by using direct memory access (DMA).

To reserve 128MByte of memory for capturing images, edit the file named

/boot/cmdline.txt

and append the following key-value-pair to the current line:

cma=128M

3.3.2   Modifying the '/boot/config.txt'

The Raspberry Pi provides an easy system configuration via the text file named

/boot/config.txt

If it is not present, create it and add the following entries:

dtparam=i2c_vc=on

dtoverlay=PLACEHOLDER

Selected Parameters of the /boot/config.txt
Parameter Name Meaning
dtparam=i2c_vc=on Changes the physical I²C bus 0 accessor from the default, GPU, to the CPU, since the MIPI sensor communicates over the I²C bus with the CPU kernel driver.
dtoverlay=PLACEHOLDER

Tells the capture driver and the RTOS which sensor is connected to the system:

  • The device tree will be modified by the overlay at '/boot/overlays/PLACEHOLDER.dtbo', and
  • the capture driver of '/dev/video0' uses the sensor driver module at '/lib/.../module/PLACEHOLDER.ko'.

Reboot.

3.4   First Image Acquisition Test

Connect the sensor to the CPU board as described in this document (ensure the power is turned off during the establishment of the connection between the sensor module and the CPU board).

The sensor device should be listed

v4l2-ctl --all

The following command dumps sensor data.

v4l2-ctl --stream-mmap --stream-count=-1 -d /dev/video0 --stream-to=/dev/null

If it works properly, a kernel message like the following will be generated for debugging:

dmesg | grep unicam

Example

Example Sign Sample output of the command 'dmesg | grep unicam' using a VC MIPI IMX296 module:

[ 5.887447] unicam 3f801000.csi1: Using sensor vc_mipi_imx296 0-001a for capture
[ 5.887453] unicam 3f801000.csi1: Get supported formats...
[ 5.887460] unicam 3f801000.csi1: subdev vc_mipi_imx296 0-001a: code: 300f idx: 0
[ 5.887470] unicam 3f801000.csi1: fmt 300f returned as 7f1c46c4, V4L2 FOURCC 41415270, csi_dt 2B
[ 5.887478] unicam 3f801000.csi1: matched fourcc: pRAA: code: 300f idx: 0
[ 5.887484] unicam 3f801000.csi1: subdev->enum_mbus_code idx 1 returned -22 - continue
[ 5.887487] unicam 3f801000.csi1: Done all formats
[ 5.887496] unicam 3f801000.csi1: active_fmt[0] (af1a8548) is code 300f, fourcc pRAA, depth 10
[ 5.887564] unicam 3f801000.csi1: __subdev_get_format 1440x1080 code:300f
[ 5.887572] unicam 3f801000.csi1: __subdev_get_format 1440x1080 code:300f
[ 5.887583] unicam 3f801000.csi1: unicam_calc_format_size_bpl: fourcc: pRAA size: 1440x1080 bpl:1808 img_size:1967104

Look at demo code at the homepage for more information about how to capture an image.

4   Appendix A: Dimensions MIPI camera module

./images/mipi_sensor_measures.png

Dimensions of the MIPI Sensor Module