How to configure Kinect v2 camera

# How to configure Kinect v2 ## Install libfreenect2 - https://github.com/OpenKinect/libfreenect2 - Check your Environment ```sh yubao@yubao-Z370M-S01:~/GoogleDrive$ uname -a Linux yubao-Z370M-S01 4.15.0-42-generic 45~16.04.1-Ubuntu SMP Mon Nov 19 13:02:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ``` Install some requirements: ```sh sudo apt-get install build-essential cmake pkg-config sudo apt-get install libusb-1.0-0-dev sudo apt-get install libturbojpeg libjpeg-turbo8-dev sudo apt-get install libglfw3-dev ``` Optional ```sh sudo apt-get install beignet-dev sudo apt-get install libopenni2-dev ``` Intel only ```sh sudo apt-get install libva-dev libjpeg-dev ``` ## Compile from source code ```sh git clone https://github.com/OpenKinect/libfreenect2.git cd libfreenect2 mkdir build && cd build cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/freenect2 make make install ``` ## Set up udev rules for device access: ```sh sudo cp ../platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/ ``` then replug the Kinect. It will show permission not allowed error if not do this step. ## Test Kinect2 ```sh (base) yubao@yubao-Z370M-S01:~/Software/libfreenect2/build$ ./bin/Protonect Version: 0.2.0 Environment variables: LOGFILE= Usage: ./bin/Protonect [-gpu=] [gl | cl | clkde | cuda | cudakde | cpu] [] [-noviewer] [-norgb | -nodepth] [-help] [-version] [-frames ] To pause and unpause: pkill -USR1 Protonect [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 8 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @2:3 with serial 006696152647 [Info] [Freenect2Impl] found 1 devices libva info: VA-API version 0.39.0 libva info: va_getDriverName() returns -1 libva error: va_getDriverName() failed with unknown libva error,driver_name=(null) [Error] [VaapiRgbPacketProcessorImpl] vaInitialize(display, &major_ver, &minor_ver): unknown libva error [Info] [Freenect2DeviceImpl] opening... [Info] [Freenect2DeviceImpl] transfer pool sizes rgb: 20*16384 ir: 60*8*33792 ``` *Result:* ## Install iai_kinect2 for ROS [iai_kinect2-Github](https://github.com/code-iai/iai_kinect2) ```sh cd ~/catkin_ws/src/ git clone https://github.com/code-iai/iai_kinect2.git cd iai_kinect2 rosdep install -r --from-paths . cd ~/catkin_ws catkin_make -DCMAKE_BUILD_TYPE="Release" ``` Connect your sensor and run kinect2_bridge: ```sh roslaunch kinect2_bridge kinect2_bridge.launch ``` Image_view ```sh rosrun image_view image_view image:=/kinect2/qhd/image_color ``` ## Kinect2 Calibration - [code-iai: iai_kinect2](https://github.com/code-iai/iai_kinect2/tree/master/kinect2_calibration) - [kinect2的标定](https://www.cnblogs.com/li-yao7758258/p/7445429.html) - [openni_launchTutorialsIntrinsicCalibration](http://wiki.ros.org/openni_launch/Tutorials/IntrinsicCalibration) - [Ros Simulation - iai_kinect2](https://github.com/code-iai/iai_kinect2) ## Possible Errors ### kinect run error: Wrong JPEG library version It tooks me one night to solve this error: - Error description ```sh yubao@yubao-Z370M-S01:~/Software/libfreenect2$ ./build/bin/Protonect Version: 0.2.0 Environment variables: LOGFILE= Usage: ./build/bin/Protonect [-gpu=] [gl | cl | clkde | cuda | cudakde | cpu] [] [-noviewer] [-norgb | -nodepth] [-help] [-version] [-frames ] To pause and unpause: pkill -USR1 Protonect [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 8 usb devices connected [Info] [Freenect2Impl] found valid Kinect v2 @2:3 with serial 006696152647 [Info] [Freenect2Impl] found 1 devices Wrong JPEG library version: library is 90, caller expects 80 ``` Solution Refer: https://github.com/OpenKinect/libfreenect2/issues/1004 ```sh conda install jpeg=8 ``` Remember to rebuild libfreenect2. ### nvidia Jetson xavier ```sh nvidia@xavier:~/catkin_ws/src/libfreenect2/build$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/freenect2 -- using tinythread as threading library -- Checking for modules 'libva;libva-drm' -- Found libva, version 1.1.0 -- Found libva-drm, version 1.1.0 CMake Warning at cmake_modules/FindTegraJPEG.cmake:17 (MESSAGE): Tegra drivers have wrong checksum: /usr/lib/aarch64-linux-gnu/libv4lconvert.so: FAILED Call Stack (most recent call first): CMakeLists.txt:220 (FIND_PACKAGE) -- Found Linux4Tegra 32.1.0 CMake Warning at cmake_modules/FindTegraJPEG.cmake:57 (MESSAGE): Linux4Tegra version (32.1.0) is not recognized. Add the new source URL part to FindTegraJPEG.cmake. Call Stack (most recent call first): CMakeLists.txt:220 (FIND_PACKAGE) -- Could NOT find TegraJPEG (missing: TegraJPEG_INCLUDE_DIRS TegraJPEG_L4T_OK TegraJPEG_DRIVER_OK TegraJPEG_WORKS) CMake Error at /usr/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:137 (message): Could NOT find TurboJPEG (missing: TurboJPEG_INCLUDE_DIRS TURBOJPEG_WORKS) Call Stack (most recent call first): /usr/share/cmake-3.10/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE) cmake_modules/FindTurboJPEG.cmake:66 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) CMakeLists.txt:243 (FIND_PACKAGE) -- Configuring incomplete, errors occurred! See also "/home/nvidia/catkin_ws/src/libfreenect2/build/CMakeFiles/CMakeOutput.log". ``` Solution: ```sh sudo apt install libturbojpeg* ``` ## no Kinect2 devices found! LIBUSB_ERROR_ACCESS Access denied command ```sh rosrun kinect2_bridge kinect2_bridge ``` Error message: ```sh [Info] [OpenCLDepthPacketProcessorImpl] devices: [Info] [OpenCLDepthPacketProcessorImpl] 0: Intel(R) HD Graphics Haswell Ultrabook GT2 Mobile (GPU)[Intel] [Info] [OpenCLDepthPacketProcessorImpl] selected device: Intel(R) HD Graphics Haswell Ultrabook GT2 Mobile (GPU)[Intel] [Info] [OpenCLDepthPacketProcessorImpl] building OpenCL program... [Info] [OpenCLDepthPacketProcessorImpl] OpenCL program built successfully [Info] [Freenect2Impl] enumerating devices... [Info] [Freenect2Impl] 12 usb devices connected [Error] [Freenect2Impl] failed to open Kinect v2: @3:5 LIBUSB_ERROR_ACCESS Access denied (insufficient permissions) [Info] [Freenect2Impl] found 0 devices [ERROR] [Kinect2Bridge::initDevice] no Kinect2 devices found! [ERROR] [Kinect2Bridge::start] Initialization failed! ``` Solution: ```sh sudo cp ../platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/ ``` ## Could NOT find TegraJPEG (missing: TegraJPEG_LIBRARIES TegraJPEG_INCLUDE_DIRS TegraJPEG_L4T_OK TegraJPEG_DRIVER_OK TegraJPEG_WORKS) Problem: ```sh -- Found JPEG: /usr/lib/x86_64-linux-gnu/libjpeg.so (found version "80") -- Could NOT find TegraJPEG (missing: TegraJPEG_INCLUDE_DIRS TegraJPEG_WORKS) CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message): Could NOT find TurboJPEG (missing: TurboJPEG_INCLUDE_DIRS TURBOJPEG_WORKS) Call Stack (most recent call first): /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE) cmake_modules/FindTurboJPEG.cmake:66 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) CMakeLists.txt:243 (FIND_PACKAGE) ``` Solution: ```sh sudo apt-get install libturbojpeg0-dev ``` ## Reference - [Mahsa mahsaparsapour](https://mahsaparsapour.wordpress.com/tutorials/kinect-2/)