forked from logzhan/RobotKernal-UESTC
222 lines
7.0 KiB
Plaintext
222 lines
7.0 KiB
Plaintext
/** @mainpage API Reference
|
|
|
|
Older versions: [0.1](0.1).
|
|
|
|
Introduction
|
|
============
|
|
|
|
%libfreenect2 is an open source cross-platform driver for Kinect for Windows v2
|
|
devices. For information on installation and troubleshooting, see the
|
|
[GitHub repository](https://github.com/OpenKinect/libfreenect2).
|
|
|
|
This documentation is designed for application developers who want to extract
|
|
and use depth and color images from Kinect v2 for further processing.
|
|
Additional questions and comments not covered by this documentation can be
|
|
posted to [GitHub issues](https://github.com/OpenKinect/libfreenect2/issues).
|
|
|
|
This documentation may require some understanding on camera calibration and 3-D
|
|
geometry.
|
|
|
|
Features
|
|
========
|
|
|
|
- Color image processing
|
|
- IR and depth image processing
|
|
- Registration of color and depth images
|
|
- Multiple GPU and hardware acceleration implementations for image processing
|
|
|
|
### Issues and Future Work
|
|
|
|
- Audio. Raw audio is accessible via Linux USB audio. There is no support for
|
|
the calibrated directional audio.
|
|
- Unstable USB and crashes. Due to differences in driver support, USB problems
|
|
can happen a lot. Error handling in %libfreenect2 is not fully verified for
|
|
production use.
|
|
- Firmware upload. The protocol has been reverse engineered, but use Windows
|
|
for this right now.
|
|
- Example of multiple Kinects.
|
|
- Example utility of dumping image frames.
|
|
- API for on-demand processing.
|
|
- Verification of systematic errors through accurate calibration.
|
|
- Bindings for C, Python, Java, etc.
|
|
|
|
Getting Started
|
|
===============
|
|
|
|
To read the API documentation, start with the [Modules](modules.html) page
|
|
which nicely organizes classes according to their functionalities.
|
|
|
|
Example programs can be found in the source distribution under the `examples`
|
|
directory. There also includes an example CMake build system for a standalone
|
|
application that uses %libfreenect2 binary installation.
|
|
|
|
Many internal details are hidden from this public API. For details on Kinect
|
|
v2's USB protocols, depth decoding algorithms, calibration algorithms, and how
|
|
to implement performance optimizers, you are encouraged to read the source
|
|
code. The source code is the updated and authoritative reference for any
|
|
functionalities.
|
|
|
|
Environment Variables
|
|
=====================
|
|
|
|
There are a few environment variables providing controls for both end-users and
|
|
programmers:
|
|
|
|
* `LIBFREENECT2_LOGGER_LEVEL`: The default logging level if not explicitly set
|
|
by the code.
|
|
* `LIBFREENECT2_PIPELINE`: The default pipeline if not explicitly set by the
|
|
code.
|
|
* `LIBFREENECT2_RGB_TRANSFER_SIZE`, `LIBFREENECT2_RGB_TRANSFERS`,
|
|
`LIBFREENECT2_IR_PACKETS`, `LIBFREENECT2_IR_TRANSFERS`: Tuning the USB buffer
|
|
sizes. Use only if you know what you are doing.
|
|
|
|
You can also see the following walkthrough for the most basic usage.
|
|
|
|
Walkthrough
|
|
===========
|
|
|
|
Here is an example to walk you through the API. See `examples/Protonect.cpp`
|
|
for the full source.
|
|
|
|
Headers
|
|
-------
|
|
|
|
First, include necessary headers. `registration.h` and `logger.h` are optional
|
|
if you don't use them.
|
|
|
|
@snippet Protonect.cpp headers
|
|
|
|
Logging
|
|
-------
|
|
|
|
This shows how to set up the logger and logging level.
|
|
|
|
@snippet Protonect.cpp logging
|
|
|
|
Though @copydetails libfreenect2::createConsoleLoggerWithDefaultLevel
|
|
|
|
You can implement a custom [Logger](@ref libfreenect2::Logger) and redirect
|
|
%libfreenect2's log messages to desired places.
|
|
|
|
Here is an example to save log messages to a file.
|
|
|
|
@snippet Protonect.cpp logger
|
|
|
|
And use it
|
|
|
|
@snippet Protonect.cpp file logging
|
|
|
|
%libfreenect2 uses a single global logger regardless of number of contexts and
|
|
devices. You may have to implement thread safety measure in
|
|
[log()](@ref libfreenect2::Logger::log), which is called from multiple threads.
|
|
Console loggers are thread safe because `std::cout` and `std::cerr` are thread
|
|
safe.
|
|
|
|
Initialize and Discover Devices
|
|
-------------------------------
|
|
|
|
You need these structures for all operations. Here it uses only one device.
|
|
|
|
@snippet Protonect.cpp context
|
|
|
|
You must enumerate all Kinect v2 devices before doing anything else related to
|
|
devices.
|
|
|
|
@snippet Protonect.cpp discovery
|
|
|
|
Also, you can create a specific [PacketPipeline](@ref libfreenect2::PacketPipeline)
|
|
instead using the default one for opening the device. Alternatives include
|
|
[OpenGLPacketPipeline](@ref libfreenect2::OpenGLPacketPipeline),
|
|
[OpenCLPacketPipeline](@ref libfreenect2::OpenCLPacketPipeline), etc.
|
|
|
|
@snippet Protonect.cpp pipeline
|
|
|
|
Open and Configure the Device
|
|
-----------------------------
|
|
|
|
Now you can open the device by its serial number, and using the specific
|
|
pipeline.
|
|
|
|
@snippet Protonect.cpp open
|
|
|
|
You can also open the device without providing a pipeline, then a default is
|
|
used. There are a few alternative ways to [openDevice()](@ref libfreenect2::Freenect2::openDevice).
|
|
|
|
After opening, you need to attach [Framelisteners](@ref libfreenect2::FrameListener)
|
|
to the device to receive images frames.
|
|
|
|
This [SyncMultiFrameListener](@ref libfreenect2::SyncMultiFrameListener) will
|
|
wait until all specified types of frames are received once. Like loggers, you
|
|
may also implement your own frame listeners using the same interface.
|
|
|
|
@snippet Protonect.cpp listeners
|
|
|
|
You cannot configure the device after starting it.
|
|
|
|
Start the Device
|
|
----------------
|
|
|
|
After finishing configuring the device, you can start the device. You must
|
|
start the device before querying any information of the device.
|
|
|
|
@snippet Protonect.cpp start
|
|
|
|
You can [setIrCameraParams()](@ref libfreenect2::Freenect2Device::setIrCameraParams)
|
|
after start if you have your own depth calibration parameters.
|
|
|
|
Otherwise you can also use the factory preset parameters for
|
|
[Registration](@ref libfreenect2::Registration). You can also provide your own
|
|
depth calibration parameterss (though not color camera calibration parameters
|
|
right now). Registration is optional.
|
|
|
|
@snippet Protonect.cpp registration setup
|
|
|
|
At this time, the processing has begun, and the data flows through the pipeline
|
|
towards your frame listeners.
|
|
|
|
Receive Image Frames
|
|
--------------------
|
|
|
|
This example uses a loop to receive image frames.
|
|
|
|
@snippet Protonect.cpp loop start
|
|
|
|
[waitForNewFrame()](@ref libfreenect2::SyncMultiFrameListener::waitForNewFrame)
|
|
here will block until required frames are all received, and then you can
|
|
extract `Frame` according to the type.
|
|
|
|
See libfreenect2::Frame for details about pixel format, dimensions, and
|
|
metadata.
|
|
|
|
You can do your own things using the frame data. You can feed it to OpenCV,
|
|
PCL, etc. Here, you can perform registration:
|
|
|
|
@snippet Protonect.cpp registration
|
|
|
|
After you are done with this frame, you must release it.
|
|
|
|
@snippet Protonect.cpp loop end
|
|
|
|
Stop the Device
|
|
---------------
|
|
|
|
If you are finished and no longer need to receive more frames, you can stop
|
|
the device and exit.
|
|
|
|
@snippet Protonect.cpp stop
|
|
|
|
Pause the Device
|
|
----------------
|
|
|
|
You can also temporarily pause the device with
|
|
[stop()](@ref libfreenect2::Freenect2Device::stop) and
|
|
[start()](@ref libfreenect2::Freenect2Device::start).
|
|
|
|
@snippet Protonect.cpp pause
|
|
|
|
Doing this during `waitForNewFrame()` should be thread safe, and tests also
|
|
show well. But a guarantee of thread safety has not been checked yet.
|
|
|
|
THE END.
|
|
*/
|