Bytedeco makes native libraries available to the Java platform by offering ready-to-use bindings generated with the codeveloped JavaCPP technology. This, we hope, is the missing bridge between Java and C/C++, bringing compute-intensive science, multimedia, computer vision, deep learning, etc to the Java platform.
Core Technologies
- JavaCPP [API] – A tool that can not only generate JNI code but also build native wrapper library files from an appropriate interface file written entirely in Java. It can also parse automatically C/C++ header files to produce the required Java interface files.
Prebuilt Java Bindings to C/C++ Libraries
These are part of a project that we call the JavaCPP Presets. Many coexist in the same GitHub repository, and all use JavaCPP to wrap predefined C/C++ libraries from open-source land. The bindings expose almost all of the relevant APIs and make them available in a portable and user-friendly fashion to any Java virtual machine (including Android), as if they were like any other normal Java libraries. We have presets for the following C/C++ libraries:- OpenCV – [sample usage] [API] – More than 2500 optimized computer vision and machine learning algorithms
- FFmpeg – [sample usage] [API] – A complete, cross-platform solution to record, convert and stream audio and video
- FlyCapture – [sample usage] [API] – Image acquisition and camera control software from PGR
- Spinnaker – [sample usage] [API] – Image acquisition and camera control software from FLIR
- libdc1394 – [sample usage] [API] – A high-level API for DCAM/IIDC cameras
- OpenKinect – [sample usage] [API] [API 2] – Open source library to use Kinect for Xbox and for Windows sensors
- librealsense – [sample usage] [API] [API 2] – Cross-platform library for Intel RealSense depth and tracking cameras
- videoInput – [sample usage] [API] – A free Windows video capture library
- ARToolKitPlus – [sample usage] [API] – Marker-based augmented reality tracking library
- Chilitags – [sample usage] [API] – Robust fiducial markers for augmented reality and robotics
- flandmark – [sample usage] [API] – Open-source implementation of facial landmark detector
- Arrow – [sample usage] [API] – A cross-language development platform for in-memory data
- HDF5 – [sample usage] [API] – Makes possible the management of extremely large and complex data collections
- Hyperscan – [sample usage] [API] – High-performance regular expression matching library
- LZ4 – [sample usage] [API] – Extremely fast compression algorithm
- MKL – [sample usage] [API] – The fastest and most-used math library for Intel-based systems
- oneDNN – [sample usage] [API] [API 2] – Intel Math Kernel Library for Deep Neural Networks (DNNL)
- OpenBLAS – [sample usage] [API] – An optimized BLAS library based on GotoBLAS2 1.13 BSD version, plus LAPACK
- ARPACK-NG – [sample usage] [API] – Collection of subroutines designed to solve large scale eigenvalue problems
- CMINPACK – [sample usage] [API] – For solving nonlinear equations and nonlinear least squares problems
- FFTW – [sample usage] [API] – Fast computing of the discrete Fourier transform (DFT) in one or more dimensions
- GSL – [sample usage] [API] – The GNU Scientific Library, a numerical library for C and C++ programmers
- CPython – [sample usage] [API] – The standard runtime of the Python programming language
- NumPy – [sample usage] [API] – Base N-dimensional array package
- SciPy – [sample usage] [API] – Fundamental library for scientific computing
- Gym – [sample usage] [API] – A toolkit for developing and comparing reinforcement learning algorithms
- LLVM – [sample usage] [API] – A collection of modular and reusable compiler and toolchain technologies
- libffi – [sample usage] [API] – A portable foreign-function interface library
- libpostal – [sample usage] [API] – For parsing/normalizing street addresses around the world
- LibRaw – [sample usage] [API] – A simple and unified interface for RAW files generated by digital photo cameras
- Leptonica – [sample usage] [API] – Software useful for image processing and image analysis applications
- Tesseract – [sample usage] [API] – Probably the most accurate open source OCR engine available
- Caffe – [sample usage] [API] – A fast open framework for deep learning
- OpenPose – [sample usage] [API] – Real-time multi-person keypoint detection for body, face, hands, and foot estimation
- CUDA – [sample usage] [API] – Arguably the most popular parallel computing platform for GPUs
- NVIDIA Video Codec SDK – [sample usage] [API] – An API for hardware accelerated video encode and decode
- OpenCL – [sample usage] [API] – Open standard for parallel programming of heterogeneous systems
- MXNet – [sample usage] [API] – Flexible and efficient library for deep learning
- PyTorch – [sample usage] [API] – Tensors and dynamic neural networks with strong GPU acceleration
- SentencePiece – [sample usage] [API] – Unsupervised text tokenizer for neural-network-based text generation
- TensorFlow – [sample usage] [API] – Computation using data flow graphs for scalable machine learning
- TensorFlow Lite – [sample usage] [API] – An open source deep learning framework for on-device inference
- TensorRT – [sample usage] [API] – High-performance deep learning inference optimizer and runtime
- Triton Inference Server – [sample usage] [API] – An optimized cloud and edge inferencing solution
- ALE – [sample usage] [API] – The Arcade Learning Environment to develop AI agents for Atari 2600 games
- DepthAI – [sample usage] [API] – An embedded spatial AI platform built around Intel Myriad X
- ONNX – [sample usage] [API] – Open Neural Network Exchange, an open source format for AI models
- nGraph – [sample usage] [API] – An open source C++ library, compiler, and runtime for deep learning frameworks
- ONNX Runtime – [sample usage] [API] – Cross-platform, high performance scoring engine for ML models
- TVM – [sample usage] [API] – An end to end machine learning compiler framework for CPUs, GPUs and accelerators
- Bullet Physics SDK – [sample usage] [API] – Real-time collision detection and multi-physics simulation
- LiquidFun – [sample usage] [API] – 2D physics engine for games
- Qt – [sample usage] [API] – A cross-platform framework that is usually used as a graphical toolkit
- Skia – [sample usage] [API] – A complete 2D graphic library for drawing text, geometries, and images
- cpu_features – [sample usage] [API] – A cross platform C99 library to get cpu features at runtime
- ModSecurity – [sample usage] [API] – A cross platform web application firewall (WAF) engine for Apache, IIS and Nginx
- Systems – [sample usage] [API] – To call native functions of operating systems (glibc, XNU libc, Win32, etc)
- Add here your favorite C/C++ library, for example: Caffe2, OpenNI, OpenMesh, PCL, etc. Read about how to do that.
We will add more to this list as they are made, including those from outside the bytedeco/javacpp-presets repository.
Projects Leveraging the Presets Bindings
- JavaCV [API] – Library based on the JavaCPP Presets that depends on commonly used native libraries in the field of computer vision to facilitate the development of those applications on the Java platform. It provides easy-to-use interfaces to grab frames from cameras and audio/video streams, process them, and record them back on disk or send them over the network.
- JavaCV Examples – Collection of examples originally written in C++ for the book entitled OpenCV 2 Computer Vision Application Programming Cookbook by Robert Laganière, but ported to JavaCV and written in Scala.
- ProCamCalib – Sample JavaCV application that can perform geometric and photometric calibration of a set of video projectors and color cameras.
- ProCamTracker – Another sample JavaCV application that uses the calibration from ProCamCalib to implement a vision method that tracks a textured planar surface and realizes markerless interactive augmented reality with projection mapping.
More Project Information
Please refer to the contribute and download pages for more information about how to help out or obtain this software.
See the developer site on GitHub for more general information about the Bytedeco projects.
Latest News
Beyond Java and C++
We are happy to announce the first release of Bytedeco in this new Japanese era! Version 1.5 is available for JavaCPP, JavaCPP Presets, JavaCV, ProCamCalib, and ProCamTracker. This release comes with new presets for NumPy (yes, the Python library, more on that below), NCCL, nGraph, Qt (providing an alternative to AWT, Swing, and JavaFX), and cpu_features, as well as updates for FFmpeg (now also including the ffmpeg and ffprobe programs themselves), libfreenect, HDF5, MKL, MKL-DNN, LLVM, Leptonica, ARPACK-NG, CUDA, cuDNN, MXNet, TensorFlow, TensorRT, ONNX, LiquidFun, and Skia. Many thanks to all contributors! Since the previous post just 3 months ago, according to git shortlog 1.4.4..1.5 --summary
, they were:
Alex Merritt, Arnaud Jeansen, Greg Hart, HGuillemet, k3rnL, Samuel Audet, Simon Harris
However, as usual, this list does not contain anyone who very generously helped fix CI settings, test builds, debug code, file suggestions, make corrections, report bugs, propose ideas, update wiki pages, etc. One can find more detailed information about all these changes and fixes on the repositories listed above in the CHANGELOG.md
files, among other places.
As hinted previously though, the most important update concerns the modularization of all the artifacts to comply with the Java Platform Module System (JPMS). In theory, we simply have to add irritating module-info.java
files everywhere (we are grateful to ModiTect for the help with that), but one major restriction of JPMS is that we cannot split a Java package across multiple modules, so we had to update the packages of all the presets to comply, thus the minor version bump from 1.4.x to 1.5.x. In no small part thanks to Hervé Guillemet, we eventually managed to come up with a satisfactory naming scheme that not only complies with JPMS, but also produces top-level classes that are closer to typical Java APIs and that Javadoc can render more clearly into HTML. Instead of putting all classes in the org.bytedeco.javacpp
package, all modules now have their own packages based on the names of their artifacts, for example, org.bytedeco.opencv
in the case of OpenCV. Inside such a package we have top-level classes, further divided into subpackages when the artifact contains multiples native libraries, as well as a global
subpackage containing, you guessed it, classes with native declarations for global C/C++ functions and variables. For most downstream projects, upgrading requires modifications to the import statements only. In the case of OpenCV and TensorFlow, for example, we would typically have statements like the following:
import static org.bytedeco.javacpp.opencv_core.*;
import static org.bytedeco.javacpp.opencv_imgproc.*;
import static org.bytedeco.javacpp.opencv_calib3d.*;
import static org.bytedeco.javacpp.opencv_objdetect.*
import static org.bytedeco.javacpp.tensorflow.*;
Given these exact statements, to migrate to the new API, the only change required would be to replace them with the following:
import org.bytedeco.opencv.opencv_core.*;
import org.bytedeco.opencv.opencv_imgproc.*;
import org.bytedeco.opencv.opencv_calib3d.*;
import org.bytedeco.opencv.opencv_objdetect.*;
import org.bytedeco.tensorflow.*;
import static org.bytedeco.opencv.global.opencv_core.*;
import static org.bytedeco.opencv.global.opencv_imgproc.*;
import static org.bytedeco.opencv.global.opencv_calib3d.*;
import static org.bytedeco.opencv.global.opencv_objdetect.*;
import static org.bytedeco.tensorflow.global.tensorflow.*;
To prevent conflicts with the old API, we also changed the groupId
of all artifacts to org.bytedeco
, so no more annoyingly long names containing org.bytedeco.javacpp-presets
, which also reflects our evolving technical understanding of the world where the JVM cannot rule them all.
Java is able to abstract away the underlying operating system, and, along with tools such as Maven and now jlink to create compact self-contained standalone applications as per the opencv-stitching-jlink sample project, its abilities to manage the complexity of software in the ecosystem is unparalleled. However, the JVM itself is failing miserably at supporting other existing programming languages. We obviously need access to native libraries written in C/C++ for compute intensive algorithms, but even popular scripting languages such as JavaScript and Python have never been able to feel at home on the JVM either, notwithstanding their own arguably inferior native runtimes. Jython is stuck at Python 2.7, while Nashorn has been deprecated even before its proposed successor, Graal JavaScript, is ready to take its place. Likewise, Graal Python is truly awesome technology, but is still an “early-stage experimental implementation” that runs pretty much nothing useful yet, let alone CUDA or any kind of support for GPU no where to be found on any roadmap. Whatever the cause is for the lack of the appearance of a universal VM that a majority of users would be happy with, it is instead possible to have multiple runtimes executing in the same process to share data efficiently, an approach that is also future-proof given that Graal still needs to implement JNI and the C API of Python anyway to be compatible with tools such as JNA, JNR, or JavaCPP, and libraries like NumPy or TensorFlow. We show that an initial implementation can be easily achieved today in the case of CPython and the current JDK, starting with the JavaCPP Presets for NumPy, which leaves all the high-level dependency management to Java and Maven, where JavaCPP automatically extracts the Python package to its cache, along with MKL on supported platforms, in a portable and repeatable fashion, without the need of any other less portable tools including Docker. In effect, this makes Python and NumPy available as a sort of domain-specific scripting language to Java developers. We plan on making more Python packages available this way, continuing next with the JavaCPP Presets for TensorFlow.
If you would like to participate in this effort, please communicate via the mailing list from Google Groups, issues on GitHub, or the chat room at Gitter. In any case, stay tuned for more exciting developments later this year!