Bytedeco makes native libraries available to the Java platform by offering ready-to-use bindings generated with the codeveloped JavaCPP technology. This, we hope, is the missing bridge between Java and C/C++, bringing compute-intensive science, multimedia, computer vision, deep learning, etc to the Java platform.
Core Technologies
- JavaCPP [API] – A tool that can not only generate JNI code but also build native wrapper library files from an appropriate interface file written entirely in Java. It can also parse automatically C/C++ header files to produce the required Java interface files.
Prebuilt Java Bindings to C/C++ Libraries
These are part of a project that we call the JavaCPP Presets. Many coexist in the same GitHub repository, and all use JavaCPP to wrap predefined C/C++ libraries from open-source land. The bindings expose almost all of the relevant APIs and make them available in a portable and user-friendly fashion to any Java virtual machine (including Android), as if they were like any other normal Java libraries. We have presets for the following C/C++ libraries:- OpenCV – [sample usage] [API] – More than 2500 optimized computer vision and machine learning algorithms
- FFmpeg – [sample usage] [API] – A complete, cross-platform solution to record, convert and stream audio and video
- FlyCapture – [sample usage] [API] – Image acquisition and camera control software from PGR
- Spinnaker – [sample usage] [API] – Image acquisition and camera control software from FLIR
- libdc1394 – [sample usage] [API] – A high-level API for DCAM/IIDC cameras
- OpenKinect – [sample usage] [API] [API 2] – Open source library to use Kinect for Xbox and for Windows sensors
- librealsense – [sample usage] [API] [API 2] – Cross-platform library for Intel RealSense depth and tracking cameras
- videoInput – [sample usage] [API] – A free Windows video capture library
- ARToolKitPlus – [sample usage] [API] – Marker-based augmented reality tracking library
- Chilitags – [sample usage] [API] – Robust fiducial markers for augmented reality and robotics
- flandmark – [sample usage] [API] – Open-source implementation of facial landmark detector
- Arrow – [sample usage] [API] – A cross-language development platform for in-memory data
- HDF5 – [sample usage] [API] – Makes possible the management of extremely large and complex data collections
- Hyperscan – [sample usage] [API] – High-performance regular expression matching library
- LZ4 – [sample usage] [API] – Extremely fast compression algorithm
- MKL – [sample usage] [API] – The fastest and most-used math library for Intel-based systems
- oneDNN – [sample usage] [API] [API 2] – Intel Math Kernel Library for Deep Neural Networks (DNNL)
- OpenBLAS – [sample usage] [API] – An optimized BLAS library based on GotoBLAS2 1.13 BSD version, plus LAPACK
- ARPACK-NG – [sample usage] [API] – Collection of subroutines designed to solve large scale eigenvalue problems
- CMINPACK – [sample usage] [API] – For solving nonlinear equations and nonlinear least squares problems
- FFTW – [sample usage] [API] – Fast computing of the discrete Fourier transform (DFT) in one or more dimensions
- GSL – [sample usage] [API] – The GNU Scientific Library, a numerical library for C and C++ programmers
- CPython – [sample usage] [API] – The standard runtime of the Python programming language
- NumPy – [sample usage] [API] – Base N-dimensional array package
- SciPy – [sample usage] [API] – Fundamental library for scientific computing
- Gym – [sample usage] [API] – A toolkit for developing and comparing reinforcement learning algorithms
- LLVM – [sample usage] [API] – A collection of modular and reusable compiler and toolchain technologies
- libffi – [sample usage] [API] – A portable foreign-function interface library
- libpostal – [sample usage] [API] – For parsing/normalizing street addresses around the world
- LibRaw – [sample usage] [API] – A simple and unified interface for RAW files generated by digital photo cameras
- Leptonica – [sample usage] [API] – Software useful for image processing and image analysis applications
- Tesseract – [sample usage] [API] – Probably the most accurate open source OCR engine available
- Caffe – [sample usage] [API] – A fast open framework for deep learning
- OpenPose – [sample usage] [API] – Real-time multi-person keypoint detection for body, face, hands, and foot estimation
- CUDA – [sample usage] [API] – Arguably the most popular parallel computing platform for GPUs
- NVIDIA Video Codec SDK – [sample usage] [API] – An API for hardware accelerated video encode and decode
- OpenCL – [sample usage] [API] – Open standard for parallel programming of heterogeneous systems
- MXNet – [sample usage] [API] – Flexible and efficient library for deep learning
- PyTorch – [sample usage] [API] – Tensors and dynamic neural networks with strong GPU acceleration
- SentencePiece – [sample usage] [API] – Unsupervised text tokenizer for neural-network-based text generation
- TensorFlow – [sample usage] [API] – Computation using data flow graphs for scalable machine learning
- TensorFlow Lite – [sample usage] [API] – An open source deep learning framework for on-device inference
- TensorRT – [sample usage] [API] – High-performance deep learning inference optimizer and runtime
- Triton Inference Server – [sample usage] [API] – An optimized cloud and edge inferencing solution
- ALE – [sample usage] [API] – The Arcade Learning Environment to develop AI agents for Atari 2600 games
- DepthAI – [sample usage] [API] – An embedded spatial AI platform built around Intel Myriad X
- ONNX – [sample usage] [API] – Open Neural Network Exchange, an open source format for AI models
- nGraph – [sample usage] [API] – An open source C++ library, compiler, and runtime for deep learning frameworks
- ONNX Runtime – [sample usage] [API] – Cross-platform, high performance scoring engine for ML models
- TVM – [sample usage] [API] – An end to end machine learning compiler framework for CPUs, GPUs and accelerators
- Bullet Physics SDK – [sample usage] [API] – Real-time collision detection and multi-physics simulation
- LiquidFun – [sample usage] [API] – 2D physics engine for games
- Qt – [sample usage] [API] – A cross-platform framework that is usually used as a graphical toolkit
- Skia – [sample usage] [API] – A complete 2D graphic library for drawing text, geometries, and images
- cpu_features – [sample usage] [API] – A cross platform C99 library to get cpu features at runtime
- ModSecurity – [sample usage] [API] – A cross platform web application firewall (WAF) engine for Apache, IIS and Nginx
- Systems – [sample usage] [API] – To call native functions of operating systems (glibc, XNU libc, Win32, etc)
- Add here your favorite C/C++ library, for example: Caffe2, OpenNI, OpenMesh, PCL, etc. Read about how to do that.
We will add more to this list as they are made, including those from outside the bytedeco/javacpp-presets repository.
Projects Leveraging the Presets Bindings
- JavaCV [API] – Library based on the JavaCPP Presets that depends on commonly used native libraries in the field of computer vision to facilitate the development of those applications on the Java platform. It provides easy-to-use interfaces to grab frames from cameras and audio/video streams, process them, and record them back on disk or send them over the network.
- JavaCV Examples – Collection of examples originally written in C++ for the book entitled OpenCV 2 Computer Vision Application Programming Cookbook by Robert Laganière, but ported to JavaCV and written in Scala.
- ProCamCalib – Sample JavaCV application that can perform geometric and photometric calibration of a set of video projectors and color cameras.
- ProCamTracker – Another sample JavaCV application that uses the calibration from ProCamCalib to implement a vision method that tracks a textured planar surface and realizes markerless interactive augmented reality with projection mapping.
More Project Information
Please refer to the contribute and download pages for more information about how to help out or obtain this software.
See the developer site on GitHub for more general information about the Bytedeco projects.
Latest News
The importance of a distribution
Happy New Year! We were able to deliver yet another release on time. Version 1.4.4 is available for JavaCPP, JavaCPP Presets, JavaCV, ProCamCalib, and ProCamTracker. Many thanks to all contributors! Since the previous post, according to git shortlog 1.4.2..1.4.4 --summary
, they were:
Alex Merritt, Aman Gupta, Ao Qi, bitstormGER, deinhofer, Deividi, eguid, EmergentOrder, Gertjan Al, HGuillemet, Jarek Sacha, Jeremy Apthorp, kigkrazy, lloydmeta, louxiu, nahojjjen, Nico Hezel, renderdude, Samuel Audet, Taha Emara, vimalaguti, vincent-grosbois, wumo, Yuta Okamoto, Zayin Krige
Many thanks as well to those not on this list but who helped fix CI settings, test builds, debug code, file suggestions, make corrections, report bugs, propose ideas, update wiki pages, etc. As usual this release fixes many issues and contains a lot of updates, all the details can be found in the CHANGELOG.md
files, but one important change concerns MXNet, whose Scala API is now partially usable from Java. Consequently, the JavaCPP Presets for MXNet now also bundle the official Scala API, just as with the official Java APIs of OpenCV and TensorFlow. Given these developments, there is one question that keeps coming up over and over again: Why do we need a distribution like Bytedeco even if everyone shows up with Java APIs on their own?
Well, for one thing, the feature set exposed by these official APIs for Java is very limited, so many users still require access to the C/C++, Scala, or even Python APIs. Moreover, PyTorch is slated to become the first deep learning framework with a full-featured and easy-to-use C++ API, but with no plans in sight for Java, so once the C++ API becomes usable we intend on integrating it to Bytedeco as part of the JavaCPP Presets, see issue bytedeco/javacpp-presets#623.
Another reason could be that JavaCPP provides a set of basic classes as foundation for native libraries, such as PointerScope
introduced in the previous post. Thanks to Hervé Guillemet and Sam Carlberg, they will also soon support the Java Platfom Module System (JPMS), including jlink
, something that the upstream projects have no plans to provide for their Java APIs. (A preview version of OpenCV is already available on the jpms
branch with corresponding snapshot artifacts.) Although Project Panama promises to offer most of the features that JavaCPP already provides today, delivery is expected only in a few more years, numbers showing faster-than-JNI performance for both JIT and AOT compilers are still lacking, development for C++ has yet to start, and it does not aspire to support platforms such as Android and iOS. Though, even in the case of plain Java SE, a loader for native libraries is required, something that community members such as Johan Vos understand, but there are no plans to integrate such functionality to OpenJDK. Unless a fork like Corretto happens to stimulate evolution towards the demands of contemporary software development, I am afraid the need for a third-party tool like JavaCPP is here to stay for the foreseeable future.
Nevertheless, assuming that Project Panama succeeds and renders JavaCPP obsolete—which would be awesome, although doubtful based on the discussion above, but just for the sake of the argument—the need for a distribution like Bytedeco will remain. The redistributable binaries for CUDA are over 2 GB for all supported platforms, compressed, and similarly for MKL, which is about 0.5 GB for all supported platforms. These include aggressively optimized implementations of BLAS, LAPACK, and FFTW, among many other things, achieving at least 10× speedups over anything one could possibly write in pure Java today. (Project Panama also promises to fix this for CPUs, eventually, but not for GPUs.) Their files are currently bundled by Bytedeco and shared by, for instance, Deeplearning4j, and the JavaCPP Presets for OpenCV, Caffe, MXNet, and TensorFlow. To offer a user-friendly experience, if each of these projects were to start bundling CUDA and MKL for all platforms on their own, an application depending on all those libraries may very well end up bundling over 10 GB of duplicate code! Projects competing against each other either downstream or upstream cannot offer such redistributables as shared resources: That is the role of a distribution. The goal is to make all those libraries, including the JDK itself, work well together.
Right now, unless we are misinformed, Bytedeco is the only such distribution available for Java, however small it may be, which is indeed quite sad, but I hope this post helps spur more cooperation or, at the very least, some competition. There is still much to be done before the community at large begins to recognize the clear necessity for Java distributions of native libraries that work not only with Java SE on Linux, Mac, and Windows, but also for mobiles and embedded platforms such as Android, iOS, and Raspberry Pi.
More concretely, to attain these goals, we will have to start putting more resources in priority into the following efforts:
- Obtain more visibility, gain a wider sense of community, by
- Writing more blog posts, on a fancier web site,
- Meeting people in person at conferences, workshops, etc, and
- Participating in upstream projects, driving acceptance by their developers and users, by
- Reporting bugs and helping to fix them, and even by
- Contributing and maintaining JavaCPP-based wrappers upstream, among other things;
- Aim for fully automated bindings generation for the features of C++ most widely used by native libraries,
- In all probability with the help of libclang, see issue bytedeco/javacpp#51 and the samples in Java; and
- Keep creating new presets for additional native libraries, plus maintain them up-to-date with upstream projects.
A lot of help is going to be required to realize all this, so please spread the word! To contribute yourself to one of the items listed above, please communicate via the mailing list from Google Groups, issues on GitHub, or the chat room at Gitter. We are looking forward to continue working with everyone on all these projects this year as well!