Core Technologies

• JavaCPP [API] – A tool that can not only generate JNI code but also build native wrapper library files from an appropriate interface file written entirely in Java. It can also parse automatically C/C++ header files to produce the required Java interface files.

Prebuilt Java Bindings to C/C++ Libraries

These are part of a project that we call the JavaCPP Presets. Many coexist in the same GitHub repository, and all use JavaCPP to wrap predefined C/C++ libraries from open-source land. The bindings expose almost all of the relevant APIs and make them available in a portable and user-friendly fashion to any Java virtual machine (including Android), as if they were like any other normal Java libraries. We have presets for the following C/C++ libraries:

• OpenCV – [sample usage] [API] – More than 2500 optimized computer vision and machine learning algorithms
• FFmpeg – [sample usage] [API] – A complete, cross-platform solution to record, convert and stream audio and video
• FlyCapture – [sample usage] [API] – Image acquisition and camera control software from PGR
• Spinnaker – [sample usage] [API] – Image acquisition and camera control software from FLIR
• libdc1394 – [sample usage] [API] – A high-level API for DCAM/IIDC cameras
• OpenKinect – [sample usage] [API] [API 2] – Open source library to use Kinect for Xbox and for Windows sensors
• librealsense – [sample usage] [API] [API 2] – Cross-platform library for Intel RealSense depth and tracking cameras
• videoInput – [sample usage] [API] – A free Windows video capture library
• ARToolKitPlus – [sample usage] [API] – Marker-based augmented reality tracking library
• Chilitags – [sample usage] [API] – Robust fiducial markers for augmented reality and robotics
• flandmark – [sample usage] [API] – Open-source implementation of facial landmark detector
• Arrow – [sample usage] [API] – A cross-language development platform for in-memory data
• HDF5 – [sample usage] [API] – Makes possible the management of extremely large and complex data collections
• Hyperscan – [sample usage] [API] – High-performance regular expression matching library
• LZ4 – [sample usage] [API] – Extremely fast compression algorithm
• MKL – [sample usage] [API] – The fastest and most-used math library for Intel-based systems
• oneDNN – [sample usage] [API] [API 2] – Intel Math Kernel Library for Deep Neural Networks (DNNL)
• OpenBLAS – [sample usage] [API] – An optimized BLAS library based on GotoBLAS2 1.13 BSD version, plus LAPACK
• ARPACK-NG – [sample usage] [API] – Collection of subroutines designed to solve large scale eigenvalue problems
• CMINPACK – [sample usage] [API] – For solving nonlinear equations and nonlinear least squares problems
• FFTW – [sample usage] [API] – Fast computing of the discrete Fourier transform (DFT) in one or more dimensions
• GSL – [sample usage] [API] – The GNU Scientific Library, a numerical library for C and C++ programmers
• CPython – [sample usage] [API] – The standard runtime of the Python programming language
• NumPy – [sample usage] [API] – Base N-dimensional array package
• SciPy – [sample usage] [API] – Fundamental library for scientific computing
• Gym – [sample usage] [API] – A toolkit for developing and comparing reinforcement learning algorithms
• LLVM – [sample usage] [API] – A collection of modular and reusable compiler and toolchain technologies
• libffi – [sample usage] [API] – A portable foreign-function interface library
• libpostal – [sample usage] [API] – For parsing/normalizing street addresses around the world
• LibRaw – [sample usage] [API] – A simple and unified interface for RAW files generated by digital photo cameras
• Leptonica – [sample usage] [API] – Software useful for image processing and image analysis applications
• Tesseract – [sample usage] [API] – Probably the most accurate open source OCR engine available
• Caffe – [sample usage] [API] – A fast open framework for deep learning
• OpenPose – [sample usage] [API] – Real-time multi-person keypoint detection for body, face, hands, and foot estimation
• CUDA – [sample usage] [API] – Arguably the most popular parallel computing platform for GPUs
• NVIDIA Video Codec SDK – [sample usage] [API] – An API for hardware accelerated video encode and decode
• OpenCL – [sample usage] [API] – Open standard for parallel programming of heterogeneous systems
• MXNet – [sample usage] [API] – Flexible and efficient library for deep learning
• PyTorch – [sample usage] [API] – Tensors and dynamic neural networks with strong GPU acceleration
• SentencePiece – [sample usage] [API] – Unsupervised text tokenizer for neural-network-based text generation
• TensorFlow – [sample usage] [API] – Computation using data flow graphs for scalable machine learning
• TensorFlow Lite – [sample usage] [API] – An open source deep learning framework for on-device inference
• TensorRT – [sample usage] [API] – High-performance deep learning inference optimizer and runtime
• Triton Inference Server – [sample usage] [API] – An optimized cloud and edge inferencing solution
• ALE – [sample usage] [API] – The Arcade Learning Environment to develop AI agents for Atari 2600 games
• DepthAI – [sample usage] [API] – An embedded spatial AI platform built around Intel Myriad X
• ONNX – [sample usage] [API] – Open Neural Network Exchange, an open source format for AI models
• nGraph – [sample usage] [API] – An open source C++ library, compiler, and runtime for deep learning frameworks
• ONNX Runtime – [sample usage] [API] – Cross-platform, high performance scoring engine for ML models
• TVM – [sample usage] [API] – An end to end machine learning compiler framework for CPUs, GPUs and accelerators
• Bullet Physics SDK – [sample usage] [API] – Real-time collision detection and multi-physics simulation
• LiquidFun – [sample usage] [API] – 2D physics engine for games
• Qt – [sample usage] [API] – A cross-platform framework that is usually used as a graphical toolkit
• Skia – [sample usage] [API] – A complete 2D graphic library for drawing text, geometries, and images
• cpu_features – [sample usage] [API] – A cross platform C99 library to get cpu features at runtime
• ModSecurity – [sample usage] [API] – A cross platform web application firewall (WAF) engine for Apache, IIS and Nginx
• Systems – [sample usage] [API] – To call native functions of operating systems (glibc, XNU libc, Win32, etc)
• Add here your favorite C/C++ library, for example: Caffe2, OpenNI, OpenMesh, PCL, etc. Read about how to do that.

We will add more to this list as they are made, including those from outside the bytedeco/javacpp-presets repository.

Projects Leveraging the Presets Bindings

• JavaCV [API] – Library based on the JavaCPP Presets that depends on commonly used native libraries in the field of computer vision to facilitate the development of those applications on the Java platform. It provides easy-to-use interfaces to grab frames from cameras and audio/video streams, process them, and record them back on disk or send them over the network.
• JavaCV Examples – Collection of examples originally written in C++ for the book entitled OpenCV 2 Computer Vision Application Programming Cookbook by Robert Laganière, but ported to JavaCV and written in Scala.
• ProCamCalib – Sample JavaCV application that can perform geometric and photometric calibration of a set of video projectors and color cameras.
• ProCamTracker – Another sample JavaCV application that uses the calibration from ProCamCalib to implement a vision method that tracks a textured planar surface and realizes markerless interactive augmented reality with projection mapping.

My main field of expertise being basically computer vision, there I see something like JavaCPP playing an important role. Apart from my own efforts with JavaCV, recently I became aware of similar sentiments in the field of deep learning, which has been gaining tremendously in popularity over the past few years. Namely, Yann LeCun, one of the gurus of deep learning, has made surprisingly relevant comments last year about their own efforts:

(At Facebook) We are using Torch7 for many projects (as does Deep Mind and several groups at Google) and will be contributing to the public version.

Torch is a numerical/scientific computing extension of LuaJIT with an ML/neural net library on top.

The huge advantage of LuaJIT over Python is that it way, way faster, leaner, simpler, and that interfacing C/C++/CUDA code to it is incredibly easy and fast.

— Yann LeCun’s answers from the Reddit AMA

In other words, he admits that they would probably be using Python if the runtime were faster and could easily interface with native functionality, the lack of which are reason enough to justify taking some distance from the more vibrant communities evolving around NumPy, SciPy, and Theano. Java already has performance and simplicity on its side, and even leanness with implementations like Avian sporting a footprint of less than 1 MB, but easy and fast access to native functionality never materialized — until JavaCPP became reality. Lua can interface directly with C only, but we have a better solution here that can even use C++ template libraries for CUDA, such as Thrust, all that without writing a single line of C++: Interface Thrust and CUDA with JavaCPP. To illustrate my point and demonstrate the potential of our approach, I created Java bindings for Caffe, one of the most popular deep learning frameworks, that also happens to feature a complicated C++ API. Nevertheless, it took only a weekend’s time to create an almost complete interface, now available on GitHub as JavaCPP Presets for Caffe. At the time of this writing, we are not aware of any other existing Java wrappers, not even partial, for Caffe.

Now, for someone working in this kind of field, what else is missing from Java to become an attractive alternative to Python or Lua? An interactive command line interface would be one of those things. Even though languages such as Groovy or Scala already have REPL on top of the JVM, it might become a standard feature of the JDK itself as soon as Java 9 with JEP 222: Java Read-Eval-Print Loop (REPL). The only other important missing piece seems to be extensive library support for image processing, statistics, operations on matrices, data visualization, etc. We ourselves are attempting to provide some of these APIs by taping into available native libraries via the JavaCPP Presets, but given the recent interest expressed by the industry for deep learning, where Java is king, popular frameworks such as Deeplearning4j have already emerged, in a renewed quest to fill those gaps in Java.

That said, we hope the tools offered here can help you succeed in your own endeavors, deep learning or not. We would very much like to hear what you have to say, so do not hesitate to leave a comment. Moreover, if there is anything you need that is not currently available and would like to have a more private discussion, please contact us by email. This will also help us considerably in better orienting our future goals. Thank you very much for your time!