I published implementation of Faster R-CNN with MXNet C++ Frontend. You can use this implementation as comprehensive example of using MXNet C++ Frontend, it has custom data loader for MS Coco dataset, implements custom target proposal layer as a part of the project without modification MXNet library, contains code for errors checking (Missed in current C++ API), have Eigen and NDArray integration samples. Feel free to leave comments and proposes. The code is available on Github, here.
The code is based on python implementation from ijkguo, but implements model only with
resnet 101 head and can be trained only with coco dataset.
To be able to compile the code you should build MXNet from source code, with enabled Cpp Package, you can use the original tutorial for this.
Because I’m using Arch Linux, I’m sharing next tips which helped me to built MXNet for this platform:
- Install additional packages
- I used CMake to configure MXNet build
CMAKE_CXX_COMPILER=g++-7CMake parameter. (It resolved problems with tbb version used in MXNet)
OLDCMAKECUDACMake parameter, to be able to use Cuda >= 9.
USE_CPP_PACKAGECMake parameter, to enable MXNet Cpp Package (C++ frontend).
USE_OPENCVCMake parameter, to integrate MXNet with installed version of OpenCV.
To Use compiled MXNet library some additional steps are required:
- MXNet build script downloads mkldnn_intel library, but install script don’t copy it to installation folder, so copy it manually and specify “mkldtnn” for linker.
- MXNet Cpp Package header files are not copied to install folder automatically – so do it manually, provide path to
mxnet-cppfolder as include source for a compiler.
nnvmheader files are also missed in the install folder, so manually specify path to them for a compiler.
- It have sense to suppress -Wunused-parameter warnings, because they will foul compiler output.
If you are used not the top GPU you can reach a problem with cuda error
too many resources requested for launch it can be solved by adding
MSHADOW_CFLAGS += -DMSHADOW_OLD_CUDA=1 to
mxnet/mshadow/make/mshadow.mk file. This macro limits the threads of kernel launch, refer this link for more details. After making such change you will need to re-compile MXNet.
- After checking out of the code please update also submodules for the project, it have dependency for Eigen and Json parser libraries.
- You should use CMake to configure build scripts for compilation, please provide value for
LIBRARIES_DIRCMake parameter with a path to the directory where MXNet is installed in your environment.
There are two projects
train which should be used with next parameters:
rcnn_demoexecutable takes two parameters
path to file with trained parametersand
path to image file for classification. You can use pre-trained parameters from the original project. After processing you will get file, named
det.pngin your’s working directory, with rendered bounding boxes and printed labels. Also application will print classification results to the standard output. Commandline can looks like this “rcnn_demo check-point.params test.png”
rcnn_trainexecutable takes next parameters
path to the coco dataset,
path to the pretrained resnet model, flag
--start-trainwhich means starting training from scratch or
path to the file with saved check-point paramenters. Commandline can looks like this “rcnn_train /development/data/coco –params=/development/model/resnet-101-0000.params –start-train”. Default name for check-point file is
check-point.params. You can download pre-trained resnet parameters from MXNet model zoo.
Also you can download file with pre-trained parameters from this link, it was made for proof of the concept and for vehicles label types only also it was trained on small number of iteration, because I don’t have suitable hardware for full training cycle.
Please look in the source code for details of implementation, I left comments in the most interesting parts. This implementation has custom
proposal target layer as part of the project and don’t required MXNet library modification. Also it has custom loader for Coco dataset because I did not find existent one for C++. One of the biggest problem during development was absence of normal error reporting from MXNet C++ frontend (Usually API simply ignore all errors reported from C API), so I added some wrappers around C API to be aware of errors during runtime. Another tricky part was synchronising memory layout of Eigen data structures with MXNet NDArray class, please pay attention on this in the code if you will modify it.