Custom Data TensorFlow Object detection API

Custom Object Detection

This blog is to explain how to install TensorFlow Object detection API in Windows 10 as well as how to train train a convolution neural network to do object detection on your own data set.

Steps:

Installation and Configuration

Install Python

First we need to Install Python on Windows 10. Install from this link:

Python Release Python 3.6.5

After we have installed Python in windows, lets start to install Tensorflow.

Install TensorFlow

Open an Command Prompt and run the following code:

C:\Users\HP > pip install tensorflow = 1.15.2

Install the other necessary packages by issuing the following commands

C:\Users\HP > pip install pillow
C:\Users\HP > pip install lxml
C:\Users\HP > pip install Cython
C:\Users\HP > pip install jupyter
C:\Users\HP > pip install matplotlib
C:\Users\HP > pip install pandas
C:\Users\HP > pip install opencv-python

Download TensorFlow Object Detection API repository from GitHub

Create a folder named tensorflow1 in C, this working directory will contain all Tensorflow object detection frameworks, and also the test/train images, configuration files etc. Download the full Tensorflow object detection repository by clicking Clone or download. Open the downloaded file and extract the model-master file to the C:\tensorflow1 and rename the models-master folder to models.

Download Faster-RCNN-Inception-V2 model

This post will use the Faster-RCNN-Inception-V2 model, you could download the model here . Open the downloaded file and faster_rcnn_inception_v2_coco_2018_01_28 folder to the C:\tensorflow1\models\research\object_detection folder. (Note: The model date and version will likely change in the future, but it should still work with this post.)

Download useful python script

Download the repository here. It contains useful python scripts for generating the training data. extract all the contents directly into the C:\tensorflow1\models\research\object_detection directory

To train our own dataset we need to delete the following files

• All files in \object_detection\images\train and \object_detection\images\test
• The “test_labels.csv” and “train_labels.csv” files in \object_detection\images
• All files in \object_detection\training
• All files in \object_detection\inference_graph

Configure PYTHONPATH environment variable

C:\Users\HP > set PYTHONPATH=C:\tensorflow1\models;C:\tensorflow1\models\research;C:\tensorflow1\models\research\slim

Compile Protobufs and run “setup.py”

The compilation command posted on TensorFlow’s Object Detection API ,https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md, does not work on Windows. In the Command Prompt, change directories to the \models\research directory and copy and paste the following command into the command line and press Enter:

protoc --python_out=. .\object_detection\protos\anchor_generator.proto .\object_detection\protos\argmax_matcher.proto .\object_detection\protos\bipartite_matcher.proto .\object_detection\protos\box_coder.proto .\object_detection\protos\box_predictor.proto .\object_detection\protos\eval.proto .\object_detection\protos\faster_rcnn.proto .\object_detection\protos\faster_rcnn_box_coder.proto .\object_detection\protos\grid_anchor_generator.proto .\object_detection\protos\hyperparams.proto .\object_detection\protos\image_resizer.proto .\object_detection\protos\input_reader.proto .\object_detection\protos\losses.proto .\object_detection\protos\matcher.proto .\object_detection\protos\mean_stddev_box_coder.proto .\object_detection\protos\model.proto .\object_detection\protos\optimizer.proto .\object_detection\protos\pipeline.proto .\object_detection\protos\post_processing.proto .\object_detection\protos\preprocessor.proto .\object_detection\protos\region_similarity_calculator.proto .\object_detection\protos\square_box_coder.proto .\object_detection\protos\ssd.proto .\object_detection\protos\ssd_anchor_generator.proto .\object_detection\protos\string_int_label_map.proto .\object_detection\protos\train.proto .\object_detection\protos\keypoint_box_coder.proto .\object_detection\protos\multiscale_anchor_generator.proto .\object_detection\protos\graph_rewriter.proto

( If errors still exists , you can contact me Hamza Mustafa.)

Finally, run the following commands from the C:\tensorflow1\models\research directory:

C:\tensorflow1\models\research> python setup.py build
C:\tensorflow1\models\research> python setup.py install

2. Build your own dataset

Gather pictures

I use the To obtain a robust detector, the training data should have different background and illumination condition.There should be some images where the desired object is partially obscured, overlapped with something else, or only halfway in the picture.

For my orange detector I only have one object, orange, to detect. the original dataset is quite small, only have 160 images. To enlarge my dataset I first crop the images and then do the data agumentation on the cropped images.

Label pictures

Next download and install LabelImg, point it to your \images\train directory, and then draw a box around each object in each image. Repeat the process for all the images in the \images\test directory. This will take a while:

LabelImg could generate XML file containing label data for each image. We need to convert these XML files to singular CSV files that can be then converted to the TFRecord files which are one of the inputs to the TensorFlow trainer. These XML files should be placed in the same folder and same name with images, but with .xml-extension,

Generating training data

To do this I make use of python scripts from datitrans github. ,with some slight modifications to work with our directory structure. To begin, we’re going to use xml_to_csv.py. From the \object_detection folder, issue the following command in command prompt:

C:\tensorflow1\models\research\object_detection> python xml_to_csv.py

Now, grab generate_tfrecord.py. The only modification that you will need to make here is in the class_text_to_int function. You need to change this to your specific class. In our case, we just have ONE class, we will have it like this:

3. Create Labelmap and Configure Training

Label Map

The label map tell the classifier what each object is by definig a map from the class names to class IDs. Create a labelmap.pbtxt in\object_detection\training folder. The type in the label map in the format below

Configure training

Navigate to C:\tensorflow1\models\research\object_detection\samples\configs and copy the faster_rcnn_inception_v2_pets.config file into the \object_detection\training directory. Then make the following changes to the faster_rcnn_inception_v2_pets.config file,

• Line 9. Change num_classes to the number of different objects you want the classifier to detect. For the above basketball, shirt, and shoe detector, it would be num_classes : 3 .
• Line 110. Change fine_tune_checkpoint to: fine_tune_checkpoint : “C:/tensorflow1/models/research/object_detection/faster_rcnn_inception_v2_coco_2018_01_28/model.ckpt”
• Lines 126 and 128. In the train_input_reader section, change input_path and label_map_path to:
  - input_path : “C:/tensorflow1/models/research/object_detection/train.record”
  - label_map_path: “C:/tensorflow1/models/research/object_detection/training/labelmap.pbtxt”
• Line 132. Change num_examples to the number of images you have in the \images\test directory.
• Lines 140 and 142. In the eval_input_reader section, change input_path and label_map_path to:
  -input_path : “C:/tensorflow1/models/research/object_detection/test.record”
  -label_map_path: “C:/tensorflow1/models/research/object_detection/training/labelmap.pbtxt”

Save the file after the changes have been made. That’s it! The training job is all configured and ready to go!

4. Run Training

Move train.py from /object_detection/legacy into the /object_detection folder and issue the following command to begin training:

python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/faster_rcnn_inception_v2_pets.config

If everything has been set up correctly, TensorFlow will initialize the training. The initialization can take up to 30 seconds before the actual training begins. When training begins, it will look like this:

You’d better train your model to train until the loss consistently drops below 0.05, which will take about 20,00 steps for me.

5. Export Inference Graph

The last step is to generate the frozen inference graph (.pb file). From the \object_detection folder, issue the following command, where “XXXX” in “model.ckpt-XXXX” should be replaced with the highest-numbered .ckpt file in the training folder:

python export_inference_graph.py --input_type image_tensor --pipeline_config_path training/faster_rcnn_inception_v2_pets.config<br> --trained_checkpoint_prefix training/model.ckpt-XXXX --output_directory inference_graph

This creates a frozen_inference_graph.pb file in the \object_detection\inference_graph folder. The .pb file contains the object detection classifier.

6. Test on your own dataset

I copied some of my images into the models/object_detection/test_images directory, and renamed them to be image3.jpg, image4.jpg...etc and opening the object_detection.py with some modifications. First head to the Variables section, and let's change the model name, and the paths to the checkpoint and the labels:

MODEL_NAME = 'inference_graph'
PATH_TO_CKPT = MODEL_NAME + '/frozen_inference_graph.pb'
PATH_TO_LABELS = os.path.join('training', 'labelmap.pbtxt')
NUM_CLASSES = 1

Finally, in the Detection section, change the TEST_IMAGE_PATHS var to:
TEST_IMAGE_PATHS = [ os.path.join(PATH_TO_TEST_IMAGES_DIR, 'image{}.jpg'.format(i)) for i in range(4, 13) ]

With that we approach to end …. RUN your script and see the results as below:

If you found this article interesting, helpful and if you learn something from this article, please comment and leave feedback.