Training a custom object detection model in Tensorflow and Building an IOS app with Google Cloud Platform

Object Detection. Swift. Tensorflow. Google Cloud Platform (GCP).

A basic front app for detecting Australian spiders.

Data Collection and Preprocess

1. Data Collection: data_process folder contains two crawler python file, Flickr image crawler and google image crawler.
2. Data Notation: Once got the dataset, draw the bounding box and and get xml file for each of the images using Labelimg.
3. Generate Test and Train TFRECORD files: Split the dataset into two groups, test and train. Modifying and running the xml_to_csv.py in data_process folder to get two CSV files, test.csv and train.csv. And then modifying and running the generate_tfrecord.py as well to get corresponding .record files.
4. Create label map Create a .pbtxt file list all the classes name and format as below:

5. Select a pre-trained model and Modify .config file Select a pre-trained model from detection_models and the corresponding .config file samples/configs.

Training

1. Clone the repo models. Put the dataset, .tfrecord files, pre-trained model, and training/(label_map.pbtxt and .config) in the folder of models/research/object_detection/legacy/. Modify num_classes, train_input_path, eval_input_path and label_map_path in the .config file.
2. Start training:

python train.py --logtostderr --train_dir=YOURPATH/training/ --pipeline_config_path=YOURPATH/ssd_mobilenet_v1_pets.config

Export Model

Running the export_inference_graph.py in the foler of models/research/object_detection:

python export_inference_graph.py \
    --input_type encoded_image_string_tensor \
    --pipeline_config_path /YOURPATH/pipeline.config \
    --trained_checkpoint_prefix /YOURPATH/model.ckpt-YOUR_CHECKPOINT \
    --output_directory /YOUR_OUTPUT_PATH/

Setup Google Cloud Platform

Create a new project in GCP

Create a folder model/ in the storage of GCP. Put the saved_model.pb file from previously exported model path into the model/ directory.

Deply model in GCloud

1. Follow the instructions and install the gcloud command here
2. Create a gcloud ml-engine model: gcloud ml-engine models create MODEL_NAME
3. Show the models list: gcloud ml-engine models list
4. Deploy the model:

gcloud ml-engine versions create v1 --model=MODEL_NAME --origin=gs://BUCKET_NAME/model --runtime-version=1.12(tensorflow_running_version)

Setup Firebase project

Setup firebase project linking to your GCP project. And follow the instructions to connect the firebase with your IOS project.

Deploy Cloud Function

1. Install nvm
2. Install Firebase CLI
3. Run npm install from the /firebase/functions/ directory.
4. Update the params in index.js file with the name of your Cloud project and ML Engine model.
5. Run firebase deploy --only functions
6. Once deployed successfully, you can found a cloud function created in the GCP.

Test Cloud Function

1. Create a images/ directory in the storage.
2. Upload a test image to images/ directory.
3. If successful, the results of prediction id and confidence will be saved in the Firebase database and the outlined image will be stored in outline_img/ in the storage.