Custom object detection implementation

[Kasi21]

• rpi-deep-pantilt version: 1.2.1
• Python version: 3.7
• TensorFlow version: 2.2.0
• Operating System: Windows

Description

I am trying to implement a custom object detector trained for leopard and exported from Google Cloud AutoML (cloud.google.com/vision/automl/object-detection/docs/export-edge). I can't get this to work. I've followed through on the advice given in #40, but I'm not having any luck. Specifically:

• I cloned and edited the SSDMobileNet_V3_Coco_EdgeTPU_Quant class in a new leopardtflite.py file, which also has the code for the imports needed and has the labels changed. In this file, I also point model.path to wherever the leopard tflite file is saved.

• I created a new object detection pbtxt file

• I edited cli.py to import the leopard tf lite models and run the models when the 'leopard' label is specified.

What I Did

Here's what happens when I run the code

(.venv) pi@raspberrypi:~ $ rpi-deep-pantilt track leopard  --rotation=180
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/camera.py", line 30, in run_pantilt_detect
    model = model_cls()
  File "/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/leopardtflite.py", line 51, in __init__
    self.model.path = '/home/pi/.venv/lib/python3.7/site-packages/rpi_deep_pantilt/detect/leopard.tflite'
AttributeError: 'leopardtfliteModel' object has no attribute 'model'
^C[INFO] You pressed `ctrl + c`! Exiting...
[INFO] You pressed `ctrl + c`! Exiting...
[INFO] You pressed `ctrl + c`! Exiting...

Aborted!

Any guidance you can provide on custom model implementation would be super appreciated. 🙏 I've also attached the cli file and leopardtflite.py file for reference.

leopardDetection.zip

[leigh-johnson]

Hey @Kasi21, hope you've been well. I laid some groundwork for a v2 API that will make custom model/predictor code easier to implement.

I'm taking a look at the leopard_detection_v1.h5 you emailed me, but unfortunately it looks like this model was trained using Tensorflow 1.15? I can only support TensorFlow 2.0 models (no plans to support TensorFlow 1.x right now). I'm looking at the requirements.txt in this repo for ref:
https://github.com/xuannianz/EfficientDet/blob/master/requirements.txt
https://github.com/xuannianz/EfficientDet

Your Google Cloud AutoML model should work though! If you want to upload/email me that model, I can put together an example. =)

[timayy]

Hey @leigh-johnson, you mentioned a Google Cloud AutoML model would work? Could you explain how and edge export of one could be used; would it follow a similar format to your SSDMobileNet_V3_Coco_EdgeTPU_Quant classes?

Thanks in advance :)

Edit: Also, to make sure I'm training the right models - do you have any resources on how to train SSD v3 models (likely using ssd_mobilenet_v3_small_coco_2019_08_14 as well)?

Edit2: Ended up getting all of the above working with your newest release! Along with giving my model EdgeTPU supports (using the edgetpu-compiler (older version due to unsupported runtime).