Pi_Installation.txt

Notes from a "virgin" setup 22JAN2020

Install your OS using the normal instructions.  I'll use a Pi3B+ and Raspbian "Buster" desktop (2019-09-26-raspbian-buster-full.zip) as an example.
IMHO SD cards are cheap, so buy a big enough one to have a "real" system for testing and development, YMMV.

Once you've done the initial boot/setup steps, here are some things I like to do that aren't setup by default. 
I assume you have a Monitor, Keyboard, and Mouse connected.  IMHO its best to go "headless" only after everything is working.
I'll just outline the basic steps, if Google doesn't give you the details raise an "issue" and we'll flesh out the details.
Feel free to skip any you don't like.

These easiest to do via menu->Preferences->RaspberryPiConfiguration:
0) set a good password and change the hostname!

1) activate ssh server, these extra steps make maintaining a "headless" system almost painless.
   a) on your ssh client host create a ssh-id
   b) copy it to the Pi with ssh-copy-id pi@your_hostname with ssh-copy-id
   c) turn off the "splash" screen, I like having the boot messages in case something goes wrong.

   
These steps are best done in a terminal window (or via ssh, I like ssh so I can cut and paste from my desktop with better resources for Google searches)
2) turn off screen blanking, while it doesn't matter headless, I hate screen blanking while setting up and debugging:
   a) sudo nano /etc/xdg/lxsession/LXDE-pi/autostart
      Edit to add at the end, these two lines:
      @xset s off
      @xset -dpms


3) setup samba file sharing:
   a) sudo apt-get install samba samba-common-bin     
   b) sudo nano /etc/samba/smb.conf
      Edit these sections to match:
        [global]
	        workgroup = your_workgroup
	        mangled names = no
            # follow symlinks to USB drive
            follow symlinks = yes
            wide links = yes
            unix extensions = no
        [homes]
            comment = Home Directories
            browseable = yes
            read only = no
            writeable = yes
            create mask = 0775
            directory mask = 0775
   c) create samba password:
      sudo smbpasswd -a pi

I find it useful to have the GUI digital clock display seconds to get an idea of the latency between the cameras and computer.
Opposite-click the clock and choose "Digital Clock Settings" from the popup menu.
Change %R to %R:%S in the dialog "Clock Format" box, click "OK" button.


4) Install node-red:
   a) sudo apt-get install build-essential
   b) bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)
   c) node-red-start
   d) Make it autostart at boot:
      sudo systemctl enable nodered.service
   e) install a nodejs package that I use in a function node:
      cd ~/.node-red
      npm install point-in-polygon
      cd .. && nano .node-red/settings.js 
      and add:  insidePolygon:require('point-in-polygon')
      to functionGlobalContext: {} and restart node-red.
   f) other bits and pieces not installed by default:
      sudo apt-get install mosquitto mosquitto-dev mosquitto-clients espeak-ng
   g) extra python modules:
      sudo -H pip3 install paho-mqtt imutils



In general I recommend the Coral TPU over the Movidius NCS/NCS2, but since the Pi3 lacks USB3 it can't really take full advantage of it.
But since I have both and the Python code supports both, I'll set up both. On the Pi in a terminal (or ssh login):
5) Install OpenVINO for Raspbian, full instructions: https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_raspbian.html
   a) download OpenVINO: 
      wget https://download.01.org/opencv/2019/openvinotoolkit/R3/l_openvino_toolkit_runtime_raspbian_p_2019.3.334.tgz
      (Use the "latest" its dynamic and updated periodically typically 2-4 times a year, and an NCS3 is alledgedly on the way for 2020)
   b) Create an installation directory:
      sudo mkdir -p /opt/intel/openvino
   c) Unpack the downloaded archive:
      sudo tar -xf  l_openvino_toolkit_runtime_raspbian_p_2019.3.334.tgz --strip 1 -C /opt/intel/openvino
   d) install some needed tools:
      sudo apt install cmake
   e) activate the OpenVINO environment:
      source /opt/intel/openvino/bin/setupvars.sh
   f) To use the NCS/NCS2 you need to setup the udev "rules":
      (you don't need the add the current user to the users group, user pi is there by default)
      sh /opt/intel/openvino/install_dependencies/install_NCS_udev_rules.sh
      
      Plug-in an NCS or NCS2 and do dmesg command, should see output something like this:
        [ 3270.107646] usb 1-1.1.2: New USB device found, idVendor=03e7, idProduct=2150, bcdDevice= 0.01
        [ 3270.107664] usb 1-1.1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
        [ 3270.107673] usb 1-1.1.2: Product: Movidius MA2X5X
        [ 3270.107683] usb 1-1.1.2: Manufacturer: Movidius Ltd.
        [ 3270.107692] usb 1-1.1.2: SerialNumber: 03e72150
    
      Its worthwhile to follow the "Build and Run Object Detection Sample" section on the Intel instruction site.
   g) Optional:  Make the OpenVINO setup happen on every login with: 
      echo "source /opt/intel/openvino/bin/setupvars.sh" >> ~/.bashrc
      
   At this point you now have a nice version of OpenCV installed with some extra OpenVINO support functions installed,
   EXCEPT the OpenCV 4.1.2-openvino has issues with mp4 (h.264/h.265) decoding, which breaks using rtsp streams!
   The Pi3B+ is not very usable with rtsp streams and the eariler OpenVINO versions that do work don't support the Pi4.



6) Setup the Coral TPU: https://coral.ai/docs/accelerator/get-started/
   a) Google has recently setup a Debian repo that makes it really easy!
      echo "deb https://packages.cloud.google.com/apt coral-edgetpu-stable main" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list
      curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
      sudo apt-get update
      sudo apt-get install python3-edgetpu libedgetpu1-max libedgetpu-dev edgetpu-examples
   b) Install TensorFlow Lite API:
      ( full instructions: https://www.tensorflow.org/lite/guide/python )
      wget https://dl.google.com/coral/python/tflite_runtime-1.14.0-cp37-cp37m-linux_armv7l.whl
      pip3 install tflite_runtime-1.14.0-cp37-cp37m-linux_armv7l.whl
   c) Optional, download and run some test code from Google:
      Plug in the TPU.
      mkdir coral && cd coral
      git clone https://github.com/google-coral/tflite.git
      cd tflite/python/examples/classification
      bash install_requirements.sh
      python3 classify_image.py --model models/mobilenet_v2_1.0_224_inat_bird_quant_edgetpu.tflite \
                                --labels models/inat_bird_labels.txt \
                                --input images/parrot.jpg
      
      You should get something like this:
        INFO: Initialized TensorFlow Lite runtime.
        ----INFERENCE TIME----
        Note: The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory.
        131.2ms
        11.1ms
        11.2ms
        11.1ms
        11.1ms
        -------RESULTS--------
        Ara macao (Scarlet Macaw): 0.76562

   The OpenVINO version of OpenCV will work if your cameras do Onvif snapshots, or don't trigger the above mentioned h.264/h.265 decoding issues. 
    
   You need OpenCV for my code, install it (note that as of Jan 2020 there are still issues with newer OpenCV versions):
   sudo apt-get install libhdf5-103 libatlas3-base libjasper1 libqtgui4 libqt4-test libqtcore4
   sudo -H pip3 install opencv-contrib-python==4.1.0.25

   I have a netcam with "h.265+" using a Coral TPU and OpenCV-4.1.2-openvino it gets ~3.3 fps, monotonically increasing latency and eventually crashes.
   Using pip installed OpenCV-4.1.0 it gets 5 fps (what the camera is set for) and latency is the typical rtsp ~2 seconds.
   
   


7) At this point you can download and run my Python code.  After download and unpacking, put the folder in /home/pi
   and rename it to AI, otherwise you'll have to edit all the controller scripts for the different path and name.
   a) cd /home/pi/AI
   b) chmod 755 AI_dev.py AI_OVmt.py TPU.py *.sh
   c) create a file to specify your onvif snapshot URLs or rtsp stream URLs:
   d) There are three variations of the AI code: AI_dev.py, AI_OVmt.py, TPU.py
      AI_dev.py has the most options and can run multiple AI inference threads, its mostly for development and testing, it suports PiCamera module.
      AI_OVmt.py by only supports NCS/NCS2 and CPU AI (useless on Pi usable on i3-4025 or better), defults to a single Movidius NCS device.
      TPU.py supports a single Coral TPU, no Movidius, no CPU.
      If using a Coral TPU and local KVM in a terminal window or ssh -X login:
         ./AI_dev.py -cam snapshots.txt -d 1 -nTPU 1 -ls
         -- OR --
         ./TPU.py -cam snapshots.txt -d 1 -ls
      If using a Movidius NCS or NCS2:
         ./AI_dev.py -cam snapshots.txt -d 1 -nNCS 1 -ls
         -- OR --
         ./AI_OVmt.py -cam snapshots.txt -d 1 -ls
      The saved detection images will be in /home/pi/AI/detect/yyyy-mm-dd/
   e) Exit the AI with Ctrl-C in the terminal window for "q" in the openCV display window


Need to do some node-red installation. You can skip this if you just want to use the AI and do your own thing with integration.
If not familar with using node-red, start here:  https://nodered.org/docs/tutorials/
and here:  http://www.steves-internet-guide.com/node-red-overview/
and here:  https://notenoughtech.com/home-automation/nodered-home-automation/nodered-for-beginners-2/
Being a "graphical programming environment" its a bit hard to describe using only text.
Another good set of tutorials, especially to help understand the "dashboard" is:  http://noderedguide.com/
If you learn by watching videos this is a good place to start:  https://www.youtube.com/watch?v=3AR432bguOY
8) Connect to your Pi at:  http: local.ip.addr:1880  (or localhost:1880 if running the browser on the Pi)
   a) From the "hamburger" menu dropdown choose: Manage Pallet
   b) Click the "Install" tab
   c) In the search modules box, enter
      1) node-red-node-base64 and click the install button that pops up
      2) node-red-dashboard and click the install on the one that exactly matches the search string
      3) node-red-node-email and click the install on the one that exactly matches the search string
   d) Open the Pi_AI_Controller-Viewer.json file from the distribution in your text editor and copy the contents to the clipboard
   e) From the node-red menu choose: Import
      1) paste the clipboard into the dialog that pops up
      2) press the Import to "current flow" button
      3) press the red Import button
      4) position the graphics and and click, then press the red Deploy button next to the menu
      5) if you get "successfully deployed" there is still configuration to be done, but it should be a starting point.
   f) Open a new browser tab and connect to: http: local.ip.addr:1880/ui
      This views the "dashboard" which does the basic functions like setting the notification mode, viewing a camera, etc.
      When headless, you control it from this webpage, or via MQTT messages from your home automation system.  This is only
      a starting point for you, but you can evaluate the AI performance, and with a WiFi connected Cell Phone adjust camera positions.
   
      
9) Now you can run the AI same as in step seven but leaving off the -ls option.  Node-red saves the detections which
   makes it easier to change the paths and add meaningful names for the cameras.  You can also change -d 1 to -d 0
   which will improve performance by skipping the X display of the live images.  You can view them one camera at a time
   in the UI webpage.  Viewing the UI webpage and modifying the node-red flow works best with a browser running on a different machine.
   a) If using a Coral TPU and local KVM in a terminal window or ssh -X login:
         ./AI_dev.py -cam snapshots.txt -d 1 -nTPU 1
         -- OR --
         ./TPU.py -cam snapshots.txt -d 1
      If using a Movidius NCS or NCS2:
         ./AI_dev.py -cam snapshots.txt -d 1 -nNCS 1
         -- OR --
         ./AI_OVmt.py -cam snapshots.txt -d 1
      Now saved detection images will be in /home/pi/detect/yyyy-mm-dd/
   b) You can change the node-red flow to meet your needs and redeploy without stopping and restarting the AI which can be a
      real time saver when testing.
   c) the startup scripts used by the "Launch" inject nodes use -d 0 option.


10) Real world advice.  
    a) SD cards are not the most reliable storage, I recommend formatting  a USB stick ext4 and creating a symlink to it
       for the detections, either:
       1) If using local save (delete /home/pi/AI/detect that might have got created while testing):
          ln -s /media/pi/filesystemLabel  /home/pi/AI/detect
       2) If using node-red to save detections (delete /home/pi/detect that might have been created while testing):
          ln -s /media/pi/filesystemLabel  /home/pi/detect
    b) Once you've tested things and are ready to go "live": 
       1) check "Inject once after 0.5 seconds" on the appropiate Launch inject node
       2) wire the watchdog timer to the reboot node
       3) prepare for "headless" operation with raspi-config:  BootOptions->Desktop/CLI->Console
       4) reboot.  The system should now be an "appliance" that simply boots and runs the AI
    c) For testing you can use a USB webcam by creating a camera.rtsp  file containing
       /dev/video0
       No blank line at the end!
       launch with the option -rtsp camera.rtsp instead of -cam snapshots.txt
    d) To use the PiCamera module, launch AI_dev.py with the --PiCam option in addition to any others.
       You can mix the PiCmaera module with other cameras, but the Pi3 is pretty useless with rtsp streams other than /dev/video
    e) The "Send Test Email" injection node is useful for setting up your gmail account for sending notifications.
       It needs to have "less secure access" enabled or you need to create an "app key" for it.
    f) I find MMS texts usually arrive faster (to the same phone) than Email with attachment, but I send myself both.  All
       the four major US carriers have MMS text to Email gateways.  The minor carriers are hit and miss,  Cricket does, Ting doesn't,
       for example.  A google search for your carrier and "Email to MMS Gateway" should get the answer.
    g) When using rtsp cameras, its best to add 2>/dev/null at the end of the launch command as many (most) cameras throw warnings
       when decoded with OpenCV that make the log files large and mostly useless.