Return to site

How to classify a cucumber by CurieNeurons?

· tutorial,CurieNeurons,CurieNeurons kit

Have you heard the news that a Japanese cucumber farmer used deep learning and TensorFlow to build a cucumber sorting machine? Well, inspired by this, we have also built a cucumber sorting machine using the CurieNeurons Kit.

The major difference is that, with CurieNeuron network, each cucumber can be identified locally, you don't need to upload all the files to cloud anymore.

Here is a tutorial video of how we built it. Enjoy!

Below is the complete guide of how to do it.

 

If you have ecountered questions, feel free to comment below. We will do our best to help!

Below is the complete guide of how to do it.

1. Preparation

Materials

  • ArduCam expansion board V2
  • OV2640 camera module
  • MicroSD memory card (4G or more)
  • Arduino / Genuino 101 development board
  • Mobile phone stand

2. Programming the development board

1). Download and install library files

First, download and install the corresponding library files:

  • CurieNeurons Pro
  • Could be found in the CurieNeurons Kit, or purchased separately at Big Bang.
  • ArduCAM and UTFT4ArduCAM_SPI
  • Could be downloaded from the ArduCam official website or be downloaded here.

After decompression, you can see the corresponding ArduCAM and UTFT4ArduCAM_SPI library folders. In the library manager folder, select and install these two libraries.

2). Library configuration

In the directory ArduCam / memorysaver.h, where the library file is located, follow the prompts to uncomment the following two lines:

  • #define ARDUCAM_SHIELD_V2
  • #define OV2640_CAM

The complete code is as follows:

    #ifndef _MEMORYSAVER_

    #define _MEMORYSAVER_

     

    //Only when using raspberry,enable it

    //#define RASPBERRY_PI

     

    //There are two steps you need to modify in this file before normal compilation

    //Only ArduCAM Shield series platform need to select camera module, ArduCAM-Mini series platform doesn't

     

    //Step 1: select the hardware platform, only one at a time

    //#define OV2640_MINI_2MP

    //#define OV5642_MINI_5MP

    //#define OV5642_MINI_5MP_BIT_ROTATION_FIXED

    //#define OV5642_MINI_5MP_PLUS

    //#define OV5640_MINI_5MP_PLUS

     

    //#define ARDUCAM_SHIELD_REVC

    #define ARDUCAM_SHIELD_V2

     

    //Step 2: Select one of the camera module, only one at a time

    #if (defined(ARDUCAM_SHIELD_REVC) || defined(ARDUCAM_SHIELD_V2))

    //#define OV7660_CAM

    //#define OV7725_CAM

    //#define OV7670_CAM

    //#define OV7675_CAM

    #define OV2640_CAM

    //#define OV3640_CAM

    //#define OV5642_CAM

    //#define OV5640_CAM

     

    //#define MT9D111A_CAM

    //#define MT9D111B_CAM

    //#define MT9M112_CAM

    //#define MT9V111_CAM

    //#define MT9M001_CAM

    //#define MT9T112_CAM

    //#define MT9D112_CAM

    #endif

     

    #endif //_MEMORYSAVER_

    You can directly replace the two lines of code and save the program.

    The configuration code here configues the expansion board and camera module model that we use.

    3). Upload the Program

    CurieNeurons Pro provides examples that can help you quickly learn how to use it:

    File -> example -> CurieNeuronsPro -> CurieNeurons_ArduCam_Demo_3Feat

    This example requires the use of the Shutter button on the ArduCam expansion board, which blocks fixed operation, so to make it simple, we rewrite it so that we could send operational instructions and monitoring results directly to the serial monitor interface.

    The code could be downloaded here:

    Open it directly, and upload it to Arduino / Genuino 101.

    Click the Upload button to start uploading.

    Uploading......

    If the information box shows the following information, it means that you need to press the MASTER_RESET button on the board:

    1. Flashing is taking longer than expected
    2. Try pressing MASTER_RESET button

    After the upload is successful, the message box will show:

    1. Starting download script...
    2. SUCCESS: Sketch will execute in about 5 seconds.

    Then we can disconnect the Genuino101 power supply and continue to the next step.

    3. Assemble the hardware

    First, assemble the Genuino 101 and ArduCam expansion boards:

    And then install the MicroSD card into the MicroSD card slot on the ArduCam expansion board:

    Connect the OV2640 camera module to the expansion board:

    Finally, fix the mobile phone stand:

    Overview of the assembled hardware:

    After that, we put the assembled set onto the tripod.

    4. Prepare the cucumbers as samples

    First, we have to prepare some cucumber as learning samples

    We roughly divide them into four categories:

    • Good - cucumber
    • Bad - cucumber (curved)
    • Good - Dutch cucumber
    • Bad - Dutch cucumber (listless)

    5. Let Curie learn

    First, connect the Genuino 101 with a data cable, then open the Arduino IDE on your computer.

    Open the "serial monitor":

    After connecting to the serial monitor, the device will be initialized and if the following interface shows up, it indicates that it's ready:

    Now, we can let Curie study the four groups of cucumber samples:

    For example, for this "good - cucumber", if we want to let Genuino 101 study for the classification 1 (cat = 1), we need to do the following:

    First, put the cucumber into the center of the screen in the picture as far as possible to ensure that the learning sample is wrapped in the screen flashing red box.

    Then, enter "1" in the serial monitor:

    After clicking the send button, Genuino 101 will automatically sample and learn the data, and return the current number of neurons occupied:

    When completed, it will automatically make the identification:

    You can see that it has already started to correctly identify the cucumbers.

    And so on and so forth, we let Genuino 101 learn the four groups of cucumbers successively:

    "Good - cucumber" - category 1 (cat = 1), send "1";

    "Good - Dutch cucumber" - category 2 (cat = 2), send "2";

    "Bad - cucumber" - category 3 (cat = 3), send "3";

    "Bad - Dutch Cucumber" - Category 4 (cat = 4), send "4".

    In order to reduce the interference caused by the background of the picture, we can also send "0" to let Genuino 101 learn background image data as class 0 (cat = 0 / Forget), that is, to eliminate the effect. This operation will not take up neurons :

    Suggested learning process:

    Learn all the cucumber samples -> learn the background -> learn all the cucumber samples -> learn the background

    In this way, the basic learning is completed.

    5. Test the results

    Let's test the final results of recognition.

    Put a good cucumber in:

    We can view in the serial monitor that it has been correctly identified.

    Let's then put a good Dutch cucumber in:

    And a bad cucumber:

    And next, a bad Dutch cucumber:

    All identified correctly!

    If there is any error during the identification process, you can simply open the serial monitor and repeat the learning process.

    6. Technical Analysis

    How can Curie learn the image data?

    The area that we let Curie learn is in the middle of the picture, with the size 121 * 121 pixels. Based on CurieNeurons Pro's (Curie Neuron Development Tools Professional Edition) advanced features, in this case, for every learning and recognition operations, there are 3 sets of classifiers to extract the 121 * 121 pixel raw data into the following three features for Curie to learn:

    1. The data of 121 * 121 is divided into 11 * 11 pixel size blocks, and the blocks are averaged;
    2. RGB histogram;
    3. "Composite Profile"

    To put it simple:

    Each learning instruction we issued is followed by three kinds of concurrent information learning. Identification is also through the following three kinds of information.

    1. Shape, contour
    2. Object color
    3. General Information.

    During recognition, when two of the three conditions are satisfied, the recognition result is confirmed. That is what we see in the recognition results -- "matches = *" :

    7. CurieNeurons vs. TensorFlow

    Relevent information about the Tensorflow case

    The hardware used by the image recognition section:

    •     Raspberry Pi 3
    •     camera(from above)
    •     camera(from below)
    •     camera(from side)
    •     PC(windows)

    The number of training samples and time consumed:

    •     7,000 pictures
    •     A total of 2 to 3 days to complete the training
    •     About 24.7 seconds to 37 seconds for each picture training time.

    The quality of the training samples:

    •     80 * 80 pixels
    •     Learning features are only shape-related information

    Relevent information about the CurieNrueons case

    Image recognition part used by the hardware:

    1. Genuino 101
    2. ArduCAM expansion board + modules

    Training time consumed:

    • Learning and recognition by the Neural network takes only a few nanoseconds (ns, 1 nanosecond = 0.000000001 seconds);
    • The sum of the processing, transmission and learning time for each set of data is in milliseconds;
    • Take the slowest speed as 500ms/image for the 7000 images, it only takes 58 minutes to complete all the sample learning.

    The quality of the training samples:

    •     121 * 121 pixels
    •     The sample information learned
      •  Shape and contour
      • Object color
      • General Information

    Why can CurieNeurons be so fast and efficient?

    The reason why Curie's neural network has been able to learn and recognize in a few nanoseconds (ns, 1 nanosecond = 0.000000001 seconds) is due to to General Vision's NeuroMem® technology, which completely solidifies the neural network into the chip. Its unique Daisy Chain makes each data into the neural network be able to simultaneously reach each neuron for processing.

    All of these Curie's hidden super power can only be completely released using the CurieNeurons Pro (Curie neural network tool professional version) by General Vision.

    All Posts
    ×

    Almost done…

    We just sent you an email. Please click the link in the email to confirm your subscription!

    OKSubscriptions powered by Strikingly