Farmers can fight invasive insects with AI and a robotic arm

A Carnegie Mellon team has developed an agricultural robot designed to protect plants and crops from a voracious invasive species. Armed with a camera, AI, and a robotic arm, TartanPest can prowl the wild, detecting and destroying the egg masses of the spotted lanternfly.

Since first being (ahem) spotted in Pennsylvania in 2014, the spotted lanternfly has spread to 14 states up and down the East Coast, reaching into the Rust Belt and making it as far west as Indiana. They’re almost certainly going to keep going, too.

These beautiful aliens — shaped something like a press-on nail, with prominent black spots on ashen wings that open to reveal a flash of red when the insect makes one of its signature leaps — pose an increasing economic threat as they spread beyond their native homes in Vietnam and China.

Thus far, the lanternflies do not appear to be causing significant damage to crops, with grapes appearing to be most at risk from their plant-slashing feeding behavior. But research is ongoing, and the USDA’s list of at-risk crops and plants is a long and comprehensive one, including apples, almonds, cherries, grapes, peaches, Christmas trees, oak trees, and hops. (They better leave off that last one. Messing with the beer? Too far, bug. Too far.)

“Currently, spotted lanternflies are concentrated in the eastern portion of the nation, but they are predicted to spread to the whole country,” Carolyn Alex, an undergrad researcher and member of the TartanPest team, said. “By investing in this issue now, we will be saving higher costs in the future.”

TartanPest is built off an electric “microtractor” made by California robotics company Farm-ng. A mounted robotic arm identifies the egg masses of the spotted lanternfly, which can resemble a brush stroke of whitish to brownish paint on hard surfaces, like tree branches. When it spots one, it uses a drill brush to scrub off the eggs.

The robot uses a deep learning model to identify the spotted lanternfly egg masses. The team worked with Iowa State biomedical professor RT Singh to gather enough images to train TartanPest. “They have a dataset with a large number of lanternfly eggs that people capture just from their cell phones,” Srinivasan Vijayarangan, a PhD student at CMU’s Robotics Institute, said in a video about the project. 

Singh’s dataset contains around 700 images — considerably less than the millions of images an AI would optimally have to learn on. To help get around the problem, the team modified the photos they did have in numerous ways, essentially creating more images; “rotation, translation, and zoom, things like that,” Vijayarangan said. 

A robotic option to stem the spotted lanternfly invasion could protect farms, lower costs, and reduce the amount of chemical pesticides used, Alex said. And if it works for this infestation, robotic pest control could become an increasingly feasible option.

We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at tips@freethink.com.