Facebook has released an open-source platform named Droidlet – an all-in-one destination for creating robots with the help of robotics, NLP, and computer vision. It is an open-source platform accessible in GitHub that allows researchers to test various computer vision and natural language processing models.

Droidlet to help researchers and hobbyists

Droidlet allows developers to try various computer vision algorithms with their robots or swap one natural language processing model. Researchers believe that combining machine learning models and perceptual modules to perceive the environment can create new robots capable of completing complicated tasks more rapidly.

It is a framework for creating embodied agents that can recognize, respond to, and navigate their surroundings. It facilitates integrating all types of cutting-edge machine learning algorithms into these systems, allowing users to prototype new ideas quicker than ever before!

Also, the Droidlet allows creators to quickly create agents capable of doing complicated tasks in the real world or simulated settings such as Minecraft or Habitat. The Droidlet is enhanced with an interactive dashboard, which researchers may utilize as an operational interface while developing agents.

It contains debugging and visualization tools and an interface for real-time agent error correction and crowdsourcing annotation. Like the rest of the agents, the dashboard emphasizes modularity and makes it simple for academics or hobbyists to add additional widgets and features.

This platform is solid and adaptable, and it may be utilized independently of the complete agent. The Droidlet will grow more robust over time as they add additional jobs depending on sensory modalities or other hardware configurations that others have contributed to.

How is Droidlet created?

The Droidlet consists of a combination of heuristic and learned components that may be trained with static data when handy or dynamic data when suitable. Droidlet has a memory system that acts as a store for real-time data from various modules. A complete set of perceptual modules that process real-time data from the real-life environment, a collection of lower-level tasks that affect changes in the robots’ territory, and a controller that executes orders from the memory system.

This is not the first time Facebook is keen on developing real-time Robots. Researchers at Facebook AI are training legged robots to walk like humans using Rapid Motor Adaption.

Module-to-module interactions

  • A memory system that serves as a repository for data from different modules
  • A collection of perceptual modules that process and store information from the outside environment.
  • A controller determines which tasks to perform depending on the memory system’s status and so on.

The modules mentioned above can be further subdivided into trainable or heuristic components, and the modules – developers or researchers can use the dashboards outside the Droidlet ecosystem.

Droidlet also includes battery-powered devices that aid in perceiving the surroundings through pre-trained object identification and posture estimation algorithms. The Droidlet is also renowned for assisting researchers in developing embodied agents that integrate Artificial Intelligence and machine learning algorithms with little friction.

It is expected that Facebook Droidlet will enable academics and developers to cooperate and create more intelligent robots in less time than is now possible.