Planning in Hybrid Geometric and Object-Level Representations
Navigation in novel environments is challenging
In this work, we would like to enable a robotic agent to quickly and intelligently find promising trajectories through structured, unknown environments. A common assumption is to optimistically treat space the that has not yet been observed as unoccupied, but this assumption can lead to the myopic navigation behaviors. In more extreme cases, a robot might not even be able to find a plan in it’s allotted computation budget if considers all unknown space as being useful to find a plan.
Figure: To navigate to the goal (red), a simulated robot (green) builds an occupancy map using a depth sensor with a limited field of view and range (black is free space, white is occupied, and grey is unobserved space). To make forward progress, the agent optimistically assumes that all unknown space is unoccupied, leading to myopic and greedy behavior, where several unnecessary rooms are visited on the the way to the goal.
We propose to use both geometric and object-level semantic representations to inform planning. Semantic information can provide high-level structural cues about the environment. For example, a door is a likely step in traversing between rooms. Hand-coding rules such as “go through doors” is insufficient, as the optimal trajectory is highly dependent on the surrounding geometry and the objective of the robot. Therefore, we must determine how to model the relationship between geometry and semantics to determine where good plans lie.
We learn a sampling distribution informed by both occupancy maps and object-level semantic maps
A key insight in our work is that probability distributions are a natural way to capture this richness. The framework of sampling based motion planning gives us a way to utilize a sampling distribution learned from experience to focus our planning efforts on promising navigation strategies. We train a deep neural network to predict a sampling distribution, which is then used by a sampling based motion planner to direct the robot to the goal.
Figure: In this example, the learned sampling distribution (blue indicates high predicted probability and grey indicated low predicted probability of the location lying on the optimal trajectory to the goal) guides the robot down the hallway and to the goal.
To read more about this project, including quantitative results, see our paper or check out our ICRA talk.