Prospection: Interpretable plans from language by predicting the future

Paxton, Chris; Bisk, Yonatan; Thomason, Jesse; Byravan, Arunkumar; Foxl, Dieter

doi:10.1109/icra.2019.8794441

Cited by 35 publications

(32 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…0 resulting in poor forecasts jF N ðk þ 1Þ ÀF N ðkÞj ! 0 from Eq (4), showing that the reference forces are not utilized efficiently for prediction. Small λ cause ΔU N (k) > 0 which results in efficient forecasts through optimal utilization of the reference forces.…”

Section: Plos Onementioning

confidence: 99%

“…We define a third performance metric called the stability indicator,S to check for BIBO stability of Eq (4), which can be computed as shown below:…”

Section: Force Datamentioning

confidence: 99%

“…The inputs to the algorithm are the axial force data from a force sensor,f a ðkÞ, normalized needle-tip position from a position sensor, l(k), commanded insertion velocity, ν, and total insertion time K s . When the insertion begins from k = 0, the optimal mean forces are generated fromf a ðkÞ at every time k > 0, to continuously check for force increase alerts through sigmoid transformation, steps (3)(4). After ignoring the first N values, the CFDL-MFP parameters are initialized according to the procedure described in Algorithm 1, (steps 4-6).…”

Section: A Complete Prediction Algorithmmentioning

confidence: 99%

“…There has been growing interest in developing prediction-based control strategies through sensors, to prevent undesirable events in autonomous intelligent robots, human-robot interactions, and multi-agent robots, using some danger metric. This approach of using metrics to foresee an upcoming danger enables the robot to take preventive control actions before the danger occurs [2][3][4].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Developing a novel force forecasting technique for early prediction of critical events in robotics

Narayan

Fey

2020

PLoS ONE

View full text Add to dashboard Cite

Safety critical events in robotic applications can often be characterized by forces between the robot end-effector and the environment. One application in which safe interaction between the robot and environment is critical is in the area of medical robots. In this paper, we propose a novel Compact Form Dynamic Linearization Model-Free Prediction (CFDL-MFP) technique to predict future values of any time-series sensor data, such as interaction forces. Existing time series forecasting methods have high computational times which motivates the development of a novel technique. Using Autoregressive Integrated Moving Average (ARIMA) forecasting as benchmark, the performance of the proposed model was evaluated in terms of accuracy, computation efficiency, and stability on various force profiles. The proposed algorithm was 11% more accurate than ARIMA and maximum computation time of CFDL-MFP was 4ms, compared to ARIMA (7390ms). Furthermore, we evaluate the model in the special case of predicting needle buckling events, before they occur, by using only axial force and needle-tip position data. The model was evaluated experimentally for robustness with steerable needle insertions into different tissues including gelatin and biological tissue. For a needle insertion velocity of 2.5mm/s, the proposed algorithm was able to predict needle buckling 2.03s sooner than human detections. In biological tissue, no false positive or false negative buckling detections occurred and the rates were low in artificial tissue. The proposed forecasting model can be used to ensure safe robot interactions with delicate environments by predicting adverse force-based events before they occur.

show abstract

Section: Plos Onementioning

confidence: 99%

“…We define a third performance metric called the stability indicator,S to check for BIBO stability of Eq (4), which can be computed as shown below:…”

Section: Force Datamentioning

confidence: 99%

Section: A Complete Prediction Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Developing a novel force forecasting technique for early prediction of critical events in robotics

Narayan

Fey

2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…To communicate efficiently, people model both a listener's mental state and the effects of their actions on the world. Modeling future worlds in navigation (Anderson et al, 2019) and control (Paxton et al, 2019) are open research questions, and we approximate solutions through a Recursive Mental Model (RMM) of a conversational partner. Our agent spawns instances of itself to simulate the ef-fects of dialogue acts before asking a question or generating an answer to estimate their effects on navigation.…”

Section: Introductionmentioning

confidence: 99%

RMM: A Recursive Mental Model for Dialogue Navigation

Roman¹,

Bisk

Thomason

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

Self Cite

View full text Add to dashboard Cite

Language-guided robots must be able to both ask humans questions and understand answers. Much existing work focuses only on the latter. In this paper, we go beyond instruction following and introduce a two-agent task where one agent navigates and asks questions that a second, guiding agent answers. Inspired by theory of mind, we propose the Recursive Mental Model (RMM). The navigating agent models the guiding agent to simulate answers given candidate generated questions. The guiding agent in turn models the navigating agent to simulate navigation steps it would take to generate answers. We use the progress agents make towards the goal as a reinforcement learning reward signal to directly inform not only navigation actions, but also both question and answer generation. We demonstrate that RMM enables better generalization to novel environments. Interlocutor modelling may be a way forward for human-agent dialogue where robots need to both ask and answer questions.

show abstract