Konferenzpaper | 2023 | Peer reviewed

Generation of Robot Manipulation Plans Using Generative Large Language Models

Autor(en)

Töberg, Ja. Philipp ; Cimiano, Philipp

Herausgeber*innen als Körperschaft)

"IEEE Computer Society"

Name der Konferenz

2023 Seventh IEEE International Conference on Robotic Computing (IRC)

DOI

10.1109/IRC59093.2023.00039

ISBN-13

9798350395747

URL

https://pub.uni-bielefeld.de/record/2987117

Weitere Details

Sprache

englisch

Darstellungsform

Text

Seitenbereich

190 - 197

Schlagwörter

Action Similarity , CRAM , GPT , Large Language Models , Robot Plan Generation

Designing plans that allow robots to carry out actions such as grasping an object or cutting a fruit is a time-consuming activity requiring specific skills and knowledge. The recent success of Generative Large Language Models (LLMs) has opened new avenues for code generation. In order to evaluate the ability of LLMs to generate code representing manipulation plans, we carry out experiments with different LLMs in the CRAM framework.

In our experimental framework, we ask an LLM such as ChatGPT or GPT-4 to generate a plan for a specific target action given the plan (called designator within CRAM) for a given reference action as an example. We evaluate the generated designators against a ground truth designator using machine translation and code generation metrics, as well as assessing whether they compile. We find that GPT-4 slightly outperforms ChatGPT, but both models achieve a solid performance above all evaluated metrics. However, only ~36% of the generated designators compile successfully.
In addition, we assess how the chosen reference action influences the code generation quality as well as the compilation success. Unexpectedly, the action similarity negatively correlates with compilation success. With respect to the metrics, we obtain either a positive or negative correlation depending on the used model.

Finally, we describe our attempt to use ChatGPT in an interactive fashion to incrementally refine the initially generated designator. On the basis of our observations we conclude that the behaviour of ChatGPT is not reliable and robust enough to support the incremental refinement of a designator.

Zitierstile

Töberg J-P, Cimiano P. Generation of Robot Manipulation Plans Using Generative Large Language Models. Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA.

In die Zwischenablage einfügen

Töberg, J. - P., & Cimiano, P. (2023). Generation of Robot Manipulation Plans Using Generative Large Language Models. Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA. https://doi.org/10.1109/IRC59093.2023.00039

In die Zwischenablage einfügen

Töberg, Jan-Philipp, and Cimiano, Philipp. 2023. “Generation of Robot Manipulation Plans Using Generative Large Language Models”. Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA , ed. IEEE Computer Society, 190-197. Piscataway, NJ: IEEE.

In die Zwischenablage einfügen

Töberg, J. - P., and Cimiano, P. (2023).“Generation of Robot Manipulation Plans Using Generative Large Language Models”. Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA.

In die Zwischenablage einfügen

Töberg, J.-P., & Cimiano, P., 2023. Generation of Robot Manipulation Plans Using Generative Large Language Models. Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA.

In die Zwischenablage einfügen

J.-P. Töberg and P. Cimiano, “Generation of Robot Manipulation Plans Using Generative Large Language Models”, Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA, Piscataway, NJ: IEEE, 2023.

In die Zwischenablage einfügen

Töberg, J.-P., Cimiano, P.: Generation of Robot Manipulation Plans Using Generative Large Language Models. Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA (2023).

In die Zwischenablage einfügen

Töberg, Jan-Philipp, and Cimiano, Philipp. “Generation of Robot Manipulation Plans Using Generative Large Language Models”. Presented at the 2023 Seventh IEEE International Conference on Robotic Computing (IRC), Laguna Hills, CA, USA, Piscataway, NJ: IEEE, 2023.

AMA

APA (6th ed.)

Chicago

Frontiers

Harvard

IEEE

LNCS

MLA