Feat: Add external tool support to `ChatAgent` & Refactor #830

WHALEEYE · 2024-08-16T01:51:20Z

Description

Add a set of tools named external_tools into ChatAgent, allowing users to directly get tool calling requests for a certain set of external tools. This will also do refactoring to the step() and step_async() function we currently have.

Motivation and Context

This agent can facilitate the integration of CAMEL into CRAB.
We'll see if there's any workaround at CRAB's side in the future.
This will also close #894.

coderabbitai · 2024-08-16T01:51:35Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

…eature/adapt-crab

Wendong-Fan

Thanks @WHALEEYE ! Does this agent need structured output if the target of this agent is just to return the tool calling request from external tools?

Also for now if external tools is added or both internal and external tools added, this agent would not have any content in the response, does this make sense?

Could you also add some test for this agent? Thanks!

WHALEEYE · 2024-08-30T20:51:59Z

Does this agent need structured output if the target of this agent is just to return the tool calling request from external tools?

I'm doing this mainly out of the reason that we may further integrate external tool support directly into the ChatAgent, instead of making it a special agent. I demonstrated this in a case in the example (also unit test), where user doesn't want to get the external tool requests but just want normal responses (some internal tools can be called in the process).

Also for now if external tools is added or both internal and external tools added, this agent would not have any content in the response, does this make sense?

Yes, if the external tools are called, then response.msg would be empty and the tool calling request would be set in response.info. However, if no external tools are called, the agent just acts like a ChatAgent. I think this would not cause much confusion if a user chooses to pass in external tools, which means that he intentionally want the agent to return tool calling requests directly.

Wendong-Fan

Thanks @WHALEEYE , left some comments below, overall LGTM

examples/external_tools/use_external_tools.py

camel/agents/chat_agent.py

Wendong-Fan

Thanks @WHALEEYE , left some comments

Wendong-Fan · 2024-09-10T05:56:37Z

camel/agents/chat_agent.py

-                self.is_tools_added()
-                and isinstance(response, ChatCompletion)
-                and response.choices[0].message.tool_calls is not None
+                not self.is_tools_added()
+                or not isinstance(response, ChatCompletion)
+                or response.choices[0].message.tool_calls is None


if user sets tool_choice="required" in LLM config, then tools will always be added, which would lead to infinite while loop

I think it's the desired behavior in current design. Acutally the tool_choice config should be managed by the agent rather than the user.

Hey @dandansamax , I think since currently we support tool_choice in LLM parameter then we need to make sure there will not have this kind of infinite loop in ChatAgent, I discussed with @WHALEEYE , we can handle tool_choice separately when doing ChatAgent refactor further

camel/agents/chat_agent.py

dandansamax

Generally great. But so many legacy problems in the old code. We should open up new issues to tackle them.

camel/agents/chat_agent.py

dandansamax · 2024-09-10T12:59:53Z

camel/agents/chat_agent.py

-            # Format messages and get the token number
-            openai_messages: Optional[List[OpenAIMessage]]
-
+            # Check if token has exceeded
            try:
                openai_messages, num_tokens = self.memory.get_context()
            except RuntimeError as e:


Not related to this PR: why treats all RuntimeError as step token exceed?

Let's open up a new issue for this

dandansamax · 2024-09-10T13:16:08Z

camel/agents/chat_agent.py

-                base_message_item.content = str(info['tool_calls'][0].result)
+            # Normal function calling
+            tool_call_records.append(
+                await self._step_tool_call_and_update_async(response)


The async implementation is meaningless. We are only using the sequential behavior. Please open up an issue to remove all duplicated async code.

test/agents/test_chat_agent.py

camel/agents/chat_agent.py

test/agents/test_chat_agent.py

WHALEEYE · 2024-09-10T20:51:15Z

After discussing with @dandansamax, I think we'd better do some refactoring to the whole ChatAgent for a better code quality and maintainability. But I think we can merge it for now because the external tool has a high priority, and create a issue later on the refactoring. wdyt @Wendong-Fan

dandansamax

LGTM. Thanks

Wendong-Fan

As discussed, some enhancement would be implemented in another refactor PR by @WHALEEYE

add external tool agent support

50f1f2e

Merge branch 'master' into feature/adapt-crab

56b8ae6

WHALEEYE changed the title ~~add external tool agent support~~ Feat: add external tool agent support Aug 16, 2024

Wendong-Fan assigned WHALEEYE Aug 16, 2024

Wendong-Fan added the Agent Related to camel agents label Aug 16, 2024

Wendong-Fan added this to the Sprint 10 milestone Aug 16, 2024

WHALEEYE added 5 commits August 17, 2024 00:10

fix: change tool calls format

0834c86

add ext tool agent to init

7f8be4e

fix: small issues on schema

ef2c05e

Merge branch 'feature/adapt-crab' of github.com:camel-ai/camel into f…

bb3afaa

…eature/adapt-crab

Merge branch 'master' into feature/adapt-crab

b82ee34

WHALEEYE marked this pull request as ready for review August 22, 2024 16:45

WHALEEYE requested a review from Wendong-Fan August 22, 2024 16:45

Merge branch 'master' into feature/adapt-crab

4ff1e6a

Wendong-Fan reviewed Aug 30, 2024

View reviewed changes

WHALEEYE added 3 commits August 30, 2024 10:51

feat: add example for external tool agents

221d490

test: add test for external tool agents

149a553

fix: small fixes

c880929

WHALEEYE added 4 commits August 31, 2024 18:34

Merge branch 'master' into feature/adapt-crab

cd1cbd3

refactor(ChatAgent): step and step_async

24c6f84

small fix

aa6902f

small fix

1cbe8f6

WHALEEYE changed the title ~~Feat: add external tool agent support~~ [Draft] Feat: add external tool support to ChatAgent Sep 3, 2024

WHALEEYE marked this pull request as draft September 3, 2024 18:45

WHALEEYE added 3 commits September 3, 2024 16:54

refactor step()

cce8bdd

step() refactoring & bug fix

daa1ae2

Merge branch 'master' into feature/adapt-crab

4b094a5

WHALEEYE marked this pull request as ready for review September 5, 2024 02:31

WHALEEYE changed the title ~~[Draft] Feat: add external tool support to ChatAgent~~ Feat: Add external tool support to ChatAgent & Refactor Sep 5, 2024

WHALEEYE added the enhancement New feature or request label Sep 5, 2024

WHALEEYE requested review from raywhoelse, lightaime and Wendong-Fan September 5, 2024 02:32

fix type hint of output schema

f09e6e2

Wendong-Fan reviewed Sep 5, 2024

View reviewed changes

examples/external_tools/use_external_tools.py Outdated Show resolved Hide resolved

camel/agents/chat_agent.py Show resolved Hide resolved

camel/agents/chat_agent.py Outdated Show resolved Hide resolved

camel/agents/chat_agent.py Outdated Show resolved Hide resolved

WHALEEYE added 4 commits September 5, 2024 20:21

fix reviews

6cc0ffe

test: enhance example and test

564f39e

small fix

30f638f

Merge branch 'master' into feature/adapt-crab

7ba3ef6

WHALEEYE requested review from Wendong-Fan and dandansamax September 6, 2024 04:11

Merge branch 'master' into feature/adapt-crab

9da0d06

Wendong-Fan requested changes Sep 10, 2024

View reviewed changes

dandansamax requested changes Sep 10, 2024

View reviewed changes

camel-ai deleted a comment from dandansamax Sep 10, 2024

WHALEEYE added 4 commits September 10, 2024 10:55

fix reviews

37bcc4c

fix reviews

756671e

fix reviews

73c6da1

Merge branch 'master' into feature/adapt-crab

4967b0c

dandansamax reviewed Sep 10, 2024

View reviewed changes

camel/agents/chat_agent.py Outdated Show resolved Hide resolved

dandansamax reviewed Sep 10, 2024

View reviewed changes

camel/agents/chat_agent.py Show resolved Hide resolved

test/agents/test_chat_agent.py Outdated Show resolved Hide resolved

change step_token_exceed into private

9ac8122

dandansamax approved these changes Sep 11, 2024

View reviewed changes

Wendong-Fan approved these changes Sep 11, 2024

View reviewed changes

add release version

8046a76

Wendong-Fan merged commit 55c6fad into master Sep 11, 2024
6 checks passed

Wendong-Fan deleted the feature/adapt-crab branch September 11, 2024 16:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: Add external tool support to `ChatAgent` & Refactor #830

Feat: Add external tool support to `ChatAgent` & Refactor #830

WHALEEYE commented Aug 16, 2024 •

edited

Loading

coderabbitai bot commented Aug 16, 2024 •

edited

Loading

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

Wendong-Fan left a comment

WHALEEYE commented Aug 30, 2024 •

edited

Loading

Wendong-Fan left a comment

Wendong-Fan left a comment

Wendong-Fan Sep 10, 2024

dandansamax Sep 10, 2024

Wendong-Fan Sep 11, 2024

dandansamax left a comment

dandansamax Sep 10, 2024

dandansamax Sep 10, 2024

dandansamax Sep 10, 2024

WHALEEYE commented Sep 10, 2024 •

edited

Loading

dandansamax left a comment

Wendong-Fan left a comment

Feat: Add external tool support to ChatAgent & Refactor #830

Feat: Add external tool support to ChatAgent & Refactor #830

Conversation

WHALEEYE commented Aug 16, 2024 • edited Loading

Description

Motivation and Context

coderabbitai bot commented Aug 16, 2024 • edited Loading

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

Wendong-Fan left a comment

Choose a reason for hiding this comment

WHALEEYE commented Aug 30, 2024 • edited Loading

Wendong-Fan left a comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment

Wendong-Fan Sep 10, 2024

Choose a reason for hiding this comment

dandansamax Sep 10, 2024

Choose a reason for hiding this comment

Wendong-Fan Sep 11, 2024

Choose a reason for hiding this comment

dandansamax left a comment

Choose a reason for hiding this comment

dandansamax Sep 10, 2024

Choose a reason for hiding this comment

dandansamax Sep 10, 2024

Choose a reason for hiding this comment

dandansamax Sep 10, 2024

Choose a reason for hiding this comment

WHALEEYE commented Sep 10, 2024 • edited Loading

dandansamax left a comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment

Feat: Add external tool support to `ChatAgent` & Refactor #830

Feat: Add external tool support to `ChatAgent` & Refactor #830

WHALEEYE commented Aug 16, 2024 •

edited

Loading

coderabbitai bot commented Aug 16, 2024 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)

WHALEEYE commented Aug 30, 2024 •

edited

Loading

WHALEEYE commented Sep 10, 2024 •

edited

Loading