adverse-sapphire•5d ago

Error using composio with an openai compatible API like AIML

Hi, I'm trying to use composio's RAG and file i/o tools with crewai, with a custom backend. I'm defining the base and model as such:

OPENAI_API_BASE='https://api.aimlapi.com/v1'
OPENAI_MODEL_NAME='meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo'

OPENAI_API_BASE='https://api.aimlapi.com/v1'
OPENAI_MODEL_NAME='meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo'

While going through my crew which first embeds some documents and then does RAG query on them, I'm getting the error in the file attached: Could this be because the RAG tool internally uses an embedding model that is not available at this API? I tried switching it to together ai api, but directly got an error related to the ada embedding model not being available. When I switched my backend back to the default openai api, it worked, but while RAG querying, the agent got outputs like:

{'data': {'response': 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASBmkAEgABAHgBDwAHAAAAAAAAAAAABAAIAAAACAAAAAgAAAAIAAAACAAA\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB8kk9P3DAQxe+V+h0i37N2NoBolDWC\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}, 'error': None, 'successful': True}

It seems the response from the RAG database is consistently unreadable, likely due to some technical issue or data corruption. Without the ability to access the database information, it’s challenging to provide a detailed comparison with the benchmark clauses.

{'data': {'response': 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASBmkAEgABAHgBDwAHAAAAAAAAAAAABAAIAAAACAAAAAgAAAAIAAAACAAA\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB8kk9P3DAQxe+V+h0i37N2NoBolDWC\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}, 'error': None, 'successful': True}

It seems the response from the RAG database is consistently unreadable, likely due to some technical issue or data corruption. Without the ability to access the database information, it’s challenging to provide a detailed comparison with the benchmark clauses.

message.txt

20 Replies

fascinating-indigo•5d ago

Hey, can you show me some glimpses of your code, if okay to you??

adverse-sapphire•5d ago

Yes ofc, here you go. This seems to be the relevant part:

tool_set = ComposioToolSet()
rag_tools = tool_set.get_tools(
    actions=[
        Action.FILETOOL_LIST_FILES,
        Action.RAGTOOL_ADD_CONTENT_TO_RAG_TOOL,
        Action.RAGTOOL_RAG_TOOL_QUERY,
    ]
)

rag_query_tools = tool_set.get_tools(actions=[Action.RAGTOOL_RAG_TOOL_QUERY])


### Manager/Scaffolding agents here
corporate_lawyer_agent = Agent(
    role="Corporate Lawyer",
    goal="Use the documents you're given and the RAG tool to build a knowledge base of NDAs that you can refer later. First, check if the documents have already been added.",
    backstory="""You are a corporate lawyer who has vast knowledge of NDAs, different sections within them, and how they are supposed to work.
    You also have the ability to call the RAG tool to ingest new documents that using the paths of files given to you and building a knowledge base of NDAs.""",
    tools=rag_tools,
    verbose=True,
)

ingest_documents_task = Task(
    description="""Ingest benchmark NDAs that will be used as a yardstick to compare NDAs we will judge later.
    Check all the files with NDA in their title in the current directory and ingest all the documents using the RAG tool.""",
    expected_output="Yes or No based on whether you've ingested the documents given to you with the RAG tool.",
    agent=corporate_lawyer_agent,
)

tool_set = ComposioToolSet()
rag_tools = tool_set.get_tools(
    actions=[
        Action.FILETOOL_LIST_FILES,
        Action.RAGTOOL_ADD_CONTENT_TO_RAG_TOOL,
        Action.RAGTOOL_RAG_TOOL_QUERY,
    ]
)

rag_query_tools = tool_set.get_tools(actions=[Action.RAGTOOL_RAG_TOOL_QUERY])


### Manager/Scaffolding agents here
corporate_lawyer_agent = Agent(
    role="Corporate Lawyer",
    goal="Use the documents you're given and the RAG tool to build a knowledge base of NDAs that you can refer later. First, check if the documents have already been added.",
    backstory="""You are a corporate lawyer who has vast knowledge of NDAs, different sections within them, and how they are supposed to work.
    You also have the ability to call the RAG tool to ingest new documents that using the paths of files given to you and building a knowledge base of NDAs.""",
    tools=rag_tools,
    verbose=True,
)

ingest_documents_task = Task(
    description="""Ingest benchmark NDAs that will be used as a yardstick to compare NDAs we will judge later.
    Check all the files with NDA in their title in the current directory and ingest all the documents using the RAG tool.""",
    expected_output="Yes or No based on whether you've ingested the documents given to you with the RAG tool.",
    agent=corporate_lawyer_agent,
)

fascinating-indigo•5d ago

Have you seen this example: https://docs.composio.dev/examples/combined/rag-agent ??

Composio Docs

RAG Tool Agent - Composio

Composio enables your agents to connect with Various Tools and work with them

adverse-sapphire•5d ago

i haven't but let me try that approach. it seems like that example is using a different rag tool and wrapping its llm in the langchain ChatOpenAI module

fascinating-indigo•5d ago

Okay, lemme know if you get any error from there or any others 🙌

adverse-sapphire•5d ago

thank you!!

fascinating-indigo•5d ago

It was my pleasure to help you 😄

adverse-sapphire•5d ago

hey, so i used this and indeed it's working better than the rag tool from the sdk catalogue, but as soon as i switch the api to aiml instead of openai and gpt-4o, i get the model error with the rag query tool that i shared above. is there a way to specify the embedding model for the rag tool? if there isn't this is probably a good feature to suggest to the devs

fascinating-indigo•5d ago

When you are using tools, it's better to use gpt-4o and gpt-4-turbo otherwise it doesn't even connect with composio many times

adverse-sapphire•5d ago

when i use gpt-4o and gpt-4o-turbo from the same provider (aiml) instead of openai, i still get the same error so i'm fairly sure this is not a model issue, but the fact that if you change the provider from openai to something else, maybe the other provider doesn't have the embedding model that the rag tool uses internally

fascinating-indigo•5d ago

Can you tell me what's the error??

adverse-sapphire•5d ago

``````

message.txt

adverse-sapphire•5d ago

checked the composio repo, internally the ragtool uses https://docs.embedchain.ai/

Embedchain

⚡ Quickstart - Embedchain

💡 Create an AI app on your own data in a minute

adverse-sapphire•5d ago

trying to check what embedding model they use by default my error is possibly related to the fact that that model is not available on the aiml api

fascinating-indigo•5d ago

Okay lemme see