continuing-cyan•6mo ago

[Solved ✅] Error using composio with an openai compatible API like AIML

Hi, I'm trying to use composio's RAG and file i/o tools with crewai, with a custom backend. I'm defining the base and model as such:

OPENAI_API_BASE='https://api.aimlapi.com/v1'
OPENAI_MODEL_NAME='meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo'

OPENAI_API_BASE='https://api.aimlapi.com/v1'
OPENAI_MODEL_NAME='meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo'

While going through my crew which first embeds some documents and then does RAG query on them, I'm getting the error in the file attached: Could this be because the RAG tool internally uses an embedding model that is not available at this API? I tried switching it to together ai api, but directly got an error related to the ada embedding model not being available. When I switched my backend back to the default openai api, it worked, but while RAG querying, the agent got outputs like:

{'data': {'response': 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASBmkAEgABAHgBDwAHAAAAAAAAAAAABAAIAAAACAAAAAgAAAAIAAAACAAA\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB8kk9P3DAQxe+V+h0i37N2NoBolDWC\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}, 'error': None, 'successful': True}

It seems the response from the RAG database is consistently unreadable, likely due to some technical issue or data corruption. Without the ability to access the database information, it’s challenging to provide a detailed comparison with the benchmark clauses.

{'data': {'response': 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAASBmkAEgABAHgBDwAHAAAAAAAAAAAABAAIAAAACAAAAAgAAAAIAAAACAAA\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB8kk9P3DAQxe+V+h0i37N2NoBolDWC\n\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAA'}, 'error': None, 'successful': True}

It seems the response from the RAG database is consistently unreadable, likely due to some technical issue or data corruption. Without the ability to access the database information, it’s challenging to provide a detailed comparison with the benchmark clauses.

message.txt

20 Replies

sunny-green•6mo ago

Hey, can you show me some glimpses of your code, if okay to you??

continuing-cyanOP•6mo ago

Yes ofc, here you go. This seems to be the relevant part:

tool_set = ComposioToolSet()
rag_tools = tool_set.get_tools(
    actions=[
        Action.FILETOOL_LIST_FILES,
        Action.RAGTOOL_ADD_CONTENT_TO_RAG_TOOL,
        Action.RAGTOOL_RAG_TOOL_QUERY,
    ]
)

rag_query_tools = tool_set.get_tools(actions=[Action.RAGTOOL_RAG_TOOL_QUERY])


### Manager/Scaffolding agents here
corporate_lawyer_agent = Agent(
    role="Corporate Lawyer",
    goal="Use the documents you're given and the RAG tool to build a knowledge base of NDAs that you can refer later. First, check if the documents have already been added.",
    backstory="""You are a corporate lawyer who has vast knowledge of NDAs, different sections within them, and how they are supposed to work.
    You also have the ability to call the RAG tool to ingest new documents that using the paths of files given to you and building a knowledge base of NDAs.""",
    tools=rag_tools,
    verbose=True,
)

ingest_documents_task = Task(
    description="""Ingest benchmark NDAs that will be used as a yardstick to compare NDAs we will judge later.
    Check all the files with NDA in their title in the current directory and ingest all the documents using the RAG tool.""",
    expected_output="Yes or No based on whether you've ingested the documents given to you with the RAG tool.",
    agent=corporate_lawyer_agent,
)

tool_set = ComposioToolSet()
rag_tools = tool_set.get_tools(
    actions=[
        Action.FILETOOL_LIST_FILES,
        Action.RAGTOOL_ADD_CONTENT_TO_RAG_TOOL,
        Action.RAGTOOL_RAG_TOOL_QUERY,
    ]
)

rag_query_tools = tool_set.get_tools(actions=[Action.RAGTOOL_RAG_TOOL_QUERY])


### Manager/Scaffolding agents here
corporate_lawyer_agent = Agent(
    role="Corporate Lawyer",
    goal="Use the documents you're given and the RAG tool to build a knowledge base of NDAs that you can refer later. First, check if the documents have already been added.",
    backstory="""You are a corporate lawyer who has vast knowledge of NDAs, different sections within them, and how they are supposed to work.
    You also have the ability to call the RAG tool to ingest new documents that using the paths of files given to you and building a knowledge base of NDAs.""",
    tools=rag_tools,
    verbose=True,
)

ingest_documents_task = Task(
    description="""Ingest benchmark NDAs that will be used as a yardstick to compare NDAs we will judge later.
    Check all the files with NDA in their title in the current directory and ingest all the documents using the RAG tool.""",
    expected_output="Yes or No based on whether you've ingested the documents given to you with the RAG tool.",
    agent=corporate_lawyer_agent,
)

sunny-green•6mo ago

Have you seen this example: https://docs.composio.dev/examples/combined/rag-agent ??

Composio Docs

RAG Tool Agent - Composio

Composio enables your agents to connect with Various Tools and work with them

continuing-cyanOP•6mo ago

i haven't but let me try that approach. it seems like that example is using a different rag tool and wrapping its llm in the langchain ChatOpenAI module

sunny-green•6mo ago

Okay, lemme know if you get any error from there or any others 🙌

continuing-cyanOP•6mo ago

thank you!!

sunny-green•6mo ago

It was my pleasure to help you 😄

continuing-cyanOP•6mo ago

hey, so i used this and indeed it's working better than the rag tool from the sdk catalogue, but as soon as i switch the api to aiml instead of openai and gpt-4o, i get the model error with the rag query tool that i shared above. is there a way to specify the embedding model for the rag tool? if there isn't this is probably a good feature to suggest to the devs

sunny-green•6mo ago

When you are using tools, it's better to use gpt-4o and gpt-4-turbo otherwise it doesn't even connect with composio many times

continuing-cyanOP•6mo ago

when i use gpt-4o and gpt-4o-turbo from the same provider (aiml) instead of openai, i still get the same error so i'm fairly sure this is not a model issue, but the fact that if you change the provider from openai to something else, maybe the other provider doesn't have the embedding model that the rag tool uses internally

sunny-green•6mo ago

Can you tell me what's the error??

continuing-cyanOP•6mo ago

``````

message.txt

continuing-cyanOP•6mo ago

checked the composio repo, internally the ragtool uses https://docs.embedchain.ai/

Embedchain

⚡ Quickstart - Embedchain

💡 Create an AI app on your own data in a minute

continuing-cyanOP•6mo ago

trying to check what embedding model they use by default my error is possibly related to the fact that that model is not available on the aiml api

sunny-green•6mo ago

Okay lemme see

continuing-cyanOP•6mo ago

the error i'm getting only happens when the ragtool queries the db

sunny-green•6mo ago

Why are you not using OpenAI's one, if asking...??

continuing-cyanOP•6mo ago

working on this hackathon and they've provided a bunch of free credits for these apis, not for openai don't wanna use my own openai for prototyping since it's gonna need a lot of tokens also, tried changing to together ai api, and got this error in the same place:

2024-09-15 13:45:01,977 - 8666926080 - logging.py-logging:85 - INFO: Got response={'data': None, 'error': "Error code: 400 - {'error': {'message': 'Unable to access model text-embedding-ada-002. ', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_available'}}", 'successful': False} from action=<composio.cl...

2024-09-15 13:45:01,977 - 8666926080 - logging.py-logging:85 - INFO: Got response={'data': None, 'error': "Error code: 400 - {'error': {'message': 'Unable to access model text-embedding-ada-002. ', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_available'}}", 'successful': False} from action=<composio.cl...

sunny-green•6mo ago

I am sharing your issue with the team, will notify you as soon as possible.

continuing-cyanOP•6mo ago

thank you! i think the solution is basically just being able to curry the rag tool to specify the embedding model

[Solved ✅] Error using composio with an openai compatible API like AIML

Did you find this page helpful?