How to add 'message history' to llama-index based GPT-3 in Python

Question

I am fairly new to using llama-index library for training GPT-3 as well as using ChatGPT through the standard API (both in Python). I have noticed that standard ChatGPT API i could simply do the following code below to have ChatGPT get message history as context:

message_history=[]
completion = openai.ChatCompletion.create(model="gpt-3.5-turbo",messages=message_history)

Now I am using llama-index library to train GPT-3 on a more specific context, however I don't know how to have the model consider the message_history as well, here is a code that I am currently working on an i dont know how to implement message history:


def construct_index(directory_path):
    # set maximum input size
    max_input_size = 4096
    # set number of output tokens
    num_outputs = 2000
    # set maximum chunk overlap
    max_chunk_overlap = 20
    # set chunk size limit
    chunk_size_limit = 600 

    # define prompt helper
    prompt_helper = PromptHelper(max_input_size, num_outputs, max_chunk_overlap, chunk_size_limit=chunk_size_limit)
    # define LLM
    llm_predictor = LLMPredictor(llm=OpenAI(temperature=0.5, model_name="text-ada-001", max_tokens=num_outputs))
    # define context (dataset)
    documents = SimpleDirectoryReader(directory_path).load_data()
    # transform context to index format
    service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor, prompt_helper=prompt_helper)
    index = GPTSimpleVectorIndex.from_documents(documents, service_context=service_context)
    # important: index are are like map, has latitutdes and logntitudes to indicate how each city (texts) are close to each other
    index.save_to_disk("index.json")
    return index

index = GPTSimpleVectorIndex.load_from_disk("index.json")
dbutils.widgets.text("user_input", "user: ")
response = index.query(dbutils.widgets.get("user_input"),response_mode='compact')
print("Response: ", response.response)

score 0 · Answer 1 · answered Jul 24 '23 at 11:17

I had similar problem while using Llama index library to use GPT3.5 Turbo. What solved my problem was ChatMessage built-in object from openai library. I used chatengine to send my query along with my conversation context in a "messages" list of ChatMessage type. Chat Engine documentation can be found here:

https://gpt-index.readthedocs.io/en/latest/examples/chat_engine/chat_engine_openai.html

I stored my system message, query and GPT's response in the messages history to maintain the conversation history.

Try loading your "index.json" file through this:

    storage_context = StorageContext.from_defaults(persist_dir=".")
    index = load_index_from_storage(storage_context)

then store your input like this:

    messages.append(ChatMessage(role="user", content=query))

now query using chatengine() like this:

    chat_engine = index.as_chat_engine()
    response = chat_engine.chat(query, messages)

you will find your response in response.response.

How to add 'message history' to llama-index based GPT-3 in Python

1 Answers1