Call LlamaCPP from Llama-Index – EyeVision

Code

from llama_index.llms import LlamaCPP
# model_url = "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/llama-2-13b-chat.ggmlv3.q4_0.bin"
llm = LlamaCPP(
    temperature=0.1,
    max_new_tokens=256,
    context_window=3900,
    generate_kwargs={},
    verbose=True,
)

response = llm.complete(
    "Hello! Can you tell me a poem about humans and dogs?"
)
print(response)
# print(response.response)
# print("streaming_response")
# streaming_response.print_response_stream()

His Response

Sure, here’s one for you:

Humans and Dogs

Two species, yet so alike,
Both with hearts that beat and souls that strike.

We share our lives, our love, our time,
Together we explore, laugh, and shine.

With tails wagging, furry friends by our side,
We chase away the darkness, and let joy abide.

In a world filled with hate and fear,
Dogs remind us of love, and banish all tears.

Their unconditional love, a gift from above,
A bond that’s strong, a friendship that’s true and pure.

So let us cherish these furry friends,
And honor the love that never ends.

For in their eyes, we see our own worth,
And with them by our side, we find our true self on this earth.