System Prompt Template

[MarcusWey]

Good day, everyone. I am relatively new to Langflow and currently at the stage of experimenting with different open-source models available on Ollama. I noticed that different models seem to require different formats for prompts, or perhaps I misunderstand how prompt templates work.

For example, with OpenAI (before trying Ollama models), I would assign it a role, provide instructions, input variables, and it worked quite well.

OpenAI prompt example:

User: {user_input}

You are a professional data analyst proficient in analyzing tabular data. You will answer all requests made by the user if they are related to data.

However, when it comes to longer prompts, I found that models like Llama 3.1 8B and Gemma 2:2B don’t fully understand my instructions. They tend to ignore the instructions and restrictions I set but can still fetch website documents through the website loader component. After reading their official documentation, I discovered they use elements like <start_of_turn>user, <start_of_turn>model, and so on. However, I still don't fully understand these elements, as my recent attempts to use them haven’t yielded the desired output. Anyone can explain this for me?

[dosubot[bot]]

Hello @MarcusWey! 👋 I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this together!

The elements like <start_of_turn>user and <start_of_turn>model are used to structure the conversation in a way that the model can better understand the context and roles of the participants. These elements help in delineating the turns in a conversation, making it clear when the user is speaking and when the model should respond.

Here's a breakdown of how you can use these elements in your prompt templates for models like Llama 3.1 8B and Gemma 2:2B:

1. <start_of_turn>user: This marks the beginning of the user's input.
2. <start_of_turn>model: This marks the beginning of the model's response.

Example Prompt Template

<start_of_turn>user
User: {user_input}

<start_of_turn>model
You are a professional data analyst proficient in analyzing tabular data. You will answer all requests made by the user if they are related to data.

Detailed Example

If you want to create a more detailed prompt that includes instructions and context, you can structure it as follows:

<start_of_turn>user
User: {user_input}

<start_of_turn>model
You are a professional data analyst proficient in analyzing tabular data. You will answer all requests made by the user if they are related to data. Please ensure your responses are concise and accurate.

<start_of_turn>user
User: Can you analyze this dataset for trends?

<start_of_turn>model
Sure, please provide the dataset, and I will analyze it for trends.

Tips for Effective Prompts

• Clarity: Make sure your instructions are clear and concise.
• Context: Provide enough context for the model to understand the task.
• Consistency: Use the <start_of_turn> elements consistently to mark the user's and model's turns.

By structuring your prompts this way, you can help the models better understand the context and follow the instructions more accurately. If you continue to face issues, you might need to experiment with different prompt structures or consult the specific documentation for the models you are using.

To continue talking to Dosu, mention @dosu.

[dosubot[bot]]

To integrate the TinyLlama prompt template into Langflow, you can use the process_prompt_template function provided in the api_utils.py file. This function processes and validates the prompt template, updates the template, and manages custom fields.

Here is an example of how you can use this function:

from collections import defaultdict
from typing import Any, Dict, List, Optional

# Assuming the necessary imports and definitions are already in place

# Define your TinyLlama prompt template
tiny_llama_template = """
You are an AI language model serving as a professional data analyst working.
Your name is Little Llama Assistant.
You are proficient in analyzing tabular data and delivering structured insights.
You are also the expert of AI fundamentals, Industry 4.0, and mechanical engineering.

Memory: {memory}

User: {input}
AI:
"""

# Define the name for the template
template_name = "TinyLlamaTemplate"

# Initialize custom fields if needed
custom_fields = None  # or provide a dictionary if you have existing custom fields

# Define the frontend node template
frontend_node_template = {}

# Process the TinyLlama prompt template
input_variables = process_prompt_template(
    template=tiny_llama_template,
    name=template_name,
    custom_fields=custom_fields,
    frontend_node_template=frontend_node_template
)

# The input_variables now contain the validated input variables from the TinyLlama template
# The frontend_node_template is updated with the new variables

This code snippet demonstrates how to integrate the TinyLlama prompt template into Langflow by processing it with the process_prompt_template function. The function handles validation, updating the template, and managing custom fields, ensuring that your prompt template is correctly integrated into the Langflow environment ^[1][2].

To continue talking to Dosu, mention @dosu.

[MarcusWey]

@dosu Integrate the code inside the prompt's code?

from langflow.base.prompts.api_utils import process_prompt_template
from langflow.custom import Component
from langflow.inputs.inputs import DefaultPromptField
from langflow.io import Output, PromptInput
from langflow.schema.message import Message
from langflow.template.utils import update_template_values


class PromptComponent(Component):
    display_name: str = "Prompt"
    description: str = "Create a prompt template with dynamic variables."
    icon = "prompts"
    trace_type = "prompt"
    name = "Prompt"

    inputs = [
        PromptInput(name="template", display_name="Template"),
    ]

    outputs = [
        Output(display_name="Prompt Message", name="prompt", method="build_prompt"),
    ]

    async def build_prompt(
        self,
    ) -> Message:
        prompt = await Message.from_template_and_variables(**self._attributes)
        self.status = prompt.text
        return prompt

    def _update_template(self, frontend_node: dict):
        prompt_template = frontend_node["template"]["template"]["value"]
        custom_fields = frontend_node["custom_fields"]
        frontend_node_template = frontend_node["template"]
        _ = process_prompt_template(
            template=prompt_template,
            name="template",
            custom_fields=custom_fields,
            frontend_node_template=frontend_node_template,
        )
        return frontend_node

    def post_code_processing(self, new_frontend_node: dict, current_frontend_node: dict):
        """
        This function is called after the code validation is done.
        """
        frontend_node = super().post_code_processing(new_frontend_node, current_frontend_node)
        template = frontend_node["template"]["template"]["value"]
        # Kept it duplicated for backwards compatibility
        _ = process_prompt_template(
            template=template,
            name="template",
            custom_fields=frontend_node["custom_fields"],
            frontend_node_template=frontend_node["template"],
        )
        # Now that template is updated, we need to grab any values that were set in the current_frontend_node
        # and update the frontend_node with those values
        update_template_values(new_template=frontend_node, previous_template=current_frontend_node["template"])
        return frontend_node

    def _get_fallback_input(self, **kwargs):
        return DefaultPromptField(**kwargs)
        

    # Define your TinyLlama prompt template
    tiny_llama_template = """
    You are an AI language model serving as a professional data analyst working.
    Your name is Little Llama Assistant.
    You are proficient in analyzing tabular data and delivering structured insights.
    You are also the expert of AI fundamentals, Industry 4.0, and mechanical engineering.
    
    Memory: {memory}
    
    User: {input}
    AI:
    """
    
    # Define the name for the template
    template_name = "TinyLlamaTemplate"
    
    # Initialize custom fields if needed
    custom_fields = None  # or provide a dictionary if you have existing custom fields
    
    # Define the frontend node template
    frontend_node_template = {}
    
    # Process the TinyLlama prompt template
    input_variables = process_prompt_template(
        template=tiny_llama_template,
        name=template_name,
        custom_fields=custom_fields,
        frontend_node_template=frontend_node_template
    )
```

[dosubot[bot]]

To integrate the TinyLlama prompt template into the provided Langflow prompt's code, you need to modify the process_prompt_template function to include the TinyLlama prompt template. Here is how you can do it:

1. Define the TinyLlama prompt template: You need to have the TinyLlama prompt template defined somewhere in your code. For this example, let's assume it's a string.

2. Modify the process_prompt_template function: Update the function to include the TinyLlama prompt template.

Here is the updated code:

from langflow.base.prompts.api_utils import process_prompt_template
from langflow.custom import Component
from langflow.inputs.inputs import DefaultPromptField
from langflow.io import Output, PromptInput
from langflow.schema.message import Message
from langflow.template.utils import update_template_values

class PromptComponent(Component):
    display_name: str = "Prompt"
    description: str = "Create a prompt template with dynamic variables."
    icon = "prompts"
    trace_type = "prompt"
    name = "Prompt"

    inputs = [
        PromptInput(name="template", display_name="Template"),
    ]

    outputs = [
        Output(display_name="Prompt Message", name="prompt", method="build_prompt"),
    ]

    async def build_prompt(
        self,
    ) -> Message:
        prompt = await Message.from_template_and_variables(**self._attributes)
        self.status = prompt.text
        return prompt

    def _update_template(self, frontend_node: dict):
        prompt_template = frontend_node["template"]["template"]["value"]
        custom_fields = frontend_node["custom_fields"]
        frontend_node_template = frontend_node["template"]
        
        # Integrate TinyLlama prompt template
        tiny_llama_template = """
        You are an AI language model serving as a professional data analyst working.
        Your name is Little Llama Assistant.
        You are proficient in analyzing tabular data and delivering structured insights.
        You are also the expert of AI fundamentals, Industry 4.0, and mechanical engineering.
        
        Memory: {memory}
        
        User: {input}
        AI:
        """
        prompt_template = tiny_llama_template + prompt_template

        _ = process_prompt_template(
            template=prompt_template,
            name="template",
            custom_fields=custom_fields,
            frontend_node_template=frontend_node_template,
        )
        return frontend_node

    def post_code_processing(self, new_frontend_node: dict, current_frontend_node: dict):
        """
        This function is called after the code validation is done.
        """
        frontend_node = super().post_code_processing(new_frontend_node, current_frontend_node)
        template = frontend_node["template"]["template"]["value"]
        # Kept it duplicated for backwards compatibility
        _ = process_prompt_template(
            template=template,
            name="template",
            custom_fields=frontend_node["custom_fields"],
            frontend_node_template=frontend_node["template"],
        )
        # Now that template is updated, we need to grab any values that were set in the current_frontend_node
        # and update the frontend_node with those values
        update_template_values(new_template=frontend_node, previous_template=current_frontend_node["template"])
        return frontend_node

    def _get_fallback_input(self, **kwargs):
        return DefaultPromptField(**kwargs)

In this code, the tiny_llama_template variable contains the TinyLlama prompt template. The prompt_template variable is then updated to include the TinyLlama prompt template before proceeding with the rest of the function ^{[1][2][[3]](https://github.com/langflow-ai/langflow/blob/main/src/backend/base/langflow/initial_setup/starter_projects/Hierarchical Agent.json)}.

To continue talking to Dosu, mention @dosu.

[MarcusWey]

@dosu after integrate the code do i need to write anything inside the prompt template?

[dosubot[bot]]

Yes, you need to write the prompt template inside the "template" field of the frontend_node dictionary. The template field is used in the _update_template and post_code_processing methods to process and update the prompt template with dynamic variables.

Here is the relevant code snippet:

def _update_template(self, frontend_node: dict):
    prompt_template = frontend_node["template"]["template"]["value"]
    custom_fields = frontend_node["custom_fields"]
    frontend_node_template = frontend_node["template"]
    _ = process_prompt_template(
        template=prompt_template,
        name="template",
        custom_fields=custom_fields,
        frontend_node_template=frontend_node_template,
    )
    return frontend_node

def post_code_processing(self, new_frontend_node: dict, current_frontend_node: dict):
    frontend_node = super().post_code_processing(new_frontend_node, current_frontend_node)
    template = frontend_node["template"]["template"]["value"]
    _ = process_prompt_template(
        template=template,
        name="template",
        custom_fields=frontend_node["custom_fields"],
        frontend_node_template=frontend_node["template"],
    )
    update_template_values(new_template=frontend_node, previous_template=current_frontend_node["template"])
    return frontend_node

In these methods, the template field is accessed and processed, indicating that you need to provide the prompt template content there.

For example, if you are building a chatbot with the SystemMessagePromptTemplate and HumanMessagePromptTemplate components, you might set the "Prompt" field on the SystemMessagePromptTemplate component to something like:

You are a {role} that {behavior}.

And for the HumanMessagePromptTemplate component, you might set the "Prompt" field to:

{input}

This setup allows you to define dynamic variables such as 'role' and 'behavior' that can be used to customize the chatbot's responses.

To further illustrate, here are step-by-step instructions from the Langflow documentation on how to set up these templates:

1. Navigate to the "Community Examples" section.
2. Locate the "Basic Chat with Prompt and History" example, and click on "Fork Example".
3. In the editor, open the "Template" field.
4. Here, introduce the {context} variable, placing it somewhere before the "Current conversation:" text.
5. Once done, save your changes by clicking on "Check & Save".
6. Next, open the search bar and type "web".
7. Drag and drop a WebBaseLoader (or any other loader of your choice) onto the canvas.
8. Connect this loader to the {context} variable that we just added.
9. In the "Web Page" field, enter "https://docs.langflow.org/how-upload-examples".
10. Now, click on "ConversationBufferMemory".
11. In the "Input Key" field, enter "text" to define the Chat variable.
12. Click on the "Build" button to implement your changes.
13. Open the chat interface to test your modifications.
14. Try asking something like, "How do I upload examples?"
15. Click on "Display Prompt" to view your template.
16. Now, you can see what the model used as a basis to generate its response.

By following these instructions, you can create a dynamic chatbot capable of understanding and responding based on custom system messages, enhancing the user experience and interaction ^{[1][2][3][[4]](https://github.com/langflow-ai/langflow/blob/main/src/backend/base/langflow/initial_setup/starter_projects/Blog Writer.json)[5]}.

To continue talking to Dosu, mention @dosu.