"라마의 모든것" 노션 데이터로 스터디장의 업무를 자동화!

    custom_prompt_template = PromptTemplate(
        "당신은 이름은 빅라마 입니다. 라마인덱스 스터디의 스터디장이고, "
        "항상 친절하고 공손한 어조로 질문에 마크다운 형식으로 답변합니다."
        "스터디에 열심히 참여하도록 스터디원들을 응원합니다."
        "다음 정보를 바탕으로 질문에 답변해주세요:\n"
        "만약 답을 모른다면 \"빅라마가 답변하기 힘든 질문입니다. [email protected] 로 연락을 주시면 빠른 답변을 드릴게요! \" 라고 해줘 ."
        "---------------------\n"
        "{context_str}\n"
        "---------------------\n"
        "질문: {query_str}\n"
        "답변: "
    )

"""
Notion Reader Script

This script demonstrates the use of Notion data connector with LlamaIndex.
It retrieves data from a Notion page, creates an index, and saves documents to files.
"""

import os
import json
import logging
from typing import List

from llama_index.core import SummaryIndex, Document
from llama_index.readers.notion import NotionPageReader
from llama_index.core.prompts import PromptTemplate
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core import Settings

# Configure logging
logging.basicConfig(level=logging.DEBUG)  # Changed to DEBUG for more detailed logs
logger = logging.getLogger(__name__)

# Set up environment variables
os.environ['OPENAI_API_KEY'] = '키를 입력해주세요'
NOTION_INTEGRATION_TOKEN = "키를 입력해주세요"
PAGE_IDS = ["페이지 아이디1", "페이지 아이디2"]

def load_notion_documents(integration_token: str, page_ids: List[str]) -> List[Document]:
    """Load documents from Notion pages."""
    reader = NotionPageReader(integration_token=integration_token)
    documents = reader.load_data(page_ids=page_ids)
    logger.debug(f"Loaded {len(documents)} documents from Notion")
    return documents

def create_index(documents: List[Document]) -> SummaryIndex:
    """Create a SummaryIndex from documents."""
    return SummaryIndex.from_documents(documents)

def create_query_engine(index: SummaryIndex) -> RetrieverQueryEngine:
    """Create a custom query engine with a defined prompt template."""
    custom_prompt_template = PromptTemplate(
        "당신은 이름은 빅라마 입니다. 라마인덱스 스터디의 스터디장이고, "
        "항상 친절하고 공손한 어조로 질문에 답변합니다. "
        "스터디에 열심히 참여하도록 스터디원들을 응원합니다."
        "다음 정보를 바탕으로 질문에 답변해주세요:\n"
        "---------------------\n"
        "{context_str}\n"
        "---------------------\n"
        "질문: {query_str}\n"
        "답변: "
    )
    
    retriever = index.as_retriever()
    Settings.llm = Settings.llm or Settings.llm_defaults
    return RetrieverQueryEngine.from_args(
        retriever,
        text_qa_template=custom_prompt_template
    )

def perform_query(query_engine: RetrieverQueryEngine, query: str) -> str:
    """Perform a query and return the response."""
    response = query_engine.query(query)
    return response

def save_documents(documents: List[Document]):
    """Save each document to a separate file and its metadata to a JSON file."""
    for i, doc in enumerate(documents, 1):
        # Save document content
        content_filename = f"document_{i}_content.txt"
        try:
            with open(content_filename, "w", encoding="utf-8") as f:
                f.write(doc.text)
            logger.info(f"Saved document content to {content_filename}")
            logger.debug(f"Document {i} length: {len(doc.text)} characters")
        except Exception as e:
            logger.error(f"Error saving document {i} content: {str(e)}")

        # Save document metadata
        metadata_filename = f"document_{i}_metadata.json"
        try:
            with open(metadata_filename, "w", encoding="utf-8") as f:
                json.dump(doc.metadata, f, ensure_ascii=False, indent=2)
            logger.info(f"Saved document metadata to {metadata_filename}")
        except Exception as e:
            logger.error(f"Error saving document {i} metadata: {str(e)}")

def main():
    """Main function to run the Notion Reader script."""
    documents = load_notion_documents(NOTION_INTEGRATION_TOKEN, PAGE_IDS)
    
    # Log document details before saving
    for i, doc in enumerate(documents, 1):
        logger.debug(f"Document {i} details:")
        logger.debug(f"  Length: {len(doc.text)} characters")
        logger.debug(f"  Metadata: {doc.metadata}")
    
    # Save the loaded documents
    save_documents(documents)
    
    # Verify saved content
    for i, doc in enumerate(documents, 1):
        content_filename = f"document_{i}_content.txt"
        try:
            with open(content_filename, "r", encoding="utf-8") as f:
                saved_content = f.read()
            if len(saved_content) != len(doc.text):
                logger.warning(f"Document {i} content length mismatch: "
                               f"Original {len(doc.text)}, Saved {len(saved_content)}")
        except Exception as e:
            logger.error(f"Error verifying saved content for document {i}: {str(e)}")
    
    index = create_index(documents)
    query_engine = create_query_engine(index)

    queries = [
        "라마 스터디가 뭐야?",
        "스터디 일정을 알려줘!",
        "스터디 주차별 진행사항을 알려줘!",
        "라마 인덱스 공식 문서 링크는 어떻게 돼?"
    ]

    # Uncomment the following lines to perform queries
    for query in queries:
        response = perform_query(query_engine, query)
        print(f"질문: {query}")
        print(f"답변: {response}\n")

if __name__ == "__main__":
    main()

import streamlit as st
import os
import json
import logging
from typing import List

from llama_index.core import SummaryIndex, Document
from llama_index.readers.notion import NotionPageReader
from llama_index.core.prompts import PromptTemplate
from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core import Settings

# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# Set up environment variables
os.environ['OPENAI_API_KEY'] = '키를 입력해주세요'
NOTION_INTEGRATION_TOKEN = "키를 입력해주세요"
PAGE_IDS = ["페이지 아이디 1", "페이지 아이디 2", "페이지 아이디 3"]

@st.cache_resource
def load_notion_documents():
    """Load documents from Notion pages."""
    reader = NotionPageReader(integration_token=NOTION_INTEGRATION_TOKEN)
    documents = reader.load_data(page_ids=PAGE_IDS)
    logger.info(f"Loaded {len(documents)} documents from Notion")
    return documents

@st.cache_resource
def create_index_and_query_engine():
    """Create a SummaryIndex from documents and a query engine."""
    documents = load_notion_documents()
    index = SummaryIndex.from_documents(documents)
    
    custom_prompt_template = PromptTemplate(
        "당신은 이름은 빅라마 입니다. 라마인덱스 스터디의 스터디장이고, "
        "항상 친절하고 공손한 어조로 질문에 답변합니다."
        "스터디에 열심히 참여하도록 스터디원들을 응원합니다."
        "다음 정보를 바탕으로 질문에 답변해주세요:\n"
        "만약 답을 모른다면 \"빅라마가 답변하기 힘든 질문입니다. [email protected] 로 연락을 주시면 빠른 답변을 드릴게요! \" 라고 해줘 ."
        "---------------------\n"
        "{context_str}\n"
        "---------------------\n"
        "질문: {query_str}\n"
        "답변: "
    )
    
    retriever = index.as_retriever()
    Settings.llm = Settings.llm or Settings.llm_defaults
    query_engine = RetrieverQueryEngine.from_args(
        retriever,
        text_qa_template=custom_prompt_template
    )
    
    return query_engine

def perform_query(query_engine: RetrieverQueryEngine, query: str) -> str:
    """Perform a query and return the response."""
    response = query_engine.query(query)
    return str(response)

def main():
    st.title("라마인덱스 스터디 채팅봇")

    # Initialize session state for chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Load documents and create query engine
    query_engine = create_index_and_query_engine()

    # Display chat messages
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # Chat input
    if prompt := st.chat_input("무엇이든 물어보세요!"):
        st.session_state.messages.append({"role": "user", "content": prompt})
        with st.chat_message("user"):
            st.markdown(prompt)

        with st.chat_message("assistant"):
            response = perform_query(query_engine, prompt)
            st.markdown(response)
        st.session_state.messages.append({"role": "assistant", "content": response})

if __name__ == "__main__":
    main()결과 및 인사이트