Hi, thank you for your question! The worst-case scenario is running out of tokens when the input exceeds the model's token limit.

To avoid this, you can summarize or extract key points from the text using NLP libraries like NLTK in Python.

For a more efficient solution, I recommend using embeddings. This involves converting the raw data into vector representations, which allows you to query the data directly without sending the entire text.

If speed or simplicity is your primary concern, dividing the comments into smaller chunks is the easiest approach.