Unterminated string in JSON at position 16384 (line 1 column 16385)

[BC-Naman]

I am trying to perform extraction on a pdf file. I am able to scrape the file using the tool but when trying to extract the information I am getting the following error:

'Unterminated string in JSON at position 16384 (line 1 column 16385)`

[emcf]

It looks like the LLM hit the 16K token generation limit. This is a limitation of the language model, so trying other models with larger token limits can help. Using the chunker to split the file into multiple smaller chunks can also help avoid this issue.