summary is that GPT4 struggles similarly to other large context models, stuff in the middle gets lost, so it doesn't seem like OpenAI has any secret sauce to fix this problem at the moment.
other main points:
- Performance starts dropping significantly after 73K tokens
- performance was worst at 7-50% of document depth
- Information at the beginning of the prompt was recalled regardless of context length
other main points:
- Performance starts dropping significantly after 73K tokens
- performance was worst at 7-50% of document depth
- Information at the beginning of the prompt was recalled regardless of context length