The embedded mini-app ecosystem is a real model in China. Wechat being the most popular platform. Douyin also has a good share of it. You even see such ecosystem in payment apps like Alipay. I'm also certain there are special deals with Apple structured around this concept.
Using a massive context window is akin to onboarding a new employee before every mundane task performed. While a trained employee will take new task easily with existing context.
The trade off is simply cost. The cost remains in the LLM scene with regards to speed of execution and token cost.