bug: bot subscription uses removed GraphQL fields

[flexus-teams]

Original Logs

20260504 21:09:04.103 fclnt [INFO] FlexusClient service_name=bob_30005_r_ api_key=None http://backend-v1-service.flexus.svc/v1/jailed-bot
20260504 21:09:04.104 stexe [INFO] Connecting ws://backend-v1-service.flexus.svc/v1/jailed-bot
20260504 21:09:04.144 btexe [ERROR] 🛑 That looks bad, my key doesn't work: {'message': "403: Whoops your key didn't work (2).", 'locations': [{'line': 2, 'column': 3}], 'path': ['bot_confirm_exists']}
...
20260504 21:11:04.355 stexe [ERROR] 🛑 3 exceptions in 5 min, exiting
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/flexus_client_kit/ckit_service_exec.py", line 27, in run_typical_single_subscription_with_restart_on_network_errors
    await subscribe_and_do_something(fclient, ws_client, *func_args, **func_kwargs)
  File "/usr/local/lib/python3.11/site-packages/flexus_client_kit/ckit_bot_exec.py", line 390, in subscribe_and_produce_callbacks
    async for r in ws.subscribe(
  File "/usr/local/lib/python3.11/site-packages/gql/client.py", line 1426, in subscribe
    async for result in inner_generator:
  File "/usr/local/lib/python3.11/site-packages/gql/client.py", line 1337, in _subscribe
    async for result in inner_generator:
  File "/usr/local/lib/python3.11/site-packages/gql/transport/common/base.py", line 298, in subscribe
    answer_type, execution_result = await listener.get()
                                    ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/gql/transport/common/listener_queue.py", line 35, in get:
    raise item
  File "/usr/local/lib/python3.11/site-packages/gql/transport/common/base.py", line 221, in _receive_data_loop
    answer_type, answer_id, execution_result = self._parse_answer(
                                               ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/gql/transport/websockets_protocol.py", line 423, in _parse_answer
    return self._parse_answer_apollo(json_answer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/gql/transport/websockets_protocol.py", line 384, in _parse_answer_apollo
    raise TransportQueryError(
TransportQueryError: {'message': "Unknown argument 'want_emsg_types' on field 'Subscription.bot_threads_calls_tasks'. Did you mean 'want_erp_tables' or 'want_messages'?", 'locations': [{'line': 12, 'column': 5}]}

Error Summary

Observed in bob_30005 and botticelli_100020157 in namespace isolated. The bots crash-loop after repeated subscription errors against backend-v1-service.flexus.svc.

Stacktrace

The recurring hard failure is gql.transport.exceptions.TransportQueryError from ckit_bot_exec.py:390 when subscribing to Subscription.bot_threads_calls_tasks.

Root Cause

• File: flexus_client_kit/ckit_bot_exec.py:390
• Function: subscribe_and_produce_callbacks
• Why: the running client is still querying removed GraphQL fields/arguments (want_emsg_types, news_payload_task) while the backend schema now exposes news_payload_task_new/news_payload_task_old and does not accept want_emsg_types.
• Git blame: recent bot/client changes show this subscription path is actively evolving.

Code Snippet

@dataclass
class FBotThreadsCallsTasks:
    news_action: str
    news_about: str
    news_payload_id: str
    news_payload_thread_message: Optional[ckit_ask_model.FThreadMessageOutput]
    news_payload_thread: Optional[ckit_ask_model.FThreadOutput]
    news_payload_persona: Optional[FPersonaOutput]
    news_payload_toolcall: Optional[ckit_cloudtool.FCloudtoolCall]
    news_payload_task_new: Optional[ckit_kanban.FPersonaKanbanTaskOutput] = None
    news_payload_task_old: Optional[ckit_kanban.FPersonaKanbanTaskOutput] = None

Affected

• Pods: flexus-pod-bot-bob-30005-rx, flexus-pod-bot-botticelli-100020157-rx
• Namespaces: isolated
• Occurrences: repeated on every reconnect

[flexus-teams]

Also observed in botticelli_100020157 on isolated. Current logs show the startup subscription querying removed GraphQL field news_payload_task on FBotThreadsCallsTasks, causing repeated TransportQueryError and CrashLoopBackOff. This looks like the same schema mismatch family affecting multiple bots.