Skip to content

Integrate SerpApi Google News API - #10773

Open
magentaqin wants to merge 3 commits into
AstrBotDevs:masterfrom
magentaqin:feature/serpapi-news-api
Open

magentaqin wants to merge 3 commits into
AstrBotDevs:masterfrom
magentaqin:feature/serpapi-news-api

Conversation

@magentaqin

@magentaqin magentaqin commented Oct 26, 2026 •

Copy link
Copy Markdown

Modifications / 改动点

Add Google News search support through SerpApi so agents can retrieve current news articles with structured metadata and source links.

  • Migrated shared SerpApi requests to the official Python SDK.
  • Added the news_search_serpapi tool backed by the Google News API.
  • Added article title, publisher, authors, publication time, snippet, thumbnail URL, and source URL to the tool output.
  • Added support for country, language, and result-limit parameters.
  • Registered the tool in agent and chat workflows.
  • Updated configuration metadata, translations, changelog, and documentation.
  • Added unit tests for result mapping, grouped stories, deduplication, validation, SDK responses, and API key rotation.
  • This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

1)automated tests:
Screenshot 2026-10-26 at 07 05 32
2)End-to-end verification
Configured an OpenAI chat model and a SerpApi API key in AstrBot, then sent the following request through the WebUI:

  • Google Search API
Screenshot 2026-10-26 at 07 11 13 - Google News API Screenshot 2026-10-26 at 07 12 39 - Google Image Light API Screenshot 2026-10-26 at 07 14 10 - Google Finance API Screenshot 2026-10-26 at 07 15 15 - Google Scholar API Screenshot 2026-10-26 at 07 16 41 -SerpApi Dashboard Searches Records Screenshot 2026-10-26 at 07 17 56

Checklist / 检查清单

  • 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
    / 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。

  • 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
    / 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”。

  • 📚 I checked the affected WebUI instructions and screenshots in docs/zh and docs/en against the changed navigation, page structure, and labels, and updated them in this PR (or explained why no documentation update is needed). For renamed, moved, or merged entry points, I included an old entry → new entry mapping in the documentation and changelog.
    / 我已对照变化后的 WebUI 入口、页面结构和术语,核对并在本 PR 中更新 docs/zh 和 docs/en 的相关操作说明与截图(或说明无需更新文档的原因)。入口改名、移动或合并时,已在文档和 changelog 中补充 旧入口 → 新入口 对照。

  • 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
    / 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到 requirements.txt 和 pyproject.toml 文件相应位置。

  • 😮 My changes do not introduce malicious code.
    / 我的更改没有引入恶意代码。

Summary by Sourcery

Enable agents to retrieve current, citation-ready news articles through SerpApi Google News.

New Features:

  • Add a SerpApi-backed Google News search tool that returns structured article metadata and source links.
  • Expose Google News search in agent, chat, and citation workflows with country, language, and result-limit options.

Enhancements:

  • Adopt the official SerpApi Python SDK for shared SerpApi requests.

Documentation:

  • Update English and Chinese configuration and web-search documentation to describe SerpApi Google News support.

Tests:

  • Add coverage for Google News result mapping, grouped stories, deduplication, validation, credential handling, retries, and tool registration.

Chores:

  • Update configuration labels, translations, and the unreleased changelog for the new capability.

@magentaqin magentaqin changed the title Integrate SerpApi Google News search Integrate SerpApi Google News API Oct 26, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/tools/web_search_tools.py" line_range="9" />
<code_context>
 from urllib.parse import quote, urlsplit

 import aiohttp
+import serpapi
 from pydantic import Field
 from pydantic.dataclasses import dataclass as pydantic_dataclass
</code_context>
<issue_to_address>
**Installations fail at startup**

When astrBot is installed from requirements.txt rather than resolving dependencies from pyproject.toml, `web_search_tools.py` imports `serpapi` unconditionally, but `requirements.txt` omits it, so startup and smoke-test imports raise `ModuleNotFoundError`.

Add the SerpApi dependency to `requirements.txt` so installations using that manifest can import the search tools.

Also at `pyproject.toml:77`.
</issue_to_address>

### Comment 2
<location path="astrbot/core/tools/web_search_tools.py" line_range="697" />
<code_context>
+                    {
+                        "title": article.get("title") or "",
+                        "url": source_url,
+                        "snippet": article.get("snippet") or "",
+                        "source": source.get("name") or source.get("title") or "",
+                        "authors": source.get("authors") or [],
</code_context>
<issue_to_address>
**News links can hijack the dashboard tab**

When an attacker-controlled publisher URL appears in Google News results and a user clicks its citation, the publisher URL is returned as a citation and the dashboard opens it with `window.open(url, "_blank")` without disabling `window.opener`; a clicked attacker-controlled article can navigate the original AstrBot tab to a phishing page.

Open citation links with `noopener` (and preferably `noreferrer`) or use an anchor with a safe `rel` attribute.
</issue_to_address>

Sourcery assessment

Approval pending. 2 findings to address first.

Blocking findings: astrbot/core/tools/web_search_tools.py:9, astrbot/core/tools/web_search_tools.py:697


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread astrbot/core/tools/web_search_tools.py Outdated
Comment thread astrbot/core/tools/web_search_tools.py
Comment thread astrbot/core/tools/web_search_tools.py Outdated
continue
raise last_error
except (aiohttp.ClientError, asyncio.TimeoutError):
client = serpapi.Client(api_key=api_key, timeout=60)

@Soulter Soulter Oct 26, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, Thanks for your contribution! Is it possible to make requests to SerpApi directly via the HTTP API? This would avoid the need to include a SerpApi python package, helping to keep the main codebase clean.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem! I just updated the code!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants