feat: add GraphRAG pipeline#17
Open
onemorelight2024 wants to merge 68 commits into
Open
Conversation
- Add flashcard data models and API endpoint - Implement LLM-based flashcard generation service - Create FlashcardGenerator and FlashcardViewer components - Integrate flashcard feature into NotebookView - Add development startup scripts - Update Vite config for remote development Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
Implement comprehensive Quiz functionality similar to NotebookLM: - Single-choice questions with 4 options (A/B/C/D) - Skip functionality for questions - Statistics tracking (Right/Wrong/Skipped) - Review Quiz with highlighted correct answers and detailed explanations - Retake Quiz capability - LLM-based question generation with quality prompts Backend changes: - Add Quiz data models (QuizOption, QuizQuestion, etc.) in schemas.py - Create quiz_service.py for LLM-based question generation - Add /generate-quiz API endpoint in kb.py - Implement JSON parsing with error recovery for truncated responses Frontend changes: - Create QuizGenerator component for quiz creation - Create QuizQuestion component for single question display - Create QuizResults component with circular progress and statistics - Create QuizReview component with answer explanations - Create QuizContainer component as main orchestrator - Integrate Quiz into NotebookView with Brain icon - Update types to include 'quiz' in ToolType Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
feat: Add Quiz feature with LLM-generated questions
…-to-load Add backend read endpoints (list-flashcard-sets, list-quiz-sets, get-flashcard-set, get-quiz-set) and wire frontend outputFeed items to load saved sets from disk, surviving page refresh. Co-Authored-By: Claude <noreply@anthropic.com>
…/flashcard/quiz) - Rewrite /outputs disk fallback to scan notebook-centric paths via get_notebook_paths - Frontend fetchOutputHistory now also calls list-flashcard-sets & list-quiz-sets - mergeOutputFeeds preserves setId field during dedup - Pass notebook_title to /outputs for reliable directory resolution Co-Authored-By: Claude <noreply@anthropic.com>
…tart guide - Rename Chinese image filenames to English for GitHub compatibility - Add screenshots for all current features (12 total, collapsible) - Update feature table: add flashcards, quizzes, web search, deep research - Rewrite Quick Start with clear API config instructions (LLM/Search/Supabase) - Add model configuration section (3-layer system) - Restructure requirements-base.txt with categorized dependencies and version pins Co-Authored-By: Claude <noreply@anthropic.com>
- Fix introduce modal title (zh): remove outdated audio/video reference - Add local embedding server (Octen-Embedding-0.6B) and download script - Add logo assets for both frontends - Update flashcard/quiz UI styling across en/zh frontends - Update Dashboard and NotebookView with latest UI improvements - Add docs guides (context_optimization, kb_source_flow) - Remove obsolete requirements files (paper, win-base) Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix openai version constraint (>=1.104.2,<2.0.0) to match langchain-openai==0.3.33 - Add missing deps: paddlepaddle, paddleocr, mineru_vl_utils, loguru - Align requirements-backup.txt versions to notebook env (100 packages updated) Co-Authored-By: Claude <noreply@anthropic.com>
完整集成 Alibaba DeepResearch 到 Open-NotebookLM 项目，支持双模式研究： - 完整版 DeepResearch：多轮 ReAct 推理，支持工具调用 - 简化版：快速搜索和内容爬取 主要变更： - 添加 DeepResearch 核心模块到 fastapi_app/deep_research/ - 实现 MultiTurnReactAgent 支持多轮推理 - 集成 5 个工具：Search, Visit, Python, Scholar, FileParser - 修改 token 计数使用 tiktoken 替代 HuggingFace tokenizer - 修复 API 调用支持完整 URL 和正确的 API key 传递 - 修复工具环境变量动态读取，避免模块加载时的静态绑定 - 添加 Jina API 错误处理，防止无限重试 - 实现 DeepResearchIntegration 服务类 - 更新 API 端点支持参数传递和双模式切换 - 减少重试次数从 10 次到 3 次 技术细节： - 使用 qwen-agent 框架作为基础 - 支持 Serper API（搜索）和 Jina API（网页访问） - 兼容 OpenAI API 格式的 LLM 服务 - 最大迭代次数可配置（默认 50 次） - Token 上限 110K，超出自动截断 配置支持： - 通过 API 请求传递配置参数 - 通过 .env 文件设置默认值 - 支持自定义 LLM API、搜索 API 和网页访问 API Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
…lt model for generating
feat: 集成阿里巴巴 DeepResearch 深度研究功能
fix: resolve frontend and backend default port conflict
…v3.2 Merged PR OpenDCAI#6 and corrected port references across all docs and configs to use port 8000. Reverted default LLM model change from gpt-4o back to deepseek-v3.2 in frontend_zh. Co-Authored-By: Claude <noreply@anthropic.com>
- ppt_tool.py: 在 ocr_images_to_ppt 和 convert_images_dir_to_pdf_and_ppt_api 两处文本框写入逻辑中，复用已有 is_cjk() 函数，对含中文的文本自动设置 Microsoft YaHei 字体，英文内容保持 Arial - ppt_text_fit.py: TextFitStyle 默认字体从 Arial 改为 Microsoft YaHei， 避免字体拟合时使用不含中文字形的字体 Co-Authored-By: Huangdingcheng <Apollo2023666@outlook.com>
…ana 2 support - Embedding server: auto-pick free GPU via nvidia-smi, fallback to CPU - Main app: skip re-launching embedding subprocess on --reload if already running - DeepResearch: add missing deps (qwen-agent, alibabacloud-docmind, sandbox-fusion, etc.) - DeepResearch: set API_KEY/API_BASE/SUMMARY_MODEL_NAME env vars for visit tool - Frontend: always send search_api_key for all providers (including Serper) - Frontend: show Search API Key input for Serper provider in settings - PPT generation: add Nano Banana 2 image model option - Docs: remove invalid `pip install -e .`, add changelog Co-Authored-By: Claude <noreply@anthropic.com>
…ana 2 support - Embedding server: auto-pick free GPU via nvidia-smi, fallback to CPU - Main app: skip re-launching embedding subprocess on --reload if already running - DeepResearch: add missing deps (qwen-agent, alibabacloud-docmind, sandbox-fusion, etc.) - DeepResearch: set API_KEY/API_BASE/SUMMARY_MODEL_NAME env vars for visit tool - Frontend: always send search_api_key for all providers (including Serper) - Frontend: show Search API Key input for Serper provider in settings - PPT generation: add Nano Banana 2 image model option - Docs: remove invalid `pip install -e .`, add changelog Co-Authored-By: Claude <noreply@anthropic.com>
- Implement block-based editor with drag-and-drop support - Add slash command menu with multiple block types (text, headings, lists, code, etc.) - Integrate AI chat feature for in-note assistance - Support text auto-wrap and Enter key text splitting - Add block operations (add, delete, type conversion) - Implement both English and Chinese versions Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
- Implement block-based editor with drag-and-drop support - Add slash command menu with multiple block types (text, headings, lists, code, etc.) - Integrate AI chat feature for in-note assistance - Support text auto-wrap and Enter key text splitting - Add block operations (add, delete, type conversion) - Implement both English and Chinese versions Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
Major Features: - AI-assisted note editing: polish, rewrite, and source-based AI generation - AI Panel: organize content with presets (summarize, outline, FAQ, etc.) - Text Selection Toolbar: quick AI actions on selected text with diff preview - Memory context: maintain conversation history for better AI responses Bug Fixes: - Fix reversed input issue by replacing contentEditable with textarea - Fix first character deletion issue by consolidating event handling - Fix numbered list indexing for mixed block types - Simplify Backspace logic to avoid conflicts Technical Improvements: - Consolidate onChange/onInput handling to prevent event conflicts - Use textarea.value consistently instead of mixed value/textContent - Support both English and Chinese versions Co-Authored-By: Hunagdingcheng <Apollo6662023@outlook.com>
feat: add Notion-style block editor for notes
- Phase 1: Split env config into .env and .env.models - Phase 2: Remove 19 unused workflows, keep only 3 active - Phase 3: Remove unused TTS adapters, keep Qwen and FireRed - Update settings.py to load both env files - Add TTS config fields to AppSettings
- 补充 Deep Research 相关依赖（qwen-agent, alibabacloud SDK, sandbox-fusion） - 添加 TTS 依赖（qwen-tts, fireredtts2） - 补充数据库客户端依赖（supabase 及相关包） - 添加 sentence-transformers 用于 embedding - 同步更新 requirements-base.txt 和 requirements-backup.txt Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- 日志系统：添加 contextvars 支持，自动追踪 request_id 和用户信息 - 中间件：新增 LoggingMiddleware，自动记录所有请求 - 异常处理：统一错误处理，记录详细日志但返回安全消息 - CORS 修复：APIKeyMiddleware 跳过 OPTIONS 预检请求 - 前端配置：支持 VITE_API_BASE_URL 配置后端地址 - 启动脚本：简化为 start.sh 和 stop.sh，自动清理端口 - 文档更新：README 添加前端配置说明 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
… features - Add guest login functionality to bypass Supabase authentication - Fix numbered lists to preserve original AI-generated numbers (1. 2. 3.) - Clean markdown format symbols (**bold**, *italic*) while preserving content - Support indented bullet list parsing - Add note editing capability with Edit/Delete buttons in output list - Add timestamp to filenames to prevent same-name file conflicts - Add 1-second delay for file list refresh after saving Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
- Add guest login functionality to Chinese frontend - Add Edit/Delete buttons for notes in Chinese version - Add note icon display in output feed - Support note editing with markdown loading Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
- 后端：改为chunk级别的引用映射，每个检索片段独立编号 - 前端：优化性能，从state读取引用数据而非DOM属性 - 前端：优化tooltip显示效果（字体、行高、滚动条） - 前端：全面支持LaTeX公式 \(...\) 和 \[...\] 格式 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- 后端配置更新 - 前端功能完善 - 工作流优化 - 添加客户端缓存服务 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fix provider detection for Gemini TTS models - Update frontend voice defaults from 'vivian' to 'Kore' - Add voice name validation in ApiYiGeminiProvider - Fix file_ids attribute error in podcast workflow - Improve mindmap generation error handling Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
- Accept main's note editor improvements - Keep TTS voice validation fixes from this branch
This reverts commit 7ba068e.
Major Features: - AI-assisted note editing: polish, rewrite, and source-based AI generation - AI Panel: organize content with presets (summarize, outline, FAQ, etc.) - Text Selection Toolbar: quick AI actions on selected text with diff preview - Memory context: maintain conversation history for better AI responses Bug Fixes: - Fix reversed input issue by replacing contentEditable with textarea - Fix first character deletion issue by consolidating event handling - Fix numbered list indexing for mixed block types - Simplify Backspace logic to avoid conflicts Technical Improvements: - Consolidate onChange/onInput handling to prevent event conflicts - Use textarea.value consistently instead of mixed value/textContent - Support both English and Chinese versions Co-Authored-By: Hunagdingcheng <Apollo6662023@outlook.com>
… features - Add guest login functionality to bypass Supabase authentication - Fix numbered lists to preserve original AI-generated numbers (1. 2. 3.) - Clean markdown format symbols (**bold**, *italic*) while preserving content - Support indented bullet list parsing - Add note editing capability with Edit/Delete buttons in output list - Add timestamp to filenames to prevent same-name file conflicts - Add 1-second delay for file list refresh after saving Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
- 后端：实现完整的 Supabase 认证代理 API（登录/注册/登出/会话刷新） - 后端：使用 HTTP-only cookies 存储访问令牌，提升安全性 - 后端：auth 端点从 API key 验证中排除 - 前端：移除直接 Supabase 连接，改用后端代理 API - 前端：修复 401 响应后的加载状态问题 - 前端：优化会话刷新逻辑 - MinerU：启用 vLLM 模式，配置 GPU 利用率（TTS 65% + MinerU 30%） - MinerU：修复端口配置，从环境变量读取 - 脚本：更新启动脚本的 GPU 配置 - 测试：添加 Supabase 连接测试脚本 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- 同步 App.tsx, supabase.ts, authStore.ts 到英文版 - 同步 vite.config.ts 和 NotebookView.tsx - 确保中英文前端功能一致 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- 添加后端 OTP 验证和重发端点 - 前端实现 OTP 验证功能 - 登录/注册/验证成功时自动创建用户目录 - 优化速率限制错误提示 - 添加启动脚本 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ving szl-dev behavior
…ing szl-dev behavior
…tation - Replace Chinese screenshots with English versions in README.md - Add note feature documentation with screenshots (both EN and ZH versions) - Enhance README layout with GitHub Stars badge and quick navigation - Improve core features section with two-column layout Co-Authored-By: Huangdingcheng <Apollo6662023@outlook.com>
docs: Update README with English screenshots and note feature documentation
…lit subgraph/Wikidata pipeline
…lit subgraph/Wikidata pipeline
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.