我花了三个月搭了一套自建 RAG 知识库。篇技术文章灌进去，向量库跑起来一切正常。其实，

接下来我用“Redis Cluster 怎么扩容”搜了一下。

返回的结果前三篇分别是：《Redis 为什么快》、《Go 语言的并发模型》、《我们团队的代码规范》。没一篇回答了我的问题，

痛点在这里——

• 技术文档虽多，却因为分块策略失误导致检索命中率低。
• embedding 模型和向量数据库都没问题，却无法精准定位我真正想找的信息。

下面就从分块策略这一步讲起，帮你避免类似痛点。按理说，

三种策略怎么选

不想看完整分析的话，这张表够用了：

为什么分块策略被低估了

大部分 RAG 教程的流程是：装向量库 → 选 embedding 模型 → 调检索参数 → 上线。话说回来，但，分块决定了 embedding 向量的信息密度.

• a chunk 太大。包含多个不相关概念，embedding 就会变成“平均脸”，什么都像。说起来，
• a chunk 太小。信息碎片化，LLM 必须自己拼图。
• 就像数据库索引设计一样，不当的切割等同于把所有资源都放进一个无效索引里。不过，
• A 某些场景下反而击败语义切割与层级切割。说明没有银弹，内容类型决定一切。

结构感知切割主要思路

- 保留父级标题作为上下文前缀；- max_size 是“软上限”，不会在代码或完整概念内部断开。话说回来，

from langchain.text_splitter import RecursiveCharacterTextSplitter
# 多级回退拆分器
fallback_splitter = RecursiveCharacterTextSplitter(
chunk_size=4000,chunk_overlap=200,separators=,length_function=len。)
# 按标题先拆
header_chunks =
for header in headers_to_split_on:
# ...
header_chunks.append
final_chunks =
for chunk in header_chunks:
if len <= 4000:
        final_chunks.append
    else:
        sub_chunks = fallback_splitter.split_text
        for i, sub in enumerate:
            sub_chunk = type(
                page_content=sub,
                metadata={**chunk.metadata, "sub_chunk_index": i}
            )
            final_chunks.append
print} 个 chunk")

This strategy keeps each chunk coherent and context‑rich;I saw hit rates around %80–85 %,a huge jump from naive hard‑cut approach.

模板/段落切割

This method first splits on “## ” titles;if section exceeds max_size it falls back to paragraph boundaries.

from langchain.text_splitter import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
chunk_size=2500,chunk_overlap=100,separators=,length_function=len
)
chunks = splitter.create_documents(
texts=,metadatas=
)
print} 个 chunks")

The hit rate is around %75–80 %,slightly lower than strategy A mainly because it lacks explicit title anchors.

Q&A 对分块

This approach treats every “question‑answer pair” as a single coherent block.

import re
def split_qa_pairs:
pattern = r''
matches = re.findall
qa_pairs=
for match in matches:
parts=re.split'。match,maxsplit=1)
if len==2:
qa_pairs.append({
'question':parts.strip,'answer':parts.strip
})
return qa_pairs


This strategy already boosts hit rates by **~10 %** on FAQ‑heavy sections.

我的迭代过程的观点是，三个版本的教训 









四个主要原则 


保持语义完整性：Avoid slicing mid‑sentence,mid‑paragraph or mid‑code block.

保留上下文锚点：Add article & heading info into metadata so LLM knows source.

内容类型决定策略：Tutorials & docs → A；FAQ/Community → C；Plain text → B 或者语义拆除.

"max_size" 的平衡点：Mistakes happen when too short or too long . Empirically ~4000 Chinese chars works well.
.

Overlap ≈5–10 % of size works as a safety net for edge sentences but avoid>30 %.


综合评分 


Total chunks from my corpus : X;
average per article : Y . 
Li Strategy A covered Z%;B covered W%;C currently only Q%.
 .


Your hit rate jump will be visible even without changing embeddings or rerankers.
.
---

常见问题 

Pain Point:


Q: chunk_overlap 设多少合适？- 学术有人发现 -5 % overlap 在某些 benchmark 上无显著收益，但对关键句边界有保护作用。经验值为 % of size .

Q: Context Cliff 与分块关系？话说回来，- 当单次召回>~100 tokens 或一次性给 LLM>10 chunks 时会触发 Context Cliff。方法是小而精准召回，仅送最相关 ~5–7 chunks 给 LLM.

Q: 中文与英文的差异？说起来，- 中文字符密度低。可将 max_size 调高至 ~4000 字符；英文则约为 700–900 tokens。中文标点应加入 separators 列表。话说回来，

Q: 短文章是否需要拆分？- 若全文 ≤ max_size。可直接作为单个 chunk 入库，仅需保留标题 metadata。

Q: 能否混用三种策略？- 可以但需在入库时标记 content_type，并让文本拆解器根据标签选择对应方法。话说回来，

Q: 如何客观评估效果？其实，- 用真实使用者提问构造测试集。对每种方案计算 Top‑N 检索命中率，并对照实际回答质量评估改进幅度。

💬 *你现在用的是哪种分块策略？按理说，* A 固定硬切 / B 按标题 / C 默认配置 / D 未使用 RAG。请评论，我猜最多有人选 A 和 C 😄

我的建议 


You should first audit your current splitting logic—if it's just hard-cutting by character count,you're likely losing>50 % of potential hits.

If you can only change one thing without re‑ingesting everything:。tweak your splitter – add title context and set a sensible soft limit . You’ll see an instant lift in precision while keeping latency low.

The core idea is simple yet powerful—make each piece self‑contained and contextually rich so that embedding vectors truly reflect meaningful semantic units rar than arbitrary byte blocks.
You’ll often find that biggest gains come from this small adjustment rar than expensive model swaps or database migrations.
If your knowledge base still struggles with recall after this tweak,n consider exploring advanced rerankers or fine‑tuned embeddings—but start with split logic first.
Remember that good splits reduce hallucination because model has clearer context anchors.
Let me know if you’ve tried any of se strategies!




参考资料:


LlamaIndex – Node Parser 模块;Chunking Strategies for RAG;Levels of Text Splitting;Superlinked VectorHub – RAG Chunking Benchmark;Jina AI Late Chunking .
​​ ​ ​ ​ ​ ​ ​ ​ ​


 ​​​​