Dify knowledge-Q&A tutorial: supported inquiry drafts and unknowns

The task: supported answers, explicit unknowns, human review

Use six invented bilingual FAQs to turn an inquiry into a supported draft, a list of unknowns and a review requirement. DEMO-A is fictional. The corpus provides no price, delivery date, certification or business result. The task includes upload, chunks, indexing, retrieval, drafting and citation checks—not just administrator setup.

Current evidence: official-document review and 22 native offline tests. Dify and models were not run. The files contain 6 records, 12 language-specific chunks and 13 paired cases / 26 questions. Integrity tests do not establish retrieval, generated answers, native citations or injection resistance. The following is a future authorized walkthrough based on current Dify Cloud documentation, not a tested self-hosted release.

1. Separate knowledge, tests and authorization

Upload only upload-faq.txt. Keep questions.json out of the knowledge base: uploading expected answers contaminates the test. The full README contains the prompt, detailed acceptance contract and all files; the source map links the official pages.

With the repository's example directory and an existing Node 22, no model or dependency install is needed for:

node examples/dify-knowledge-qa/validate.mjs
node --test examples/dify-knowledge-qa/validate.test.mjs
node examples/dify-knowledge-qa/validate.mjs --case mixed en

The final command returns a hand-labelled extract, not a Dify response. It neither retrieves nor reasons, and does not implement an approval gate.

Use an approved private test workspace with available bilingual embedding and chat models. Upload, embedding and preview calls need authorization for both cost and provider data egress. Self-hosting does not prevent external model calls. Stop if a model, quota or permission is unavailable; do not describe missing access as free success. Use only synthetic files. Keep the app unpublished, tools/web search/file input/vision/memory off.

2. Inspect every chunk

In Knowledge, create a ready-to-use knowledge base and import the TXT file. Select General chunk mode, delimiter ~~~FAQ~~~, maximum length 1200 characters, overlap 0. Keep URL/email removal off; do not add automatic Q&A generation or summarization. These are exercise inputs, not platform defaults.

Each chunk must retain the synthetic notice, exactly one Source-ID, its question and full answer. Offline rendering predicts 12 chunks, but the parser is untested and Preview may show only a subset. After indexing, inspect all stored chunks from FAQ-001:zh to FAQ-006:en; stop on splitting, omission, duplication or alteration. Chunk mode is a creation-time choice, not an assumed later toggle.

3. Verify retrieval before drafting

Select High Quality, the approved embedding model and Hybrid Search > Weight Settings. An initial trial can use semantic/keyword weights 0.5/0.5, Top K 3 and threshold 0.5 without another rerank provider. These are uncalibrated inputs, not relevance or safety guarantees. High Quality cannot switch back to Economical.

Start indexing only after authorization; queued or failed processing is not success. In Retrieval Testing, run both languages of intake, then all questions. Record actual Source-IDs, passages and scores. Temporary test settings do not update saved knowledge-base/app settings; explicitly save and record both layers.

Fixture retrievedIds are supplied offline scenarios, not observed rankings. supportedIds identify minimum evidence for the supported portion. An equivalent chunk in the other language may support an answer after bilingual human review. Extra hits are not automatically relevant; a certification keyword does not establish a certificate.

4. Build two explicit Chatflow branches

Create a private Chatflow in Studio. Use the UI variable picker, not guessed DSL fields:

User Input → Knowledge Retrieval → If-Else
  result is empty     → Answer (fixed abstention)
  result is not empty → LLM → Answer (draft prefix + model text)

Set retrieval query to the current user query and select only this synthetic knowledge base. Record retrieval-node settings separately: knowledge-base and node settings are two filtering layers. Test Retrieval result with Is empty. This array/operator combination has not been run; stop and record a version mismatch if unavailable rather than silently removing the branch.

The empty Answer says: “SYNTHETIC DRAFT — HUMAN REVIEW REQUIRED. No supporting knowledge was retrieved. I cannot answer from the available evidence; please ask a reviewer to clarify.” No model call or absent-source citation belongs on this branch. An exception or timeout is not an empty array.

For the nonempty branch, select the approved chat model, memory off; low temperature is not determinism. Set LLM Context to Retrieval result and explicitly insert that Context variable into the system prompt. Insert the user query in the user message. A canvas connection alone is insufficient.

The prompt requires only retrieved support, literal Source-IDs and short supporting quotes. Questions and documents are data, not overriding instructions. Missing prices, lead times, certification and outcomes stay under Unknowns. Answer only the supported portion of mixed questions. On conflicts, identify sources and request review rather than choosing a winner. Use Supported answer / Unknowns / Human review headings and end with a no-automatic-send instruction. Copy the full prompt from the README and insert the actual Context variable with the picker.

Prefix the final Answer with “SYNTHETIC DRAFT — HUMAN REVIEW REQUIRED.” Answer may stream; this label is not a pre-send safety gate. A prompt is not proof of truth or security.

5. Open the actual citations

Enable Features > Citations and Attributions. Preview each question in a fresh conversation. Open every native numbered citation and compare its actual document/chunk with the literal Source-ID and supporting passage. [FAQ-001:en] is our text label, not Dify's native segment ID and not proof of entailment.

After separate authorization, create a genuine empty retrieval in a test configuration—for example, a metadata filter matching no documents. Record the actual empty array and branch trace, restore settings, then rerun supported cases. An unfamiliar question does not prove the empty branch. This corpus has no conflict pair; conflict runtime behavior is also untested.

6. Errors and the stopping point

Missing providers, rate limits, failed indexing, retrieval errors and LLM timeouts stop the test. Record sanitized unavailable and the failing node; do not silently switch providers, use model memory or return empty success. Wrong citations and unsupported embellishment return the draft for correction. Check chunks, Context wiring, variables, both retrieval-setting layers and citation settings; rerun positive and negative cases.

Every draft requires an authorized person to compare the passages and bilingual meaning, review Unknowns, and record revise/reject/approve with identity and time. This exercise implements no approval service or sending channel.

A later runtime record needs UTC time, Dify edition/version or “not exposed,” provider/plugin/model IDs, fixture hashes, saved chunk/retrieval settings, question, actual passages/scores, branch, draft, citation-to-chunk mapping, errors and real reviewer verdict. Retain synthetic data only, never keys. Until those records exist, upload/index/retrieval/generation/citations remain not-run, not passed.

FAQ

Do 22 passing tests mean the Q&A app works?

No. They check bilingual files, chunk text, source references and expected extracts. They neither run Dify nor evaluate model reliability.

Should a nonempty retrieval always produce an answer?

No. Scores and keyword overlap are not factual evidence. Irrelevant, stale or conflicting passages still require abstention or review.

When should I use n8n instead?

For explicit data checks, review labels and repeatable outputs, start with the n8n synthetic inquiry exercise. These are not exclusive capabilities. Both platforms can participate in a system, but this exercise does not implement their integration.

Use the Dify hub for official entry points. For real knowledge permissions, approval and existing-system requirements, see Solutions. Bring field names and process requirements, not raw customer documents.