Skip to main content

📥 Sources

Sources are the raw material of your knowledge base. Add text, crawl your website, upload PDFs, transcribe YouTube videos, or pull in X posts — the agent indexes everything into searchable knowledge it answers from.

Overview​

From the Sources tab you can:

  • Add five kinds of entries: Website, Text, PDF, YouTube, and X
  • Crawl a whole website with page limits, link depth, and path filters
  • Transcribe a single YouTube video or a batch of a channel's recent videos
  • Watch entries process, then inspect their indexed content in a detail drawer
  • Filter, search, and bulk-delete entries as the knowledge base grows

Configuration​

To manage sources:

  1. Open your project in the Fren One webapp.
  2. Select the Discord server you want to configure.
  3. In the sidebar under Agent Configuration, select Knowledge Base, then the Sources tab.
  4. Select Add Entry and pick a type.

Entry types​

The Type selector offers five kinds of entry, each with its own flow:

TypeHow it's added
TextPaste content directly into a Content field. Required.
PDFEither Upload PDF from your computer or provide a PDF URL — one or the other.
WebsiteEnter a URL; the crawl modal lets you scope and confirm the crawl (see below).
YouTubeA single video URL ingests directly; a channel URL opens a modal to pick recent videos (see below).
XA post URL, or a profile to pull its recent posts.

Crawling a website​

Adding a website routes through the Crawl modal, where you preview the crawl, see a credit estimate, confirm, and then watch live progress. Crawl options:

OptionDescription
Max pagesThe maximum number of pages to crawl.
Link depthHow many links deep the crawler follows from the starting URL.
Include paths / Exclude pathsGlob patterns, one per line, to scope which paths are crawled.
Only crawl pages on the same domainKeeps the crawler from following links to other domains.

Before you confirm, the modal shows an estimate — "Up to N pages · at most ~N credits" — and notes: "You're only charged for pages actually indexed."

Ingesting YouTube videos​

A single video URL is ingested directly. Pasting a channel URL opens the YouTube channel modal, where you choose how many of the channel's recent videos to transcribe. Credits are charged per video, and videos that are already indexed are skipped automatically.

Credits

Website crawls and YouTube ingestion consume credits — the crawl modal estimates the cost up front, and you're only charged for pages actually indexed.

Managing entries​

Each entry row shows its title or URL, a type icon, its chunk count, and a Processing state while it's being ingested. Once you have 5 or more entries, a filter bar appears with a type filter and search. Select mode enables multi-select with bulk delete.

Selecting an entry opens the detail drawer, with three tabs — Content, Details, and Sources — and three actions:

ActionDescription
RefreshRe-ingests the entry.
Re-crawlRuns the crawl again to pick up changed pages.
DeleteRemoves the entry, after confirmation.
Deleting a source

Deleting an entry means "The agent will lose this source" — and it cannot be undone. The agent stops answering from that content immediately.

Saving your changes​

Changes on this page apply immediately — adding an entry starts ingestion right away, and there is no page-level Save bar. Deleting an entry (or a bulk selection) asks for confirmation first.

Best practices​

Building a useful knowledge base
  • Start with your highest-signal content — official docs and FAQs beat raw chat logs.
  • Scope website crawls with Include paths / Exclude paths so you index docs, not marketing pages — it also keeps the credit cost down.
  • Use Re-crawl after major website updates so the agent doesn't answer from stale pages.
  • Test in the Chat tab after adding sources to confirm the agent picks them up.
Troubleshooting

An entry is stuck in Processing?

  • Ingestion runs in the background and large sources take longer — check back after a moment.
  • If it never finishes, the 🩺 Curation tab surfaces entries that failed to ingest, with a Refresh action to retry.

A crawl indexed fewer pages than expected?

  • Check Max pages, Link depth, and your path globs — pages outside the scope are skipped, and with "Only crawl pages on the same domain" on, external links aren't followed. You're only charged for pages actually indexed.

Some YouTube videos weren't ingested?

  • Videos that are already indexed are skipped automatically, so re-running a channel ingest only adds new ones.

On this page