# HTML

In an HTML knowledge source, you can enter the URL of any public website to be indexed and used for answer generation.

* You can specify conditions to consider only specific pages/sub-pages under a Root URL
* ClearFeed can automatically refresh the data from the source every 24 hours if you keep Auto-Reindex turned on.

{% hint style="info" %}
**Common Use Cases:** Product Documentation, Blogs, Case Studies, Public Websites, etc...
{% endhint %}

#### Add HTML Knowledge Source

* Go to **KBs** from the left navigation bar and click the **"Add Knowledge Source"** dropdown in the top-right corner. Select **HTML**.
* Fill out the side drawer form:
  * **Name**: Enter a title for the Knowledge Source.
  * **Tags**: Add tags to categorize the Knowledge Source.
  * **Root URL**: Provide the main URL ClearFeed will index from. Only URLs branching from this root will be indexed. For example, if the root is `https://docs.clearfeed.ai/clearfeed-help-center/product-features/ticketing`, a URL like `https://docs.clearfeed.ai/clearfeed-help-center/product-features/collections` will not be indexed.
  * **URL Conditions**: Set rules for URLs to include/exclude using options like "Starts with," "Does not start with," "Contains," or "Does not contain."
  * **Auto Reindex Regularly**: Enable this to reindex the Knowledge Source every 24 hours.
* Save your entry. The Knowledge Source will either start indexing immediately (**In Progress** status) or shortly after (**Initiated** status). Get more details on Managing Knowledge Sources here.
* **Reindexing Public Docs -** To reindex public documents, **manually reindex** the knowledge source by clicking the "Reindex" button next to it.

{% hint style="info" %}
**Optimize Indexing of Knowledge Sources with Sitemaps**

ClearFeed will check each root URL for sitemaps, as using sitemaps is faster than a full website crawl. If a sitemap is found, we'll fetch all pages under the specified root URL, excluding any URLs you've chosen to exclude, ensuring all relevant pages are considered.
{% endhint %}

#### Using AI Agents

Once you've configured the knowledge sources, learn how to build and use AI Agents

{% content-ref url="../../ai-agents/build-ai-agents" %}
[build-ai-agents](https://docs.clearfeed.ai/clearfeed-help-center/clearfeed-ai/ai-agents/build-ai-agents)
{% endcontent-ref %}

{% content-ref url="../../ai-agents/use-ai-agents" %}
[use-ai-agents](https://docs.clearfeed.ai/clearfeed-help-center/clearfeed-ai/ai-agents/use-ai-agents)
{% endcontent-ref %}

#### **Manage Knowledge Sources**

Once the status of the knowledge source has been indexed, learn more on how to manage the knowledge source below.

{% content-ref url="../managing-knowledge-sources" %}
[managing-knowledge-sources](https://docs.clearfeed.ai/clearfeed-help-center/clearfeed-ai/knowledge-sources/managing-knowledge-sources)
{% endcontent-ref %}