Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename corpus and corpora as index and indices #76

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

.DS_Store
8 changes: 4 additions & 4 deletions www/docs/api-reference/admin-apis/admin.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@ import {vars} from '@site/static/variables.json';

The admin console is a good way to get started with <Config v="names.product"/>. Once
you're ready to integrate the platform more deeply into your application, the
Admin APIs allow you to programatically manipulate corpora and perform many
Admin APIs allow you to programatically manipulate indices, also known as corpora, and perform many
other operations within the system.

## Full Definition

### Service

The full definitions of the Create, Reset, and Delete gRPC APIs are covered
below. The Create API allows corpora to be programatically created, up to the
limit defined for the account. Reset deletes all data from a corpus, without
deleting its definition, while Delete expunges both the data in the corpus and
below. The Create API allows indices to be programatically created, up to the
limit defined for the account. Reset deletes all data from an index, without
deleting its definition, while Delete expunges both the data in the index and
its definition.

The REST APIs are programatically derived from these definitions: see
Expand Down
21 changes: 11 additions & 10 deletions www/docs/api-reference/admin-apis/create-corpus.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,16 @@ sidebar_label: API Definition
import {Config} from '@site/docs/definitions.md';
import {vars} from '@site/static/variables.json';

This page describes the details of interacting with this endpoint.

## Endpoint Address

<Config v="names.product"/> exposes a REST endpoint at the following URL
to index content into a corpus:
to injest content into an index:
<code>https://<Config v="domains.rest.admin"/>/v1/create-corpus</code>
This page describes the details of interacting with this endpoint.

Only the `name` and `description` fields are mandatory when creating a corpus.
The response message returns a unique id, `corpus_id`, by which the corpus can
Only the `name` and `description` fields are mandatory when creating an index.
The response message returns a unique id, `corpus_id`, by which the index can
be subsequently referenced. Note that the name needn't be unique within an
account.

Expand All @@ -33,21 +34,21 @@ message CreateCorpusRequest {
}

message CreateCorpusResponse {
// The Corpus ID that was created.
// The index is the Corpus ID that was created.
uint32 corpus_id = 1;
Status status = 2;
}

message Corpus {
// The Corpus ID.
// The index ID.
uint32 id = 1;
// The name of the corpus.
// The name of the index.
string name = 2;
// A description for the corpus.
// A description for the index.
string description = 3;
// The time at which the corpus was provisioned.
// The time at which the index was provisioned.
int64 dt_provision = 4;
// Whether the corpus is enabled for use or not.
// Whether the index is enabled for use or not.
bool enabled = 5;


Expand Down
13 changes: 6 additions & 7 deletions www/docs/api-reference/admin-apis/delete-corpus.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,19 @@ import {Config} from '@site/docs/definitions.md';
import {vars} from '@site/static/variables.json';

<Config v="names.product"/> exposes a REST endpoint at the following URL
to index content into a corpus:
to delete an index:
<code>https://<Config v="domains.rest.admin"/>/v1/delete-corpus</code>
This page describes the details of interacting with this endpoint.

To delete a corpus, specify the **customer id** and **corpus id**. Upon
successful completion, space quota consumed by the corpus will be freed,
and the corpus will no longer be usable for future indexing or querying.
Note that the corpus id assigned to the corpus will not be reused.
To delete an index, specify the **customer_id** and **corpus_id**. Upon
successful completion, space quota consumed by the index will be freed,
and the index will no longer be usable for future indexing or querying.
Note that the corpus_id assigned to the index will not be reused.

```protobuf
message DeleteCorpusRequest {
// The Customer ID that contains the corpus to be deleted.
uint32 customer_id = 1;
// The Corpus ID to be deleted.
// The index is the Corpus ID to be deleted.
uint32 corpus_id = 2;
}

Expand Down
9 changes: 4 additions & 5 deletions www/docs/api-reference/admin-apis/reset-corpus.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,16 @@ import {Config} from '@site/docs/definitions.md';
import {vars} from '@site/static/variables.json';

<Config v="names.product"/> exposes a REST endpoint at the following URL
to index content into a corpus:
to reset an index:
<code>https://<Config v="domains.rest.admin"/>/v1/reset-corpus</code>
This page describes the details of interacting with this endpoint.

To reset a corpus, specify the **customer id** and **corpus id**. Upon
successful completion, space quota consumed by the corpus will be freed.
To reset an index, specify the **customer_id** and **corpus_id**. Upon
successful completion, space quota consumed by the index will be freed.

```protobuf
message ResetCorpusRequest {
uint32 customer_id = 1;
// The Corpus ID to be reset.
// The index is the Corpus ID to be reset.
uint32 corpus_id = 2;
}

Expand Down
6 changes: 6 additions & 0 deletions www/docs/api-reference/auth-apis/api-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ having an accidental publication of an over-privileged API key is often
organizationally "expensive." In general, it's recommended that you use
[OAuth 2.0](OAuth 2.0) if/where possible for production applications.

:::warning

:lock: Always keep your API Keys and 0Auth tokens private. Do not share them through email, Slack, Discord, forums, or other public channels because it can lead to unauthorized access. Treat these keys with the same confidentiality as your personal credentials.

:::

# Use an API Key
To use an API key in a request, you need to pass in `x-api-key` as an HTTP
header.
Expand Down
6 changes: 6 additions & 0 deletions www/docs/api-reference/auth-apis/oauth.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@ OAuth 2.0 has several advantages over API keys or simple usernames/passwords:
- OAuth 2.0 is inherently more tightly scoped than API keys
- JWT tokens are detected by many security scanning tools, allowing them to more easily be flagged in the case of accidental publication

:::warning

:lock: Always keep your OAuth tokens private. Do not share them through email, Slack, Discord, forums, or other public channels because it can lead to unauthorized access. Treat these tokens with the same confidentiality as your personal credentials.

:::

# Creating an application client
Go to [https://console.vectara.com/console/authentication/app-client](https://console.vectara.com/console/authentication/app-client)
to create a new application client. Most applications will want to use the
Expand Down
10 changes: 2 additions & 8 deletions www/docs/api-reference/indexing-apis/core_indexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,9 @@ The full definition of the gRPC interface is covered below.

### Service

The indexing service operates in two modes: _incremental_ and _batch_. In
incremental mode, individual documents or messages are sent to be indexed. In
a short period of time, generally a few minutes, the new content will become
available in the search index.
The indexing service operates by accepting individual documents or messages to be indexed. In a short period of time, generally a few minutes, the new content will become available in the search index.

The definition of the incremental service is shown below.
The definition of the service is shown below.


```protobuf
Expand All @@ -34,9 +31,6 @@ service CoreIndexService {
}
```

When there are a large number of documents to process, it is more efficient to
process them using the batch interface. This is described below.

### Document Index Request

A request to add data into a corpus consists of three key pieces of information:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,12 @@ headers.
The maximum file size supported by the server is 10 MB.

You can attach additional metadata to the file by specifying an additional
`doc_metadata` form field, which can contain a JSON string.
`doc_metadata` form field, which can contain a JSON string:

```json
doc_metadata='{ "filesize": 1234 }'
```


## Response Codes

Expand Down Expand Up @@ -71,6 +76,6 @@ corpus `151` in customer `123456` using `curl`.
{`$ jwt=eyJraWQ...
$ curl -H "Authorization: Bearer $jwt" -F file=@/tmp/instructions.pdf \\
-F doc_metadata='{ "filesize\": 1234 }' \\
https://${vars['domains.rest.indexing']}:443/v1/upload?c=123456\&o=151
'https://${vars['domains.rest.indexing']}:443/v1/upload?c=123456\&o=151'
`}
</pre>
10 changes: 2 additions & 8 deletions www/docs/api-reference/indexing-apis/indexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,9 @@ The full definition of the gRPC interface is covered below.

### Service

The indexing service operates in two modes: _incremental_ and _batch_. In
incremental mode, individual documents or messages are sent to be indexed. In
a short period of time, generally a few minutes, the new content will become
available in the search index.
The indexing service operates by accepting individual documents or messages to be indexed. In a short period of time, generally a few minutes, the new content will become available in the search index.

The definition of the incremental service is shown below.
The definition of the service is shown below.


```protobuf
Expand All @@ -44,9 +41,6 @@ service IndexService {
}
```

When there are a large number of documents to process, it is more efficient to
process them using the batch interface. This is described below.

## Index Document

A request to add data into a corpus consists of three key pieces of information:
Expand Down
6 changes: 6 additions & 0 deletions www/docs/common-use-cases/app-authn-authz/create_api_keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,12 @@ using the it. Then, click **Create**.

You can now start using the key.

:::warning

:lock: Always keep your API Keys and OAuth tokens private. Do not share them through email, Slack, Discord, forums, or other public channels because it can lead to unauthorized access. Treat these keys with the same confidentiality as your personal credentials.

:::

## Using a Key

To use an API key, pass it using the `x-api-key` header request.
Expand Down
36 changes: 18 additions & 18 deletions www/docs/console-ui/corpus_create.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,30 @@
---
id: creating-a-corpus
title: Creating a corpus
title: Create a corpus
---

To get begin searching your data, you first have to create a corpus. A corpus is
a container where you upload all your data to be indexed and grouped together in
a single location for querying.
To get begin searching your data, you first have to create an index. An index,
also referenced as a corpus, is a container where you upload all your data
to be injested and grouped together in a single location for querying.

1. To get started, navigate to the [Console Overview](https://console.vectara.com/overview).

2. On the left sidebar, you'll see a **Copora** section. Click on it. This will
open an overview of the corpora you have created. It will be empty if this
2. On the left sidebar, you'll see a **Indices** section. Click on it. This will
open an overview of the indices you have created.

:::note
The Indices list is empty if this
is your first time accessing the console.
:::

![Create corpus](/img/create_corpus.png)
3. Click the **Create Index** button, and a dialog box will open.
4. Enter the
name and description of the index.

3. Now click on the **Create Corpus** button, a dialog box will open. Enter the
name and description of the corpus.

![Create corpus dialog](/img/create_corpus_dialog.png)

4. Leave all settings on their default values and click create. Once created,
you will see a confirmation message and the newly created corpus listed
5. Leave all settings on their default values and click create.

Once created, you will see a confirmation message and the newly created index listed
below.

![Create corpus success](/img/create_corpus_success.png)

The corpus is now created and ready to receive your data. You might want to head
towards how to index data and corpus overview.
The index is now created and ready to receive your data. You might want to head
towards how to index data and the index overview.
37 changes: 18 additions & 19 deletions www/docs/console-ui/reset_or_delete_corpus.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,39 @@
---
id: reset-or-delete-corpus
title: Resetting and deleting a corpus
title: Reset or delete a corpus
---

If you've decided to stop using a corpus you have three options.
If you've decided to stop using an index you have three options.

1. **Disable**: Disables query or indexing requests. A disabled corpus can be
1. **Disable**: Disables query or indexing requests. A disabled index can be
reenabled at any time.
2. **Reset**: Purge all the data within the corpus, but leave the corpus
definition intact.
3. **Delete**: Purge all the data within the corpus and delete the corpus.
All your connected services to the corpus will also cease to function.
2. **Reset**: Purge all the data within the index, but leave the definition intact.
3. **Delete**: Purge all the data within the index and the delete the index.
All your connected services to the index will also cease to function.

:::caution
The **Reset** and **Delete** operations are irreversable.
:::

You can view these options on the corpus operations tab with their respective
You can view these options on the Indices operations tab with their respective
descriptions.

![Create operations](/img/corpus_operations.png)

## Resetting a corpus
## Reset an index

To reset a corpus:
1. Click on the _Reset_ button. A warning modal will appear.
2. Enter the full name of the corpus for confirmation (case insensitive).
To reset an index:
1. Click the **Reset** button. A warning modal will appear.
2. Enter the full name of the index for confirmation (case insensitive).
3. Select _Reset_ and wait for the confirmation message.

That's it, all the data within the corpus has been purged.
That's it, all the data within the index has been purged.

## Deleting a corpus
## Delete an index

To permanently delete a corpus:
1. Click on the _Delete_ button. A warning modal will appear.
2. Enter the full name of the corpus for confirmation (case insensitive).
3. Select _Delete_ and wait for the confirmation message.
To permanently delete an index:
1. Click the **Delete** button. A warning modal will appear.
2. Enter the full name of the index for confirmation (case insensitive).
3. Select **Delete** and wait for the confirmation message.

That's it, the corpus has been deleted.
That's it, the index has been deleted.
Loading