Vespa Cloud logo

Hybrid Search - Quickstart on Vespa Cloud

This is the same guide as getting-started-pyvespa, deploying to Vespa Cloud.

Refer to troubleshooting for any problem when running this guide.

Pre-requisite: Create a tenant at cloud.vespa.ai, save the tenant name.

Open In Colab

Install

Install pyvespa >= 0.38 and the Vespa CLI. The Vespa CLI is used for data and control plane key management (Vespa Cloud Security Guide).

[1]:
!pip3 install pyvespa

Install the Vespa CLI using homebrew:

[1]:
!brew install vespa-cli

Alternatively, if running in Colab, download the Vespa CLI from GitHub:

[1]:
import os
import requests

res = requests.get(
    url="https://api.github.com/repos/vespa-engine/vespa/releases/latest"
).json()
os.environ["VERSION"] = res["tag_name"].replace("v", "")
!curl -fsSL https://github.com/vespa-engine/vespa/releases/download/v${VERSION}/vespa-cli_${VERSION}_linux_amd64.tar.gz | tar -zxf -
!ln -sf /content/vespa-cli_${VERSION}_linux_amd64/bin/vespa /bin/vespa

Configure data-plane security

Create Vespa Cloud data-plane mTLS cert/key-pair. This mutual certificate pair is used to talk to your Vespa cloud endpoints. See Vespa Cloud Security Guide.

We save the paths to the credentials, for later dataplane access without using pyvespa APIs - see example at the end of this notebook.

[1]:
import os

os.environ["TENANT_NAME"] = "vespa-team"  # Replace with your tenant name
application = "hybridsearch"
vespa_cli_command = (
    f'vespa config set application {os.environ["TENANT_NAME"]}.{application}'
)

!vespa config set target cloud
!{vespa_cli_command}
!vespa auth cert -N
[1]:
from os.path import exists
from pathlib import Path

cert_path = (
    Path.home()
    / ".vespa"
    / f"{os.environ['TENANT_NAME']}.{application}.default/data-plane-public-cert.pem"
)
key_path = (
    Path.home()
    / ".vespa"
    / f"{os.environ['TENANT_NAME']}.{application}.default/data-plane-private-key.pem"
)

if not exists(cert_path) or not exists(key_path):
    print(
        "ERROR: set the correct paths to security credentials. Correct paths above and rerun until you do not see this error"
    )

Note that the subsequent deploy-call below will add data-plane-public-cert.pem to the application before deploying it to Vespa Cloud, so that you have access to both the private key and the public certificate, while Vespa Cloud only knows the public certificate.

Configure control-plane security

Authenticate to generate a tenant level control-plane API key for deploying the applications to Vespa Cloud, and save the path to it.

Warning:The generated tenant api key must be added in the Vespa Console before attempting to deploy the application.

The following step will print the following message:

To use this key in Vespa Cloud click 'Add custom key' at
https://console.vespa-cloud.com/tenant/TENANT_NAME/account/keys
and paste the entire public key including the BEGIN and END lines.
[1]:
!vespa auth api-key

from pathlib import Path

api_key_path = Path.home() / ".vespa" / f"{os.environ['TENANT_NAME']}.api-key.pem"

Follow the instrauctions from the output above and add the control-plane key in the console at https://console.vespa-cloud.com/tenant/TENANT_NAME/account/keys (replace TENANT_NAME with your tenant name).

Create an application package

The application package has all the Vespa configuration files - create one from scratch:

[1]:
from vespa.package import (
    ApplicationPackage,
    Field,
    Schema,
    Document,
    HNSW,
    RankProfile,
    Component,
    Parameter,
    FieldSet,
    GlobalPhaseRanking,
    Function,
)

package = ApplicationPackage(
    name=application,
    schema=[
        Schema(
            name="doc",
            document=Document(
                fields=[
                    Field(name="id", type="string", indexing=["summary"]),
                    Field(
                        name="title",
                        type="string",
                        indexing=["index", "summary"],
                        index="enable-bm25",
                    ),
                    Field(
                        name="body",
                        type="string",
                        indexing=["index", "summary"],
                        index="enable-bm25",
                        bolding=True,
                    ),
                    Field(
                        name="embedding",
                        type="tensor<float>(x[384])",
                        indexing=[
                            'input title . " " . input body',
                            "embed",
                            "index",
                            "attribute",
                        ],
                        ann=HNSW(distance_metric="angular"),
                        is_document_field=False,
                    ),
                ]
            ),
            fieldsets=[FieldSet(name="default", fields=["title", "body"])],
            rank_profiles=[
                RankProfile(
                    name="bm25",
                    inputs=[("query(q)", "tensor<float>(x[384])")],
                    functions=[
                        Function(name="bm25sum", expression="bm25(title) + bm25(body)")
                    ],
                    first_phase="bm25sum",
                ),
                RankProfile(
                    name="semantic",
                    inputs=[("query(q)", "tensor<float>(x[384])")],
                    first_phase="closeness(field, embedding)",
                ),
                RankProfile(
                    name="fusion",
                    inherits="bm25",
                    inputs=[("query(q)", "tensor<float>(x[384])")],
                    first_phase="closeness(field, embedding)",
                    global_phase=GlobalPhaseRanking(
                        expression="reciprocal_rank_fusion(bm25sum, closeness(field, embedding))",
                        rerank_count=1000,
                    ),
                ),
            ],
        )
    ],
    components=[
        Component(
            id="e5",
            type="hugging-face-embedder",
            parameters=[
                Parameter(
                    "transformer-model",
                    {
                        "url": "https://github.com/vespa-engine/sample-apps/raw/master/simple-semantic-search/model/e5-small-v2-int8.onnx"
                    },
                ),
                Parameter(
                    "tokenizer-model",
                    {
                        "url": "https://raw.githubusercontent.com/vespa-engine/sample-apps/master/simple-semantic-search/model/tokenizer.json"
                    },
                ),
            ],
        )
    ],
)

Note that the name cannot have - or _.

Deploy to Vespa Cloud

The app is now defined and ready to deploy to Vespa Cloud.

Deploy package to Vespa Cloud, by creating an instance of VespaCloud:

[1]:
from vespa.deployment import VespaCloud


def read_secret():
    """Read the API key from the environment variable. This is
    only used for CI/CD purposes."""
    t = os.getenv("VESPA_TEAM_API_KEY")
    if t:
        return t.replace(r"\n", "\n")
    else:
        return t


vespa_cloud = VespaCloud(
    tenant=os.environ["TENANT_NAME"],
    application=application,
    key_content=read_secret() if read_secret() else None,
    key_location=api_key_path,
    application_package=package,
)

The following will upload the application package to Vespa Cloud Dev Zone (aws-us-east-1c), read more about Vespa Zones. PyVespa currently only supports deploying to the Vespa Cloud Dev Zone, which is considered as a sandbox environment where resources are down-scaled and idle deployments are expired automatically.

Note: Deployments to dev and perf expire after 7 days of inactivity, i.e., 7 days after running deploy. This applies to all plans, not only the Free Trial. Use the Vespa Console to extend the expiry period, or redeploy the application to add 7 more days.

[1]:
app = vespa_cloud.deploy()

If the deployment failed, it is possible you forgot to add the key in the Vespa Cloud Console in the vespa auth api-key step above.

If you can authenticate, you should see lines like the following

Deployment started in run 1 of dev-aws-us-east-1c for mytenant.hybridsearch.

The deployment takes a few minutes the first time while Vespa Cloud sets up the resources for your Vespa application

app now holds a reference to a Vespa instance. We can access the mTLS protected endpoint name using the control-plane (vespa_cloud) instance. This endpoint we can query and feed to (data plane access) using the mTLS certificate generated in previous steps.

[1]:
endpoint = vespa_cloud.get_mtls_endpoint()
endpoint

Feeding documents to Vespa

In this example we use the HF Datasets library to stream the BeIR/nfcorpus dataset and index in our newly deployed Vespa instance. Read more about the NFCorpus:

NFCorpus is a full-text English retrieval data set for Medical Information Retrieval.

The following uses the stream option of datasets to stream the data without downloading all the contents locally. The map functionality allows us to convert the dataset fields into the expected feed format for pyvespa which expects a dict with the keys id and fields:

{ "id": "vespa-document-id", "fields": {"vespa_field": "vespa-field-value"}}

[1]:
from datasets import load_dataset

dataset = load_dataset("BeIR/nfcorpus", "corpus", split="corpus", streaming=True)
vespa_feed = dataset.map(
    lambda x: {
        "id": x["_id"],
        "fields": {"title": x["title"], "body": x["text"], "id": x["_id"]},
    }
)

Now we can feed to Vespa using feed_iterable which accepts any Iterable and an optional callback function where we can check the outcome of each operation. The application is configured to use embedding functionality, that produce a vector embedding using a concatenation of the title and the body input fields. This step is resource intensive.

Read more about embedding inference in Vespa in the Accelerating Transformer-based Embedding Retrieval with Vespa blog post.

Default node resources in Vespa Cloud have 2 v-cpu for the Dev Zone.

[1]:
from vespa.io import VespaResponse, VespaQueryResponse


def callback(response: VespaResponse, id: str):
    if not response.is_successful():
        print(f"Error when feeding document {id}: {response.get_json()}")


app.feed_iterable(vespa_feed, schema="doc", namespace="tutorial", callback=callback)

Querying Vespa

Using the Vespa Query language we can query the indexed data.

  • Using a context manager with app.syncio() as session to handle connection pooling (best practices)

  • The query method accepts any valid Vespa query api parameter in **kwargs

  • Vespa api parameter names that contains . must be sent as dict parameters in the body method argument

The following searches for How Fruits and Vegetables Can Treat Asthma? using different retrieval and ranking strategies.

Query the text search app using the Vespa Query language by sending the parameters to the body argument of Vespa.query.

First we define a simple routine that will return a dataframe of the results for prettier display in the notebook.

[1]:
import pandas as pd


def display_hits_as_df(response: VespaQueryResponse, fields) -> pd.DataFrame:
    records = []
    for hit in response.hits:
        record = {}
        for field in fields:
            record[field] = hit["fields"][field]
        records.append(record)
    return pd.DataFrame(records)

Next steps

This is just an intro into the capabilities of Vespa and pyvespa. Browse the site to learn more about schemas, feeding and queries - find more complex applications in examples.

Example: Document operations using cert/key pair

Above, we deployed to Vespa Cloud, and as part of that, generated a data-plane mTLS cert/key pair.

This pair can be used to access the dataplane for reads/writes to documents and running queries from many different clients. The following demonstrates that using the requests library.

Set up a dataplane connection using the cert/key pair:

[1]:
import requests

session = requests.Session()
session.cert = (cert_path, key_path)

Get a document from the endpoint returned when we deployed to Vespa Cloud above. PyVespa wraps the Vespa document api internally and in these examples we use the document api directly, but with the mTLS key/cert pair we used when deploying the app.

[1]:
url = "{0}/document/v1/{1}/{2}/docid/{3}".format(endpoint, "tutorial", "doc", "MED-10")
doc = session.get(url).json()
doc

Update the title and post the new version:

[1]:
doc["fields"]["title"] = "Can you eat lobster?"
response = session.post(url, json=doc).json()
response

Get the doc again to see the new title:

[1]:
doc = session.get(url).json()
doc

Example: Reconnect pyvespa using cert/key pair

Above, we stored the dataplane credentials for later use. Deployment of an application usually happens when the schema changes, whereas accessing the dataplane is for document updates and user queries.

One only needs to know the endpoint and the cert/key pair to enable a connection to a Vespa Cloud application:

[1]:
# cert_path = "/Users/me/.vespa/mytenant.hybridsearch.default/data-plane-public-cert.pem"
# key_path  = "/Users/me/.vespa/mytenant.hybridsearch.default/data-plane-private-key.pem"

from vespa.application import Vespa

the_app = Vespa(endpoint, cert=cert_path, key=key_path)

res = the_app.query(
    yql="select documentid, id, title from sources * where userQuery()",
    query="Can you eat lobster?",
    ranking="bm25",
)
res.hits[0]

A common problem is a cert mismatch - the cert/key pair used when deployed is different than the pair used when making requests against Vespa. This will cause 40x errors.

Make sure it is the same pair / re-create with vespa auth cert -f AND redeploy.

If you re-generate a mTLS certificate pair, and use that when connecting to Vespa cloud endpoint, it will fail until you have updaded the deployment with the new public certificate.

Delete application

The following will delete the application and data from the dev environment.

[1]:
vespa_cloud.delete()