Querying Vespa

This guide goes through how to query a Vespa instance using the Query API and https://cord19.vespa.ai/ app as an example.

Refer to troubleshooting for any problem when running this guide.

You can run this tutorial in Google Colab:

!pip3 install pyvespa

Requirement already satisfied: pyvespa in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (0.7.0+dev)
Requirement already satisfied: requests in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (2.31.0)
Requirement already satisfied: requests-toolbelt in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (1.0.0)
Requirement already satisfied: docker in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (7.0.0)
Requirement already satisfied: jinja2 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (3.1.3)
Requirement already satisfied: cryptography in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (42.0.5)
Requirement already satisfied: aiohttp in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (3.9.3)
Requirement already satisfied: tenacity in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (8.2.3)
Requirement already satisfied: typing-extensions in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from pyvespa) (4.10.0)
Requirement already satisfied: aiosignal>=1.1.2 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from aiohttp->pyvespa) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from aiohttp->pyvespa) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from aiohttp->pyvespa) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from aiohttp->pyvespa) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from aiohttp->pyvespa) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from aiohttp->pyvespa) (4.0.3)

Requirement already satisfied: cffi>=1.12 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from cryptography->pyvespa) (1.16.0)
Requirement already satisfied: packaging>=14.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from docker->pyvespa) (24.0)
Requirement already satisfied: urllib3>=1.26.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from docker->pyvespa) (2.2.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from requests->pyvespa) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from requests->pyvespa) (3.6)
Requirement already satisfied: certifi>=2017.4.17 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from requests->pyvespa) (2024.2.2)
Requirement already satisfied: MarkupSafe>=2.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from jinja2->pyvespa) (2.1.5)

Requirement already satisfied: pycparser in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/stable/lib/python3.9/site-packages (from cffi>=1.12->cryptography->pyvespa) (2.22)

Connect to a running Vespa instance.

from vespa.application import Vespa
from vespa.io import VespaQueryResponse
from vespa.exceptions import VespaError

app = Vespa(url="https://api.cord19.vespa.ai")

See the Vespa query language for Vespa query api request parameters.

The YQL userQuery() operator uses the query read from query. The query also specificies to use the app specific bm25 rank profile. The code uses context manager with session statement to make sure that connection pools are released. If you attempt to make multiple queries, this is important as each query will not have to setup new connections.

with app.syncio() as session:
  response:VespaQueryResponse = session.query(
    yql="select documentid, cord_uid, title, abstract from sources * where userQuery()",
    hits=1,
    query="Is remdesivir an effective treatment for COVID-19?",
    ranking="bm25")
  print(response.is_successful())
  print(response.url)

True
https://api.cord19.vespa.ai/search/?yql=select+documentid%2C+cord_uid%2C+title%2C+abstract+from+sources+%2A+where+userQuery%28%29&hits=1&query=Is+remdesivir+an+effective+treatment+for+COVID-19%3F&ranking=bm25

Alternatively, if the native Vespa query parameter contains “.”, which cannot be used as a kwarg, the parameters can be sent as HTTP POST with the body argument. In this case ranking is an alias of ranking.profile, but using ranking.profile as a **kwargs argument is not allowed in python. This will combine HTTP parameters with a HTTP POST body.

with app.syncio() as session:
  response:VespaQueryResponse = session.query(hits=1,
    body = {
        "yql": "select documentid, cord_uid, title, abstract from sources * where userQuery()",
        "query": "Is remdesivir an effective treatment for COVID-19?",
        "ranking.profile": "bm25",
        "presentation.timing": True
        }
    )
  print(response.is_successful())

True

The query specified that we wanted one hit:

response.hits

[{'id': 'id:covid-19:doc::534720',
  'relevance': 37.65838267960925,
  'source': 'content',
  'fields': {'title': 'A Review on <hi>Remdesivir</hi>: A Possible Promising Agent for the <hi>Treatment</hi> of <hi>COVID</hi>-<hi>19</hi>',
   'abstract': '<sep />manufacturing of specific therapeutics and vaccines to treat <hi>COVID</hi>-<hi>19</hi> are time-consuming processes. At this time, using available conventional therapeutics along with other <hi>treatment</hi> options may be useful to fight <hi>COVID</hi>-<hi>19</hi>. In different clinical trials, efficacy of <hi>remdesivir</hi> (GS-5734) against Ebola virus has been demonstrated. Moreover, <hi>remdesivir</hi> may be an <hi>effective</hi> therapy in vitro and in animal models infected by SARS and MERS coronaviruses. Hence, the drug may be theoretically <hi>effective</hi> against SARS-CoV-2. <hi>Remdesivir</hi><sep />',
   'documentid': 'id:covid-19:doc::534720',
   'cord_uid': 'xej338lo'}}]

Example of iterating over the returned hits obtained from respone.hits, extracting the cord_uid field:

[hit["fields"]["cord_uid"] for hit in response.hits]

['xej338lo']

Access the full JSON response in the Vespa default JSON result format:

response.json

{'timing': {'querytime': 0.006, 'summaryfetchtime': 0.0, 'searchtime': 0.008},
 'root': {'id': 'toplevel',
  'relevance': 1.0,
  'fields': {'totalCount': 14209},
  'coverage': {'coverage': 100,
   'documents': 976355,
   'full': True,
   'nodes': 2,
   'results': 1,
   'resultsFull': 1},
  'children': [{'id': 'id:covid-19:doc::534720',
    'relevance': 37.65838267960925,
    'source': 'content',
    'fields': {'title': 'A Review on <hi>Remdesivir</hi>: A Possible Promising Agent for the <hi>Treatment</hi> of <hi>COVID</hi>-<hi>19</hi>',
     'abstract': '<sep />manufacturing of specific therapeutics and vaccines to treat <hi>COVID</hi>-<hi>19</hi> are time-consuming processes. At this time, using available conventional therapeutics along with other <hi>treatment</hi> options may be useful to fight <hi>COVID</hi>-<hi>19</hi>. In different clinical trials, efficacy of <hi>remdesivir</hi> (GS-5734) against Ebola virus has been demonstrated. Moreover, <hi>remdesivir</hi> may be an <hi>effective</hi> therapy in vitro and in animal models infected by SARS and MERS coronaviruses. Hence, the drug may be theoretically <hi>effective</hi> against SARS-CoV-2. <hi>Remdesivir</hi><sep />',
     'documentid': 'id:covid-19:doc::534720',
     'cord_uid': 'xej338lo'}}]}}

Query Performance

There are several things that impact end-to-end query performance

• HTTP layer performance, connecting handling, mututal TLS handshake and network round-trip latency

  • Make sure to re-use connections using context manager with vespa.app.syncio(): to avoid setting up new connections for every unique query. See http best practises

  • The size of the fields and the number of hits requested also greatly impacts network performance, a larger payload means higher latency.

  • By adding "presentation.timing": True as a request parameter, the Vespa response includes the server side processing (also including reading the query from network, but not delivering the result over the network). This can be handy to debug latency.

• Vespa performance, the features used inside the Vespa instance.

with app.syncio(connections=12) as session:
  response:VespaQueryResponse = session.query(hits=1,
    body = {
        "yql": "select documentid, cord_uid, title, abstract from sources * where userQuery()",
        "query": "Is remdesivir an effective treatment for COVID-19?",
        "ranking.profile": "bm25",
        "presentation.timing": True
        }
    )
  print(response.is_successful())

True

Error handling

Vespa’s default query timeout is 500ms, PyVespa will by default retry up to 3 times for queries that return response codes like 429, 500,503 and 504. A VespaError is raised if retries did not end up with success. In the following example we set a very low timeout of 1ms which will cause Vespa to time out the request and it returns a 504 http error code. The underlaying error is wrapped in a VespaError with the payload error message returned from Vespa:

with app.syncio(connections=12) as session:
    try:
        response:VespaQueryResponse = session.query(hits=1,
            body = {
                "yql": "select * from sources * where userQuery()",
                "query": "Is remdesivir an effective treatment for COVID-19?",
                "timeout": "1ms"
                }
            )
        print(response.is_successful())
    except VespaError as e:
        print(str(e))

[{'code': 12, 'summary': 'Timed out', 'source': 'content', 'message': "Error in execution of chain 'content': Chain timed out."}]

In the following example we forgot to include the query parameter, but still reference it in the yql, this cause a bad client request response (400):

with app.syncio(connections=12) as session:
    try:
        response:VespaQueryResponse = session.query(hits=1,
            body = {
                "yql": "select * from sources * where userQuery()"
                }
            )
        print(response.is_successful())
    except VespaError as e:
        print(str(e))

[{'code': 3, 'summary': 'Illegal query', 'source': 'content', 'message': 'No query'}]