Queries
This guide goes through how to query a Vespa instance using the Query API and https://cord19.vespa.ai/ app as an example.
Refer to troubleshooting for any problem when running this guide.
You can run this tutorial in Google Colab:
[1]:
!pip3 install pyvespa
Requirement already satisfied: pyvespa in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (0.7.0+dev)
Requirement already satisfied: requests in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (2.31.0)
Requirement already satisfied: pandas in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (2.1.0)
Requirement already satisfied: docker in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (6.1.3)
Requirement already satisfied: jinja2 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (3.0.3)
Requirement already satisfied: cryptography in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (41.0.4)
Requirement already satisfied: aiohttp in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (3.8.5)
Requirement already satisfied: tenacity in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (8.2.3)
Requirement already satisfied: typing-extensions in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pyvespa) (4.8.0)
Requirement already satisfied: attrs>=17.3.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from aiohttp->pyvespa) (23.1.0)
Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from aiohttp->pyvespa) (3.2.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from aiohttp->pyvespa) (6.0.4)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from aiohttp->pyvespa) (4.0.3)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from aiohttp->pyvespa) (1.9.2)
Requirement already satisfied: frozenlist>=1.1.1 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from aiohttp->pyvespa) (1.4.0)
Requirement already satisfied: aiosignal>=1.1.2 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from aiohttp->pyvespa) (1.3.1)
Requirement already satisfied: cffi>=1.12 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from cryptography->pyvespa) (1.15.1)
Requirement already satisfied: packaging>=14.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from docker->pyvespa) (23.1)
Requirement already satisfied: urllib3>=1.26.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from docker->pyvespa) (2.0.4)
Requirement already satisfied: websocket-client>=0.32.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from docker->pyvespa) (1.6.3)
Requirement already satisfied: idna<4,>=2.5 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from requests->pyvespa) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from requests->pyvespa) (2023.7.22)
Requirement already satisfied: MarkupSafe>=2.0 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from jinja2->pyvespa) (2.1.3)
Requirement already satisfied: numpy>=1.22.4 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pandas->pyvespa) (1.26.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pandas->pyvespa) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pandas->pyvespa) (2023.3.post1)
Requirement already satisfied: tzdata>=2022.1 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from pandas->pyvespa) (2023.3)
Requirement already satisfied: pycparser in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from cffi>=1.12->cryptography->pyvespa) (2.21)
Requirement already satisfied: six>=1.5 in /home/docs/checkouts/readthedocs.org/user_builds/pyvespa/envs/latest/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->pyvespa) (1.16.0)
Set the query endpoint:
[2]:
from vespa.application import Vespa
app = Vespa(url="https://api.cord19.vespa.ai")
See the Vespa query language for the query parameters.
The userQuery() operator uses the query test from query.
The query uses the weakAnd query operator and bm25 rank profile:
[3]:
body = {
'yql': 'select cord_uid, title, abstract from sources * where userQuery()',
'hits': 5,
'query': 'Is remdesivir an effective treatment for COVID-19?',
'type': 'weakAnd',
'ranking': 'bm25'
}
results = app.query(body=body)
results.number_documents_retrieved
[3]:
14104
The query specified 5 hits:
[4]:
len(results.hits)
[4]:
5
Iterate over results.hits:
[5]:
[hit["fields"]["cord_uid"] for hit in results.hits]
[5]:
['ysml5abq', 'xej338lo', '8art2tyj', 'npoje09j', 'zpcuwvn4']
Get a specific hit:
[6]:
results.hits[0]
[6]:
{'id': 'index:content/1/848ca5db53648911e0a58c78',
'relevance': 32.65003860273437,
'source': 'content',
'fields': {'title': 'Cost utility analysis of <hi>Remdesivir</hi> and Dexamethasone <hi>treatment</hi> for hospitalised <hi>COVID</hi>-<hi>19</hi> patients - a hypothetical study',
'abstract': '<sep />: Sars-Cov-2 is a novel corona virus associated with significant morbidity and mortality. <hi>Remdesivir</hi> and Dexamethasone are two <hi>treatments</hi> that have shown to be <hi>effective</hi> against the Sars-Cov-2 associated disease. However, a cost-effectiveness analysis of the two <hi>treatments</hi> is still lacking. OBJECTIVE: The cost-utility of <hi>Remdesivir</hi>, Dexamethasone and a simultaneous use of the two drugs with respect to standard of care for <hi>treatment</hi> <hi>Covid</hi>-<hi>19</hi> hospitalized patients is evaluated, together with the effect<sep />',
'cord_uid': 'ysml5abq'}}
Access the full response in the default result format:
[7]:
results.json
[7]:
{'timing': {'querytime': 0.007,
'summaryfetchtime': 0.001,
'searchtime': 0.009000000000000001},
'root': {'id': 'toplevel',
'relevance': 1.0,
'fields': {'totalCount': 14104},
'coverage': {'coverage': 100,
'documents': 976355,
'full': True,
'nodes': 2,
'results': 1,
'resultsFull': 1},
'children': [{'id': 'index:content/1/848ca5db53648911e0a58c78',
'relevance': 32.65003860273437,
'source': 'content',
'fields': {'title': 'Cost utility analysis of <hi>Remdesivir</hi> and Dexamethasone <hi>treatment</hi> for hospitalised <hi>COVID</hi>-<hi>19</hi> patients - a hypothetical study',
'abstract': '<sep />: Sars-Cov-2 is a novel corona virus associated with significant morbidity and mortality. <hi>Remdesivir</hi> and Dexamethasone are two <hi>treatments</hi> that have shown to be <hi>effective</hi> against the Sars-Cov-2 associated disease. However, a cost-effectiveness analysis of the two <hi>treatments</hi> is still lacking. OBJECTIVE: The cost-utility of <hi>Remdesivir</hi>, Dexamethasone and a simultaneous use of the two drugs with respect to standard of care for <hi>treatment</hi> <hi>Covid</hi>-<hi>19</hi> hospitalized patients is evaluated, together with the effect<sep />',
'cord_uid': 'ysml5abq'}},
{'id': 'index:content/1/eec37c88ff9aa825c5c79d81',
'relevance': 31.983392834139273,
'source': 'content',
'fields': {'title': 'A Review on <hi>Remdesivir</hi>: A Possible Promising Agent for the <hi>Treatment</hi> of <hi>COVID</hi>-<hi>19</hi>',
'abstract': '<sep />manufacturing of specific therapeutics and vaccines to treat <hi>COVID</hi>-<hi>19</hi> are time-consuming processes. At this time, using available conventional therapeutics along with other <hi>treatment</hi> options may be useful to fight <hi>COVID</hi>-<hi>19</hi>. In different clinical trials, efficacy of <hi>remdesivir</hi> (GS-5734) against Ebola virus has been demonstrated. Moreover, <hi>remdesivir</hi> may be an <hi>effective</hi> therapy in vitro and in animal models infected by SARS and MERS coronaviruses. Hence, the drug may be theoretically <hi>effective</hi> against SARS-CoV-2. <hi>Remdesivir</hi><sep />',
'cord_uid': 'xej338lo'}},
{'id': 'index:content/1/4678cba4fd31160fd163d063',
'relevance': 31.983392834139273,
'source': 'content',
'fields': {'title': 'A Review on <hi>Remdesivir</hi>: A Possible Promising Agent for the <hi>Treatment</hi> of <hi>COVID</hi>-<hi>19</hi>',
'abstract': '<sep />manufacturing of specific therapeutics and vaccines to treat <hi>COVID</hi>-<hi>19</hi> are time-consuming processes. At this time, using available conventional therapeutics along with other <hi>treatment</hi> options may be useful to fight <hi>COVID</hi>-<hi>19</hi>. In different clinical trials, efficacy of <hi>remdesivir</hi> (GS-5734) against Ebola virus has been demonstrated. Moreover, <hi>remdesivir</hi> may be an <hi>effective</hi> therapy in vitro and in animal models infected by SARS and MERS coronaviruses. Hence, the drug may be theoretically <hi>effective</hi> against SARS-CoV-2. <hi>Remdesivir</hi><sep />',
'cord_uid': '8art2tyj'}},
{'id': 'index:content/1/a41b357fc6aa38c6b55a4685',
'relevance': 31.549727030671207,
'source': 'content',
'fields': {'title': '<hi>Remdesivir</hi> for the <hi>Treatment</hi> of <hi>COVID</hi>-<hi>19</hi>: A Systematic Review of the Literature',
'abstract': '<sep />CoV-2 a global pandemic. To date, coronavirus disease-2019 (<hi>COVID</hi>-<hi>19</hi>) has spread to over 200 countries, leading to over 1.6 million cases and over 99,000 deaths. Given that there is neither a vaccine nor proven <hi>treatment</hi> for <hi>COVID</hi>-<hi>19</hi>, there is currently an urgent need for <hi>effective</hi> pharmacotherapy. To address the need for an <hi>effective</hi> <hi>treatment</hi> of SARS-CoV-2 during the worldwide pandemic, this systematic review of intravenous (IV) <hi>remdesivir</hi> was performed. <hi>Remdesivir</hi>, an anti-viral prodrug originally developed to treat Ebola virus disease, has shown broad spectrum activity against the Coronavirus family. A recent case report reported improvement of clinical symptoms with <hi>remdesivir</hi> in a patient with <hi>COVID</hi>-<hi>19</hi>. After<sep />',
'cord_uid': 'npoje09j'}},
{'id': 'index:content/0/afade29b15d5f83d8dd4d16a',
'relevance': 31.39580074485896,
'source': 'content',
'fields': {'title': '<hi>Remdesivir</hi> is <hi>effective</hi> for moderately severe patients: A re-analysis of the first double-blind, placebo-controlled, randomized trial on <hi>remdesivir</hi> for <hi>treatment</hi> of severe <hi>covid</hi>-<hi>19</hi> patients conducted in wuhan city',
'abstract': '<sep /> with baseline score, day of assessment, <hi>treatment</hi> group, baseline by <hi>treatment</hi> interaction, and day by <hi>treatment</hi> interaction as covariates. The binary endpoint was supported by the recent FDA<sep />oxygen (but not NIV/HFNC)) – the moderately severe category. The response rate was 85% for <hi>remdesivir</hi>-treated patients with baseline disease point=3 versus 70% response rate for likewise placebo-treated<sep />opinions. This result supports the preliminary findings of ACTT that <hi>remdesivir</hi> is <hi>effective</hi> for patients who were not critically severe. This result also suggests that<sep />',
'cord_uid': 'zpcuwvn4'}}]}}
Next steps
In the learntorank repository you will find the Query Model API. This is targeted for Data Science experiements.