Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] [BioPortal] Feedback from Solene Grosdidier

Michael Dorf mdorf at stanford.edu
Fri Jun 14 15:15:38 PDT 2019


Hi Solene,

What you’re describing is a known issue: https://github.com/ncbo/ontologies_linked_data/issues/88.

Background: At some point in the past, we’ve implemented a system that prevents expensive COUNT queries going live against our 4store backend. These queries used to really bog down our servers, often resulting in downtime. The COUNT queries used to be executed on paged REST services, like the one that retrieves all mappings for a given ontology.  So, in order to determine the correct number of pages for a given call, our system used to first execute a COUNT query, storing the result in the output. The new system would pre-cache these counts, so when a paged service call is made, the count would be retrieved from a static repository. Unfortunately, there appears to be a bug in this process that triggers the behavior you are seeing.

For your specific example, it’s best to simply use an iterator to go through ALL pages of available mappings until you hit an empty collection instead of relying on the reported totalCount.

Thanks again for your report. Hope this works as a workaround for what you are trying to accomplish.

Michael


On Jun 14, 2019, at 4:08 AM, support at bioontology.org<mailto:support at bioontology.org> wrote:


Name: Solene Grosdidier

Email: s.grosdidier at erasmusmc.nl<mailto:s.grosdidier at erasmusmc.nl>

Location: https%3A%2F%2Fbioportal.bioontology.org<http://2Fbioportal.bioontology.org>%2Fmappings

Feedback:

Dear Bioportal team,

I am trying to retrieve all the mappings available in Bioportal between SNOMEDCT and NCIT through the API. Unfortunately, after the 321st page, I do get empty json for the following pages (reproducible on 2 different computers). Below is a copy-paste of my script.

#---------------------------------------------------------------------------------------------------------

#!/usr/bin/python3

import urllib.request, urllib.error, urllib.parse
import simplejson as json
import requests
REST_URL = "http://data.bioontology.org"
API_KEY = "2c84c2c2-3510-46fa-b7af-732659784401"

def get_json(url):
opener = urllib.request.build_opener()
opener.addheaders = [('Authorization', 'apikey token=' + API_KEY)]
return json.loads(opener.open(url).read())
#print(REST_URL + "/mappings?ontologies=MEDDRA,SNOMEDCT")
mapping = get_json(REST_URL+"/mappings?ontologies=NCIT,SNOMEDCT")
#print(json.dumps(mapping,indent=4))

pages = mapping["pageCount"]

print(str(pages))
for i in range (1,pages+1):
print("page: " + str(i))
mapping2 = get_json(REST_URL+"/mappings?ontologies=NCIT%2CSNOMEDCT&page=" + str(i))
#print(json.dumps(mapping2,indent=4))
try:
for element in mapping2["collection"]:
print(element["source"] + "\t" + element["classes"][0]["@id"]+ "\t" + element["classes"][1]["@id"])
except:
print("NO COLLECTION")
#-------------------------------------------------------------------------------------------------------------

Can you help me understanding what is happening?
Thank you very much for your help. I am looking forward to hearing from you.

Best,
Solene Grosdidier

_______________________________________________
bioontology-support mailing list
bioontology-support at lists.stanford.edu<mailto:bioontology-support at lists.stanford.edu>
https://mailman.stanford.edu/mailman/listinfo/bioontology-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/bioontology-support/attachments/20190614/5ff7fe7e/attachment.html>


More information about the bioontology-support mailing list