Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] Error using the Annotator service in Linux

Darren Weber dlweber at stanford.edu
Tue Feb 18 09:52:11 PST 2014


To evaluate the REST overload issue, you might try sleep, i.e.

http://docs.oracle.com/javase/tutorial/essential/concurrency/sleep.html

http://stackoverflow.com/questions/3342651/how-can-i-delay-a-java-program-for-a-few-seconds

If you throttle the client in this way, the REST service may not kick 
back some of your requests.  Although I haven't worked on the details, 
there may be code in the REST service to throttle excessive requests 
from a single user.  (Paul, could you comment on this?)

-- Darren



On 2/18/14, 3:45 AM, Leyla Jael García Castro wrote:
> Hi Darren,
>
> The problematic text, with the same algorithm, works fine in a Windows 
> machine, but it returns an API key error message in Linux, also in 
> Mac. The Linux and Mac machines are both faster than the Windows one, 
> so it could be related with the number of requests per second to the 
> annotator with the same API key. As it work in Windows, I think the 
> problem is not the API key but something else. Is it possible that 
> this API key error is reported when some other error has been thrown? 
> Is it possible to get the full error somehow?
>
> Regards,
> lj
>
>
> On Tue, Feb 18, 2014 at 12:15 AM, Darren Weber <dlweber at stanford.edu 
> <mailto:dlweber at stanford.edu>> wrote:
>
>
>     Hi Leyla,
>
>     If you have a specific example text that is always failing, we can
>     try to replicate it, using your parameters (other than API key). 
>     If it's not easily replicated, it's not likely we can resolve it. 
>     Have you tried the problematic text in the web UI?
>
>     -- Darren
>
>
>
>     On 2/17/14, 12:00 PM, Leyla Jael García Castro wrote:
>>     Hi Darren,
>>
>>     Yes, my API key works fine with the URI
>>     http://data.bioontology.org/ontologies?apikey={YOUR_API_KEY}, I
>>     get a JSON file with a lot of information (administeredBy,
>>     acronym, name, summaryOnly, etc.).
>>
>>     The code that I sent before is used in a loop. We have a text
>>     with several paragraphs, all of the are sent to the annotator one
>>     by one. With all the others paragraphs it works fine. I have
>>     checked some other texts, and it seems to fail always with one of
>>     the last paragraphs. It makes me think that the error response is
>>     wrong, the API key is working with other paragraphs, there must
>>     be something else going on.
>>
>>     Thanks for your help,
>>     lj
>>
>>     On Mon, Feb 17, 2014 at 4:19 PM, Darren Weber
>>     <dlweber at stanford.edu <mailto:dlweber at stanford.edu>> wrote:
>>
>>
>>         Hi Leyla,
>>
>>         Can you login to the web site with that API key?  Does it
>>         work on this URI
>>
>>         http://data.bioontology.org/ontologies?apikey={YOUR_API_KEY}
>>
>>
>>         -- Darren
>>
>>
>>
>>         On 2/16/14, 1:07 PM, Leyla Jael García Castro wrote:
>>>         Hi Paul,
>>>
>>>         We just realized that the parameter "stopWords" should be
>>>         "stop_words", but that was not the problem. Nor is the
>>>         number of requests that we do. Just tried to send all the
>>>         paragraphs at once and it did not work,same error about the
>>>         API key. Would it help if I send you the file we are
>>>         annotating? Is RDF/XML.
>>>
>>>         Regards,
>>>         lj
>>>
>>>
>>>         On Sat, Feb 15, 2014 at 9:55 PM, Leyla Jael García Castro
>>>         <leylajael at gmail.com <mailto:leylajael at gmail.com>> wrote:
>>>
>>>             Hi Paul,
>>>
>>>             Here is the excerpt of the code that is calling the
>>>             annotation. btw, I have been playing around with the
>>>             text that is giving us troubles, for instance I moved it
>>>             above other paragraphs that are correctly annotated.
>>>             Then, it was another paragraph the one that gave us the
>>>             error. Is there maybe a maximum API calls per second?
>>>             Could be that why at some point the API key seems to be
>>>             not valid any more? I tried to look for hidden chars but
>>>             there was nothing special, just regular text with tabs
>>>             and spaces.
>>>             We also tried to replace the "+" added by the encoder
>>>             instead of the spaces to "%20", and to spaces again, but
>>>              we got the same API key problem.
>>>
>>>             String ontologiesToAnnotate =
>>>             "MDDB,MEDLINEPLUS,suicideo,MO,NCIT,NDDF,OMIM,FMA,ICD10,PW,PO,GO,MEDDRA,CHEBI,NDFRT,OBI,SNOMEDCT,SYMP,MESH";
>>>             String annotatorURL =
>>>             "http://data.bioontology.org/annotator";
>>>             String stopwords =
>>>             "a,able,about,above,abst,accordance,according,accordingly,across,act";
>>>             //and some others
>>>
>>>                     try {
>>>             paragraph =
>>>             paragraph.replaceAll("[^\\p{Alpha}\\p{Z}\\p{P}\\p{N}]",
>>>             "_");
>>>             paragraph = URLEncoder.encode(paragraph,
>>>             Publication2RDF.UTF_ENCODING);
>>>             System.out.println("TO ANNOT: " + urlContext + "\n" +
>>>             paragraph);
>>>             HttpClient client = new HttpClient();
>>>             client.getParams().setParameter(HttpMethodParams.USER_AGENT,
>>>             "Annotator Client Scientific Publications");  //Set this
>>>             string for your application
>>>             PostMethod method = new PostMethod(annotatorURL);
>>>                         // Configure the form parameters
>>>             method.addParameter("stopWords",stopWords);
>>>             method.addParameter("minimum_match_length","3");
>>>             method.addParameter("ontologies", ontologiesToAnnotate);
>>>             method.addParameter("text", paragraph);
>>>             method.addParameter("format", "xml"); //Options are
>>>             'text', 'xml', 'tabDelimited'
>>>             method.addParameter("apikey", Config.getNCBOAPIKey());
>>>
>>>                         // Execute the POST method
>>>                         int statusCode = client.executeMethod(method);
>>>                         if( statusCode != -1 ) {
>>>                             try {
>>>              InputStream temporal  = method.getResponseBodyAsStream();
>>>              //testing, we want to see the response and if it is ok
>>>             we want to process it so we need it twice
>>>              ByteArrayOutputStream baos = new ByteArrayOutputStream();
>>>              byte[] buf = new byte[1024];
>>>              int n = 0;
>>>              while ((n = temporal.read(buf)) >= 0) {
>>>                baos.write(buf, 0, n);
>>>                            }
>>>              byte[] content = baos.toByteArray();
>>>              InputStream annotatedParagraph = new
>>>             ByteArrayInputStream(content);
>>>              if (annotatedParagraph != null) {
>>>             //Reader reader = new StringReader(annotatedParagraph);
>>>             JAXBContext jc =
>>>             JAXBContext.newInstance("ws.biotea.ld2rdf.util.ncbo.annotator.jaxb.newgenerated");
>>>             Unmarshaller unmarshaller = jc.createUnmarshaller();
>>>             AnnotationCollection xml;
>>>             Object obj = new Object();
>>>             try {
>>>             obj = unmarshaller.unmarshal(annotatedParagraph);
>>>             if (obj instanceof Empty) {
>>>             return true; //no annotations were found but everything
>>>             was ok with the response
>>>             }
>>>             xml = (AnnotationCollection)obj; //otherwise,
>>>             AnnotationCollection should be the unmarshalled object
>>>             } catch (Exception e) {
>>>             InputStream temp = new ByteArrayInputStream(content);
>>>             String line;
>>>              BufferedReader reader = new BufferedReader(new
>>>             InputStreamReader(temp, "UTF-8"));
>>>              while ((line = reader.readLine()) != null) {
>>>             System.out.println(line);
>>>                            }
>>>              reader.close();
>>>              System.out.println(paragraph);
>>>             logger.fatal("- FATAL DTD ERROR ANNOTATOR - NCBO
>>>             annotations for " + this.documentId + "(" + urlContext +
>>>             ") cannot be unmarshalled: " + e.getMessage() + " -
>>>             class: " + obj.getClass());
>>>             return false;
>>>             }
>>>             [...]
>>>
>>>             then I just start processing the response. If you need,
>>>             I can send you the actual java class, but I am afraid
>>>             that it depends on some other libraries, jaxb, etc. so
>>>             unless I share you the whole project it would not work.
>>>             Please let me know should you need any further information.
>>>
>>>             Regards,
>>>             lj
>>>
>>>
>>>
>>>             On Fri, Feb 14, 2014 at 11:11 PM, Paul R Alexander
>>>             <palexander at stanford.edu
>>>             <mailto:palexander at stanford.edu>> wrote:
>>>
>>>                 How is the call being invoked? Can you share the
>>>                 code that is actually doing the call?
>>>
>>>                 Thanks,
>>>                 Paul
>>>
>>>
>>>                 On Feb 14, 2014, at 1:20 PM, Leyla Jael García
>>>                 Castro <leylajael at gmail.com
>>>                 <mailto:leylajael at gmail.com>> wrote:
>>>
>>>>                 Hi all,
>>>>
>>>>                 I am using the Annotator service successfully on
>>>>                 Windows but, what the odds, the same code with the
>>>>                 same text is not working on Linux.
>>>>
>>>>                 This is the text after URL enconding:
>>>>
>>>>                 This+analysis+evidence+that+there+is+a+large+difference+in+efficiency+of+the+tested+methods+and+that+care+has+to+be+taken+when+deciding+which+method+to+be+used+for+the+analysis+of+missing+covariates+in+nonlinear+mixed+effects+modelling.
>>>>
>>>>                 And this is the response we are getting:
>>>>
>>>>                 <?xml version="1.0" encoding="UTF-8"?>
>>>>                 <hash>
>>>>                 <status>401</status>
>>>>                 <error>You must provide a valid API Key. Your API
>>>>                 Key can be obtained by logging in at
>>>>                 http://bioportal.bioontology.org/account</error>
>>>>                 </hash>
>>>>
>>>>                 Any suggestions? This is really puzzle me.
>>>>
>>>>                 Best regards,
>>>>                 lj
>>>>
>>>>                 _______________________________________________
>>>>                 bioontology-support mailing list
>>>>                 bioontology-support at lists.stanford.edu
>>>>                 <mailto:bioontology-support at lists.stanford.edu>
>>>>                 https://mailman.stanford.edu/mailman/listinfo/bioontology-support
>>>
>>>
>>>
>>>
>>>
>>>         _______________________________________________
>>>         bioontology-support mailing list
>>>         bioontology-support at lists.stanford.edu  <mailto:bioontology-support at lists.stanford.edu>
>>>         https://mailman.stanford.edu/mailman/listinfo/bioontology-support
>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/bioontology-support/attachments/20140218/11be3817/attachment-0001.html>


More information about the bioontology-support mailing list