Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] Error using the Annotator service in Linux

Paul R Alexander palexander at stanford.edu
Tue Feb 18 10:13:51 PST 2014


There is currently no throttling in place, though we will add something in the near future.

Paul


> On Feb 18, 2014, at 9:52 AM, Darren Weber <dlweber at stanford.edu> wrote:
> 
> 
> To evaluate the REST overload issue, you might try sleep, i.e.
> 
> http://docs.oracle.com/javase/tutorial/essential/concurrency/sleep.html
> 
> http://stackoverflow.com/questions/3342651/how-can-i-delay-a-java-program-for-a-few-seconds
> 
> If you throttle the client in this way, the REST service may not kick back some of your requests.  Although I haven't worked on the details, there may be code in the REST service to throttle excessive requests from a single user.  (Paul, could you comment on this?)
> 
> -- Darren
> 
> 
> 
>> On 2/18/14, 3:45 AM, Leyla Jael García Castro wrote:
>> Hi Darren, 
>> 
>> The problematic text, with the same algorithm, works fine in a Windows machine, but it returns an API key error message in Linux, also in Mac. The Linux and Mac machines are both faster than the Windows one, so it could be related with the number of requests per second to the annotator with the same API key. As it work in Windows, I think the problem is not the API key but something else. Is it possible that this API key error is reported when some other error has been thrown? Is it possible to get the full error somehow? 
>> 
>> Regards,
>> lj
>> 
>> 
>>> On Tue, Feb 18, 2014 at 12:15 AM, Darren Weber <dlweber at stanford.edu> wrote:
>>> 
>>> Hi Leyla,
>>> 
>>> If you have a specific example text that is always failing, we can try to replicate it, using your parameters (other than API key).  If it's not easily replicated, it's not likely we can resolve it.  Have you tried the problematic text in the web UI?
>>> 
>>> -- Darren
>>> 
>>> 
>>> 
>>>> On 2/17/14, 12:00 PM, Leyla Jael García Castro wrote:
>>>> Hi Darren,
>>>> 
>>>> Yes, my API key works fine with the URI http://data.bioontology.org/ontologies?apikey={YOUR_API_KEY}, I get a JSON file with a lot of information (administeredBy, acronym, name, summaryOnly, etc.).
>>>> 
>>>> The code that I sent before is used in a loop. We have a text with several paragraphs, all of the are sent to the annotator one by one. With all the others paragraphs it works fine. I have checked some other texts, and it seems to fail always with one of the last paragraphs. It makes me think that the error response is wrong, the API key is working with other paragraphs, there must be something else going on.
>>>> 
>>>> Thanks for your help,
>>>> lj
>>>> 
>>>>> On Mon, Feb 17, 2014 at 4:19 PM, Darren Weber <dlweber at stanford.edu> wrote:
>>>>> 
>>>>> Hi Leyla,
>>>>> 
>>>>> Can you login to the web site with that API key?  Does it work on this URI
>>>>> 
>>>>> http://data.bioontology.org/ontologies?apikey={YOUR_API_KEY}
>>>>> 
>>>>> 
>>>>> -- Darren
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 2/16/14, 1:07 PM, Leyla Jael García Castro wrote:
>>>>>> Hi Paul, 
>>>>>> 
>>>>>> We just realized that the parameter "stopWords" should be "stop_words", but that was not the problem. Nor is the number of requests that we do. Just tried to send all the paragraphs at once and it did not work,same error about the API key. Would it help if I send you the file we are annotating? Is RDF/XML.
>>>>>> 
>>>>>> Regards,
>>>>>> lj
>>>>>> 
>>>>>> 
>>>>>>> On Sat, Feb 15, 2014 at 9:55 PM, Leyla Jael García Castro <leylajael at gmail.com> wrote:
>>>>>>> Hi Paul, 
>>>>>>> 
>>>>>>> Here is the excerpt of the code that is calling the annotation. btw, I have been playing around with the text that is giving us troubles, for instance I moved it above other paragraphs that are correctly annotated. Then, it was another paragraph the one that gave us the error. Is there maybe a maximum API calls per second? Could be that why at some point the API key seems to be not valid any more? I tried to look for hidden chars but there was nothing special, just regular text with tabs and spaces.
>>>>>>> We also tried to replace the "+" added by the encoder instead of the spaces to "%20", and to spaces again, but  we got the same API key problem.
>>>>>>> 
>>>>>>> String ontologiesToAnnotate = "MDDB,MEDLINEPLUS,suicideo,MO,NCIT,NDDF,OMIM,FMA,ICD10,PW,PO,GO,MEDDRA,CHEBI,NDFRT,OBI,SNOMEDCT,SYMP,MESH";
>>>>>>> String annotatorURL = "http://data.bioontology.org/annotator";
>>>>>>> String stopwords = "a,able,about,above,abst,accordance,according,accordingly,across,act"; //and some others
>>>>>>> 
>>>>>>>         try {         
>>>>>>>         
>>>>>>>                                                 paragraph = paragraph.replaceAll("[^\\p{Alpha}\\p{Z}\\p{P}\\p{N}]", "_");        
>>>>>>>                                                 
>>>>>>>         
>>>>>>>                                                 paragraph = URLEncoder.encode(paragraph, Publication2RDF.UTF_ENCODING);
>>>>>>>         
>>>>>>>                                                 System.out.println("TO ANNOT: " + urlContext + "\n" + paragraph);
>>>>>>>             HttpClient client = new HttpClient();
>>>>>>>             client.getParams().setParameter(HttpMethodParams.USER_AGENT, "Annotator Client Scientific Publications");  //Set this string for your application 
>>>>>>>             
>>>>>>>             PostMethod method = new PostMethod(annotatorURL);
>>>>>>>             
>>>>>>>             // Configure the form parameters
>>>>>>>             method.addParameter("stopWords",stopWords);
>>>>>>>             method.addParameter("minimum_match_length","3");
>>>>>>>             method.addParameter("ontologies", ontologiesToAnnotate);            
>>>>>>>             method.addParameter("text", paragraph);
>>>>>>>             method.addParameter("format", "xml"); //Options are 'text', 'xml', 'tabDelimited'   
>>>>>>>             method.addParameter("apikey", Config.getNCBOAPIKey());
>>>>>>> 
>>>>>>>             // Execute the POST method
>>>>>>>             int statusCode = client.executeMethod(method);
>>>>>>>             
>>>>>>>             if( statusCode != -1 ) {
>>>>>>>                 try {
>>>>>>> 
>>>>>>>                                                               InputStream temporal  = method.getResponseBodyAsStream();
>>>>>>> 
>>>>>>>                                                                 //testing, we want to see the response and if it is ok we want to process it so we need it twice
>>>>>>>                                                                 
>>>>>>> 
>>>>>>>                                                                 ByteArrayOutputStream baos = new ByteArrayOutputStream();
>>>>>>> 
>>>>>>>                                                                 byte[] buf = new byte[1024];
>>>>>>> 
>>>>>>>                                                                 int n = 0;
>>>>>>> 
>>>>>>>                                                                 while ((n = temporal.read(buf)) >= 0) {
>>>>>>> 
>>>>>>>                                                                                                                     baos.write(buf, 0, n);
>>>>>>> 
>>>>>>>                                                                 }
>>>>>>> 
>>>>>>>                                                                 byte[] content = baos.toByteArray();
>>>>>>> 
>>>>>>>                                                                 InputStream annotatedParagraph = new ByteArrayInputStream(content);                                  
>>>>>>> 
>>>>>>>                                                                 if (annotatedParagraph != null) {
>>>>>>>                                                                  
>>>>>>> 
>>>>>>>                                                                  //Reader reader = new StringReader(annotatedParagraph);
>>>>>>> 
>>>>>>>                                                              JAXBContext jc = JAXBContext.newInstance("ws.biotea.ld2rdf.util.ncbo.annotator.jaxb.newgenerated");
>>>>>>> 
>>>>>>>                                                              Unmarshaller unmarshaller = jc.createUnmarshaller();
>>>>>>> 
>>>>>>>                                                              AnnotationCollection xml;
>>>>>>> 
>>>>>>>                                                              Object obj = new Object();
>>>>>>> 
>>>>>>>                                                              try {
>>>>>>>                                                              
>>>>>>> 
>>>>>>>                                                              obj = unmarshaller.unmarshal(annotatedParagraph);
>>>>>>> 
>>>>>>>                                                              if (obj instanceof Empty) {
>>>>>>> 
>>>>>>>                                                              return true; //no annotations were found but everything was ok with the response
>>>>>>> 
>>>>>>>                                                              }
>>>>>>> 
>>>>>>>                                                              xml = (AnnotationCollection)obj; //otherwise, AnnotationCollection should be the unmarshalled object
>>>>>>> 
>>>>>>>                                                              } catch (Exception e) {
>>>>>>> 
>>>>>>>                                                              InputStream temp = new ByteArrayInputStream(content);
>>>>>>> 
>>>>>>>                                                              String line;
>>>>>>> 
>>>>>>>                                                     
>>>>>>>                                                                 BufferedReader reader = new BufferedReader(new InputStreamReader(temp, "UTF-8"));
>>>>>>> 
>>>>>>>                                                     
>>>>>>>                                                                 while ((line = reader.readLine()) != null) {
>>>>>>> 
>>>>>>>                                                     
>>>>>>>                                                                  System.out.println(line);
>>>>>>> 
>>>>>>>                                                     
>>>>>>>                                                                 }
>>>>>>> 
>>>>>>>                                                     
>>>>>>>                                                                 reader.close();
>>>>>>> 
>>>>>>>                                                     
>>>>>>>                                                                 System.out.println(paragraph);
>>>>>>>             
>>>>>>>                                                 logger.fatal("- FATAL DTD ERROR ANNOTATOR - NCBO annotations for " + this.documentId + "(" + urlContext + ") cannot be unmarshalled: " + e.getMessage() + " - class: " + obj.getClass());
>>>>>>> 
>>>>>>>                                                              return false;
>>>>>>>                                                 
>>>>>>> 
>>>>>>>                                                              }
>>>>>>> [...]
>>>>>>> 
>>>>>>> then I just start processing the response. If you need, I can send you the actual java class, but I am afraid that it depends on some other libraries, jaxb, etc. so unless I share you the whole project it would not work. Please let me know should you need any further information.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> lj
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On Fri, Feb 14, 2014 at 11:11 PM, Paul R Alexander <palexander at stanford.edu> wrote:
>>>>>>>> How is the call being invoked? Can you share the code that is actually doing the call?
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Paul
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Feb 14, 2014, at 1:20 PM, Leyla Jael García Castro <leylajael at gmail.com> wrote:
>>>>>>>> 
>>>>>>>>> Hi all, 
>>>>>>>>> 
>>>>>>>>> I am using the Annotator service successfully on Windows but, what the odds, the same code with the same text is not working on Linux.
>>>>>>>>> 
>>>>>>>>> This is the text after URL enconding:
>>>>>>>>> 
>>>>>>>>> This+analysis+evidence+that+there+is+a+large+difference+in+efficiency+of+the+tested+methods+and+that+care+has+to+be+taken+when+deciding+which+method+to+be+used+for+the+analysis+of+missing+covariates+in+nonlinear+mixed+effects+modelling.
>>>>>>>>> 
>>>>>>>>> And this is the response we are getting:
>>>>>>>>> 
>>>>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>>>>> <hash>
>>>>>>>>>   <status>401</status>
>>>>>>>>>   <error>You must provide a valid API Key. Your API Key can be obtained by logging in at http://bioportal.bioontology.org/account</error>
>>>>>>>>> </hash>
>>>>>>>>> 
>>>>>>>>> Any suggestions? This is really puzzle me.
>>>>>>>>> 
>>>>>>>>> Best regards,
>>>>>>>>> lj
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> bioontology-support mailing list
>>>>>>>>> bioontology-support at lists.stanford.edu
>>>>>>>>> https://mailman.stanford.edu/mailman/listinfo/bioontology-support
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> bioontology-support mailing list
>>>>>> bioontology-support at lists.stanford.edu
>>>>>> https://mailman.stanford.edu/mailman/listinfo/bioontology-support
> 
> _______________________________________________
> bioontology-support mailing list
> bioontology-support at lists.stanford.edu
> https://mailman.stanford.edu/mailman/listinfo/bioontology-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/bioontology-support/attachments/20140218/a5d33f11/attachment-0001.html>


More information about the bioontology-support mailing list