Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] [ISAtools] RE: Annotator query

Trish Whetzel whetzel at stanford.edu
Fri Dec 14 08:37:45 PST 2012


Hi Eamonn, 

I chatted with Nigam the other day about this and it was a design decision to have the Annotator recognize strings exactly as they are specified in the ontology, therefore an annotation problem. Where as partial string matching is a search problem and generally folks are used to seeing more than 1 result returned from a search versus having an exact match as the result of annotation. In this case, the methods being developed by the UMich folks may not be the appropriate solution but I wanted to loop them into the discussion in an attempt to collect additional use cases as relevant for their work. In general, when I work with research groups and they encounter this issue I suggest that text not matched to an ontology term via the Annotator be sent to the search web service to see if there are any partial matches. Nigam also suggested that this "two-step" feature may be able to be implemented as a parameter in the web service. However, any partial matches from search would need to be clearly flagged as such and these results most likely would need more human review to select an appropriate matching ontology term. 

Trish 


On Dec 14, 2012, at 4:41 AM, Eamonn Maguire wrote:

> Hi all,
> 
> So, I'm a bit more informed about how the algorithm works, however adding synonyms to me doesn't sound like a full solution to the issue. To me, making "hay fever" a synonym of "hay" or vice versa is incorrect. Similarly, "compound treatment design" is not the same as "compound treatment", I'm sure that some of the ontologists out there would have a fit of some sort if ontologies were to have such generalisations just for this use-case. 
> 
> The issue here is that between BioPortal and the Annotator, you'd expect the same  behaviour. So BioPortal works on partial string matches, whereas the Annotator does not.  The inconsistency is confusing our users. 
> 
> If this was solved, and the content in Annotator and BioPortal search was more harmonised, we'd be very grateful and are privy to giving out hugs. 
> 
> Eamonn
> 
> --
> Eamonn Maguire
> Oxford e-Research Centre & Computer Science Department
> University of Oxford
> 
> 7 Keble Road, Oxford, OX1 3QG
> T: 01865 (2) 610788 M: 07526 924483
> On Thursday, 13 December 2012 at 19:14, Meng, Fan wrote:
> 
>> Hi all:
>>  
>> I completely agree with Ray’s explanations. If a word or phrase is part of a concept but itself is not defined as a synonym of the concept or a different concepts,  Annotator will not identify that word or phrase as a match.  The only correct way for Annotator to find “compound treatment” is to define it as a synonym to “compound treatment design” in the ontology or define it as a new concept.
>>  
>> While we can modify the program to do partial match and to indicate the full concept string associated with the partial match, the result will likely be very messy.  In the example we discussed, “compound treatment”, “treatment design”, “compound”, “treatment”, “design” can be different concepts as well as be associated with many different concepts. Even if we limit the length of the partial phrase to be at least two words, “compound treatment” and  “treatment design” in this example are likely represent different concepts.
>>  
>> Best,
>>  
>> Fan
>>  
>> From: Ray Fergerson [mailto:ray.fergerson at stanford.edu] 
>> Sent: Thursday, December 13, 2012 1:56 PM
>> To: 'Susanna-Assunta Sansone'
>> Cc: isatools at googlegroups.com; Meng, Fan; 'Eamonn Maguire'; 'Trish Whetzel'; Dai, Manhong; support at bioontology.org
>> Subject: RE: [ISAtools] RE: [bioontology-support] Annotator query
>>  
>> Susanna,
>>  
>> I think that my example text was too short to get the idea across.  Here is the situation. The ontology contains two distinct concepts (1) “treatment” and (2) “compound treatment design”. For the purposes of this discussion, assume that neither term has any synonyms. The user passes in to the Annotator some free text such as:
>>  
>> “The patient was given a compound treatment: first we did treatment X and then we did treatment Y”.
>>  
>> The question which the Annotator is trying to answer is which concepts from the ontology are present in this free text. The current answer, and I believe the correct answer, is that the concept “treatment” appears three times and the concept “compound treatment design” does not appear at all. The Annotator only does a string comparison to extract concept names and synonyms and the string “compound treatment design” does not appear in the text. The annotator is not doing a “starts with” match. It is an exact match only.
>>  
>> The only way that the free text above is going to produce an annotator match to the “compound treatment design” concept is for this concept to have  a synonym that appears exactly in the free text.   
>>  
>> Ray
>>  
>>  
>>  
>> From: Susanna-Assunta Sansone [mailto:sa.sansone at gmail.com] 
>> Sent: Thursday, December 13, 2012 4:16 AM
>> To: Ray Fergerson
>> Cc: isatools at googlegroups.com; 'Meng, Fan'; 'Eamonn Maguire'; 'Trish Whetzel'; 'Dai, Manhong'; support at bioontology.org
>> Subject: Re: [ISAtools] RE: [bioontology-support] Annotator query
>>  
>> Hi Ray,
>>  
>> --
>> On 12/12/2012 21:55, Ray Fergerson wrote:
>> Eamonn,
>>  
>> In OBI there are two terms “treatment” and “compound treatment design”. If the text that you supply is “compound treatment” then this results in a match on the first term but not the second.
>> 
>> I do not get this sorry. If what you say below (about the term has to be a synonym to be found) is the case, then I do not believe that  “compound treatment” is a  synonym of "treatment" that is why this is found and  “compound treatment design” is not.
>> If the Annotator is supposed to suggest terms you have to be able to find both “treatment” and “compound treatment design” and then you leave the user to choose which one is more relevant to their context.
>> 
>> My two cents,
>> Susanna
>> 
>>  
>> This seems reasonable and it is difficult to see how it could work differently. If the ontology contained the term “hay fever” and  you supplied the text “hay” I don’t think that you should expect a match. The annotator is not a search system, it is a term matching system. “hay” is not the same term as “hay fever”.
>>  
>> If “compound treatment” were a declared synonym of “compound treatment design” then it should be found. If it is really a synonym (unclear to me) but not declared to be one in the ontology that you are using then you could consider extending the ontology or contacting the authors and asking them to add a synonym.
>>  
>> Ray
>>  
>> 
>> 
>>  
>> From: bioontology-support-bounces at lists.stanford.edu [mailto:bioontology-support-bounces at lists.stanford.edu] On Behalf Of Meng, Fan
>> Sent: Wednesday, December 12, 2012 12:49 PM
>> To: Eamonn Maguire; Trish Whetzel; Dai, Manhong
>> Cc: ISA Team; support at bioontology.org Support
>> Subject: Re: [bioontology-support] Annotator query
>>  
>> Hi Eamonn and Trish:
>>  
>> Thanks a lot for  your messages. If the terms “"compound treatment" and"compound treatment design" are both in an ontology, I believe mgrep will only give the longest match by default but list both terms as matches will be possible.
>> However, mgrep will not find them if none of them are in the ontologies you selected.
>>  
>> Manhong wrote the mgrep program and I believe it is possible to include that option although we may need to generate more complex dictionaries. Manhong can certainly provide better insights on this issue.
>>  
>> Best,
>>  
>> Fan
>>  
>> From: Eamonn Maguire [mailto:eamonnmag at gmail.com] 
>> Sent: Wednesday, December 12, 2012 12:48 PM
>> To: Trish Whetzel
>> Cc: Meng, Fan; support at bioontology.org Support; ISA Team
>> Subject: Re: [bioontology-support] Annotator query
>>  
>> Hi Trish,
>>  
>> Thanks for your reply. It would be great if this feature were integrated in to the annotator. Some users have been asking us why the annotator doesn't find things they expect it to, so it's good to have a response to this. I understand why you do it the way you do but I'd assume that an indexing system such as Lucene could help improve the speed for these types of lookups.
>>  
>> Thanks again,
>>  
>> Eamonn
>>  
>> --
>> Eamonn Maguire
>> Oxford e-Research Centre & Computer Science Department
>> University of Oxford
>>  
>> 7 Keble Road, Oxford, OX1 3QG
>> T: 01865 (2) 610788 M: 07526 924483
>> On Wednesday, 12 December 2012 at 17:45, Trish Whetzel wrote:
>> 
>> Hi Eamonn, 
>>  
>> I am not aware of any parameters that can be altered with the Annotator to get a match based on the scenario below. The dictionary that the Annotator uses is based on the preferred names and synonyms of ontology terms and there has to be an exact match in the input text to the dictionary term to return a match .. good for speed of annotation but does not handle term variations in the matching step unless the variation is included in the ontology itself.
>>  
>> Last I talked with the developers of the entity recognizer that the Annotator uses, called Mgrep, they have developed some methods to account for term permutations and there was an interest to collect use cases to test further. I have included Fan Meng in this response in case this is of interest to pursue further.
>>  
>> Best,
>> Trish 
>>  
>>  
>> On Dec 12, 2012, at 4:17 AM, Eamonn Maguire wrote:
>>  
>> Hi all,
>>  
>> I have a query about the annotator. 
>>  
>> It appears that if we search on something like "compound treatment", we don't get a hit for "compound treatment design" for instance from OBI/EFO. Do you have any idea about how we can get the search results to return a better match? Or if it's possible?
>>  
>> Many thanks,
>>  
>> Eamonn
>>  
>> --
>> Eamonn Maguire
>> Oxford e-Research Centre & Computer Science Department
>> University of Oxford
>>  
>> 7 Keble Road, Oxford, OX1 3QG
>> T: 01865 (2) 610788 M: 07526 924483
>> _______________________________________________
>> bioontology-support mailing list
>> bioontology-support at lists.stanford.edu
>> https://mailman.stanford.edu/mailman/listinfo/bioontology-support
>>  
>>  
>> -- 
>> ISA support team:
>> - Philippe Rocca-Serra
>> - Eamonn Maguire
>> - Alejandra Gonzalez-Beltran
>> - Susanna-Assunta Sansone
>> ***
>> www.isa-tools.org
>> www.isacommons.org
>>  
>>  
>>  
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/bioontology-support/attachments/20121214/2e370fbf/attachment-0001.html>


More information about the bioontology-support mailing list