Search Mailing List Archives

Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] anotator

Alexander Garcia Castro alexgarciac at
Wed Nov 15 10:38:01 PST 2017

Hi John, "In general, the only technical evaluation I've seen has shown
BioPortal first or second (if I recall correctly) in terms of speed and
completeness of annotation. " could u point me to the actual
document/paper/reference with this evaluation?

On Sat, Sep 30, 2017 at 12:32 AM, John Graybeal <jgraybeal at>

> Hi Alexander,
> A quick answer, sorry I don't have all the research data at hand but
> perhaps you can use Google Scholar to find more precise information if you
> need it.
> Since we consider BioPortal to be the biggest source of Biomedical
> Ontologies, we definitely consider our annotator the best service for
> annotations against a large and diverse number of ontologies. (And
> therefore, also the most reliable service in that category. :->)
> The article "Semantic annotation in biomedicine: the current landscape"
>  in Journal of Biomedical Semantics (https://jbiomedsem.
> has a survey of
> many, if not all, of the existing semantic annotation technologies. NCBO
> Bioportal appears in Tables 1 and 3 (scroll the table left on screen).
> While quite dated in some respects (BioPortal has over 500 BioMedical
> ontologies, and it is open source, available on GitHub) there is a lot of
> useful information here. I know there are some pipeline annotators that
> leverage BioPortal services to provide their own annotation capabilities,
> but can not offer the details off-hand, sorry.
> In general, the only technical evaluation I've seen has shown BioPortal
> first or second (if I recall correctly) in terms of speed and completeness
> of annotation. Of course, without specific evaluation criteria and
> comparisons, it is all but impossible to have a meaningful rebuttal to the
> observation "something else is better". Certainly there are features
> available with some other annotators that Bioportal does not provide, so
> there can be tradeoffs depending on specific needs. Conversely, among many
> other features, BioPortal supports both free text (paragraphs of content),
> and the ability to annotate the largest matching strings. These often are
> not available in other annotators.
> With regard to reliability, the most common concern raised about BioPortal
> is that it is not consistently responsive, especially for people with
> long-running and large-scale annotation processes and similar complex
> queries. Our monitoring services show the annotation service is up over
> 99.5% of the time, but it is definitely challenged by repeatedly annotating
> pages or more of text against all 500+ ontologies 15 times per second (per
> requestor), whether your text, or someone else's. We regularly get requests
> for the Virtual Appliance so that people can set up their own annotation
> pipeline, with only the ontologies they are interested in, and handling
> queries as fast as they can send them. That is an example of a more
> reliable service, but not a public one.
> Perhaps this information will be sufficient to help you, or will bring
> forward other comments from the user list.
> john
> On Sep 27, 2017, at 6:59 AM, Alexander Garcia Castro <
> alexgarciac at> wrote:
> Hi all. I am using the NCBO annotator and I like it. In a paper I just
> submitted the reviewer is saying that there are other NER tools that are
> better. this may be true but when processing the whole of PMC over a web
> service against all biomedical ontologies or a big portion of them then...
> is there anything as reliable as bioportal annotator? could some one help
> me out with some reference that justifies the choice of  the ncbo
> annotator? I am using it because I dont know if there is a web service that
> is as reliable when processing lots of texts and also that works against
> all of biomedical ontologies -in my case it is pretty much a choice due to
> the need to reuse other peoples infrastructure because I would not be able
> to process it on my own.
> also, there are lots of other NER tools but why was this one specifically
> selected for the NCBO NER service? as oppose to using another NER tool for
> the NCBO annotator.
> cheers.
> --
> Alexander Garcia
> _______________________________________________
> bioontology-support mailing list
> bioontology-support at
> ========================
> John Graybeal
> Technical Program Manager
> Center for Expanded Data Annotation and Retrieval /+/ NCBO BioPortal
> Stanford Center for Biomedical Informatics Research
> 650-736-1632

Alexander Garcia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the bioontology-support mailing list