[protege-discussion] DB Backend: Slowdown for classes with thousands of instances

[Cecil O. Lynch, MD, MS]

Hi Tania,

I hope this feature of limited instance retrieval is an optional switch. For
some use cases, the limitations of loading speed are forgivable and the need
to navigate the instances en masse is preferred despite the performance
penalty with the database calls.

Just my 2 cents.

Cecil

-----Original Message-----
From: protege-discussion-bounces at lists.stanford.edu
[mailto:protege-discussion-bounces at lists.stanford.edu] On Behalf Of Tania
Tudorache
Sent: Tuesday, October 31, 2006 4:59 PM
To: User support for Core Protege and the Protege-Frames editor
Subject: Re: [protege-discussion] DB Backend: Slowdown for classes with
thousands of instances

Hi Daniel,

As one would expect, the performance of the InstanceTab for database 
projects is worse than the one for the file mode, in which the whole 
ontology is loaded in memory. In the database mode, all ontology calls 
take much longer because of the database accessing/processing and 
retrieval time.

The problem with the InstancesTab is that it loads all instances of a 
class in memory. For each instance, it also gets the browser text, which 
involves several database calls. If you have 20.000 instances for a 
class, and a database call takes around 0.3 milliseconds, then 20.000 
calls will take around 6 seconds. These instances are cached, but the 
first time that you access them, it will take longer. The next time you 
browse the instances of the same class, you should have a much better 
performance, because the instances are read from the local cache. 
However, at some points the cache gets also deleted and it will have to 
be rebuilt.

You can read more about Protege performance and scalability on our wiki: 
http://protege.cim3.net/cgi-bin/wiki.pl?ScalabilityAndTuning

We have been thinking about a way to speed up the InstancesTab for large 
ontologies, and we have a solution that we will implement in one of our 
future beta releases. If the instances number of a class will exceed a 
certain number (for example, 1000), you will see only the first 1000 
instances in the instances list, and you can navigate to the next 1000 
instances by clicking on a forward or backward arrow. The instances list 
will look as it has different pages on which the user can navigate 
forward and backward, and each page will have a preset number of 
instances on it.

Tania


Daniel Holbert wrote:
> I'm running Protege using a database backend, and I get a huge slowdown
whenever I try to work 
> with a class that has more than a few thousand instances.  Has anyone else
experienced this, 
> and do you have any suggestions for dealing with it?
>
> I have one class that has 5,000 instances and another class that has
23,000 instances.  When 
> I'm in the "Instances" tab and I click on the names of these classes, the
whole Protege 
> program hangs for 15 seconds to load the 5000-instance class and about a
minute to load the 
> 23000-instance class.
>
> I've tried using both a local and a networked DB, and the slowdown
remains, so network latency 
> is not the bottleneck.  I've also tried using Oracle, MySQL, and
postgresql for the DB 
> backend, and I haven't found a noticeable difference in speed.
>
> Also, if I convert this project to a standard Protege project (without a
DB backend), then it 
> runs fairly quickly.  I've only experienced the major slowdown when using
a DB backend.
>
> Thanks!
> Daniel Holbert
> PharmGKB, http://www.pharmgkb.org
> Stanford University
> _______________________________________________
> protege-discussion mailing list
> protege-discussion at lists.stanford.edu
> https://mailman.stanford.edu/mailman/listinfo/protege-discussion
>
> Instructions for unsubscribing:
http://protege.stanford.edu/doc/faq.html#01a.03 
>
>   

_______________________________________________
protege-discussion mailing list
protege-discussion at lists.stanford.edu
https://mailman.stanford.edu/mailman/listinfo/protege-discussion

Instructions for unsubscribing:
http://protege.stanford.edu/doc/faq.html#01a.03 


-- 
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.13.21/509 - Release Date: 10/31/2006