Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start

Paul R Alexander palexander at stanford.edu
Thu Apr 11 10:45:10 PDT 2013


I'm not sure what you mean by publicly available in this context. Unless you have marked the ontologies as "private" in the Appliance somehow then they will be available to anyone who has network access to the machine.

Some of the ontologies can take a considerable amount of RAM to parse, sometimes 6-8GB.

Paul


On Apr 11, 2013, at 7:14 AM, "Stephen Granite" <sgranite at jhu.edu> wrote:

> Paul,
>  
> Thanks for all your previous help.  We were able to get the script to run after incrementing the space available to 100GB.  Actually 30GB was used.  However, we received a number of parsing errors on ontologies and ran into a DataFactory out of memory error.  We have 6GB RAM allocated on the VM and 2GB min/4GB max allocated to tomcat.  We’re thinking that we’re going to start from scratch and attempt to run the script under the admin API key account (we had done this under my account on our appliance instance, which may have caused the errors, but we are unsure).  As a side question, once all the ontologies are loaded, how does one make them publicly available on the appliance instance?
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Friday, March 22, 2013 2:02 PM
> To: Stephen Granite
> Cc: 'Kyle Reynolds'; 'Trish Whetzel'; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Stephen,
>  
> Size requirements are completely dependent on which ontologies you are trying to import and how many versions each of them has. Our total storage for ontology files (taking into account multiple versions) is around 100GB. The search index takes about 18GB. The databases containing the parsed ontologies is even larger.
>  
> Again, this is for our entire production system, which contains multiple versions of many ontologies. However, it gives you a rough idea of the scale.
>  
> For $INCLUDE_ONTOLOGIES you can just list the ontologyIds (aka virtual ids) as comma-seperated integers.
>  
> Paul
>  
>  
> On Mar 22, 2013, at 5:28 AM, "Stephen Granite" <sgranite at jhu.edu> wrote:
> 
> 
> Paul,
>  
> Thanks again for your previous guidance.  We attempted to use the import_ontologies script, but have run into space issues in the virtual appliance.  The original VM is 8 GB and we expanded it to 20GB, but still ran into a space issue.  What is a reasonable size to place the VM at, to handle importing ontologies?
>  
> Also, the directions say that $INCLUDE_ONTOLOGIES can be uncommented and used to import specific ontologies. What is meant to be entered in the $INCLUDE_ONTOLOGIES parameter?
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Thursday, March 14, 2013 2:19 PM
> To: Stephen Granite
> Cc: 'Kyle Reynolds'; 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> The user id number.
>  
> Paul
>  
>  
> On Mar 14, 2013, at 11:18 AM, "Stephen Granite" <sgranite at jhu.edu> wrote:
> 
> 
> 
> Paul,
>  
> Thanks for the guidance.  We’ll give it a shot.  One last question, is the user id for $ONTOLOGY_OWNER my username or the id number?
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Thursday, March 14, 2013 1:47 PM
> To: Stephen Granite
> Cc: 'Kyle Reynolds'; 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Ok, so my guess is that the lack of ontologies in the system is causing the error you are seeing. As you can imagine, we don't have many situations where we have to handle a completely empty ontology library, so there are possibilities for errors when the Appliance is in this state. I've filed a bug report so we can investigate this further.
>  
> As for the migration, you will want two api keys from two different systems. The first should be from an account you have with us on the BioPortal site: http://bioportal.bioontology.org/account. This is what you would use in the "FROM" option.
>  
> The other should be a key from your Appliance, either from the admin user or a user who you have created. This would be used in the "TO" option.
>  
> Let me know if you have any other questions.
>  
> Paul
>  
>  
> On Mar 14, 2013, at 10:21 AM, "Stephen Granite" <sgranite at jhu.edu> wrote:
> 
> 
> 
> 
> Paul,
>  
> We had not put any ontologies in yet because we were trying to follow the instructions in the Appliance FAQ for ontology migration:http://www.bioontology.org/wiki/index.php/Virtual_Appliance_FAQ#How_can_I_migrate_ontologies_from_BioPortal_or_previous_NCBO_Virtual_Appliance_versions_into_a_new_Appliance.3F
> I may have confused things slightly by thinking that the API key referenced in those instructions was my user API key, rather than the system API key.  So in order to migrate ontologies into our instance of the NCBO appliance, what are the values that we need for the “FROM” and “TO” API Keys?  Thanks.
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Thursday, March 14, 2013 12:47 PM
> To: Stephen Granite
> Cc: 'Kyle Reynolds'; 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Stephen,
>  
> Are there any ontologies submitted to your system?
>  
>  
> On Mar 14, 2013, at 6:34 AM, "Stephen Granite" <sgranite at jhu.edu> wrote:
> 
> 
> 
> 
> 
> Paul,
>  
> We followed your instructions and the previous errors have disappeared.  However, when I attempted to update my account (sgranite) to see my API key or make a new account (sjagranite), I received stack traces in the browser and the script/server log.  I’ve attached the log and html pages from the browser views of the same errors.  As a separate test, instead of using the update button on my account, I went to reset my password.  After I reset my password, I did get back the proper API key screen.  I’ve included just the HTML from that, as the last attachment.
>  
> Is there anything else we need to change?
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Wednesday, March 13, 2013 6:52 PM
> To: Stephen Granite
> Cc: 'Kyle Reynolds'; 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Stephen,
>  
> Sorry, one more modification. You'll need to stop running the server (ctrl+c), then modify this file:
> /srv/ncbo/rails/BioPortal/current/config/database.yml
>  
> Replace the contents of the 'development' section with the contents from the 'production' section.
>  
> Then, restart the server with script/server and the error you sent me should go away.
>  
> Paul
>  
>  
> On Mar 13, 2013, at 11:18 AM, Stephen Granite <sgranite at jhu.edu> wrote:
> 
> 
> 
> 
> 
> 
> Paul,
>  
> Attached is the spool of the output we received when following your instructions.  We ran the script in /srv/ncbo/rails/BioPortal/current.  It looks like it is having issues connecting to the mySQL database, but that is just a guess on my part, based upon the error shown.
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Tuesday, March 12, 2013 7:18 PM
> To: Stephen Granite
> Cc: 'Kyle Reynolds'; 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> No, the UI and REST services are designed to run on separate ports. Unfortunately, I can't venture a guess as to why the account creation/display process seems to be failing, especially given that they work on my version of the Appliance.
>  
> One more thing you could try is shutting down apache httpd on the machine and running the Rails app directly by doing the following while logged in as root:
> service httpd stop
> cd /srv/rails/BioPortal/current
> script/server -p 80
>  
> Then, visit the account page. When you get a failure, the stack trace should get output to the terminal where you ran the script/server command. Using that we might be able to figure it out.
>  
> Paul
>  
>  
> On Mar 11, 2013, at 11:59 AM, Stephen Granite <sgranite at jhu.edu> wrote:
> 
> 
> 
> 
> 
> 
> 
> Paul,
>  
> Sorry for the confusion.  We took the OVF from bioportal and converted/implemented it in VMware, with no changes except for IP address.  http://{your appliance IP or domain} responds and presents the web UI.  http://{your appliance IP or domain}:8080/bioportal/auth?username… responds fine with the API key information.
>  
> The question I posed is whether the UI and the REST services should both be running out of port 8080.  I ask because I’m confused as to why I can sign into BioPortal on 80 and the UI seems to be working, but none of the account capabilities provided by the UI (i.e., account creation, obtaining the API key, etc.) seem to be working, even though the REST services are operational and functioning.  Thanks.
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Monday, March 11, 2013 2:50 PM
> To: Stephen Granite
> Cc: 'Kyle Reynolds'; 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Sorry, I'm confused. Which portion of the infrastructure is running on port 80? The web UI? Or the REST services? Or both on separate machines?
>  
> If the REST service is not running on port 8080, then you will need to modify the UI's configuration file in order to have things work properly. You want to replace instances of 8080 with 80. The file is located here:
> /srv/ncbo/src/config/bioportalui/BioPortal/config/bioportal_config.rb
>  
> After saving, you'll need to do a `bpdeploy deployui` command to have it take effect.
>  
> Paul
>  
>  
> On Mar 11, 2013, at 11:12 AM, "Stephen Granite" <sgranite at jhu.edu> wrote:
> 
> 
> 
> 
> 
> 
> 
> 
> Paul,
>  
> Thanks for your help and guidance.  I ran the REST command below (edited for our IP and my account) and received this back:
>  
> <success>
> <accessedResource>/bioportal/auth</accessedResource>
> <accessDate>2013-03-11 11:09:22.628 PDT</accessDate>
> <data>
> <session>
> <attributes>
> <entry>
> <string>SECURITY_CONTEXT</string>
> <securityContext>
> <apiKey>6ef75be7-3571-4b7b-8d75-b51e0168270d</apiKey>
> <userBean>
> <id>16</id>
> <username>sgranite</username>
> <email>sgranite at jhu.edu</email>
> <firstname>Stephen</firstname>
> <lastname>Granite</lastname>
> <dateCreated class="sql-timestamp">2013-03-11 11:07:55.0</dateCreated>
> <roles>
> <string>ROLE_LIBRARIAN</string>
> </roles>
> <ontologyAcl/>
> <ontologyLicenses/>
> </userBean>
> </securityContext>
> </entry>
> </attributes>
> <creationTime>0</creationTime>
> <id>6ef75be7-3571-4b7b-8d75-b51e0168270d</id>
> <lastAccessedTime>0</lastAccessedTime>
> <isNew>false</isNew>
> <isValid>true</isValid>
> <lastUsedTime>0</lastUsedTime>
> </session>
> </data>
> </success>
>  
> While it is good that I got this back, I will note that we are running the BioPortal on port 80, rather than 8080, on the implementation of the appliance.  Should we be running everything through 8080?  Would that explain why I am getting the error messages?
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Kyle Reynolds [mailto:kyle.reynolds at jhu.edu] 
> Sent: Monday, March 11, 2013 2:06 PM
> To: 'Paul R Alexander'; Stephen Granite
> Cc: 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: RE: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> FYI, I took a look for the log and there was nothing there. The directory structure exists, but no production log was created yet.
> -Kyle
>  
> From: Paul R Alexander [mailto:palexander at stanford.edu] 
> Sent: Monday, March 11, 2013 12:59 PM
> To: Stephen Granite
> Cc: 'Trish Whetzel'; ksr at jhu.edu; 'NCBO User Support'
> Subject: Re: [bioontology-support] Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Unfortunately, I cannot reproduce either the error you get when creating an account or the error you get when trying to view the API key. Here is the log location on the Appliance. If you can see an error there I may be able to help figure it out:
> /var/log/rails/BioPortal/production.log
>  
> There is also a workaround. You can authorize an account using the REST service:
> http://{your appliance IP or domain}:8080//bioportal/auth?username={your username}&password={your password}&apikey=11111111-9dd1-11e0-a996-115056bd0024
>  
> Just replace the brackets with your information. The API key in the URL is the one included on the Appliance by default.
>  
> Paul
>  
>  
> On Mar 11, 2013, at 6:53 AM, "Stephen Granite" <sgranite at jhu.edu> wrote:
>  
> 
> Trish,
>  
> In order for us to load ontologies in, I need an API key for both Bioportal and the appliance here.  As I said before, the account here exists.  However, when I go to get the API key, I receive the same message I got when I tried to make the account in the first place: We're sorry but something has gone wrong. We have been notified of this error. 
>  
> What should we be checking for regarding this?  Is this meant to spool out in an error log somewhere?
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Trish Whetzel [mailto:whetzel at stanford.edu] 
> Sent: Friday, March 08, 2013 5:11 PM
> To: Stephen Granite
> Cc: 'Paul R Alexander'; 'NCBO User Support'; ksr at jhu.edu
> Subject: Re: Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Yes, however I do not know if that was a one-time submit web service error or a general bug in the create new user function included in that release.
>  
> Trish 
>  
>  
> On Mar 8, 2013, at 9:05 AM, Stephen Granite wrote:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Trish and Paul,
>  
> On a tangential note, I had tried to make an account in the BioPortal instance, but the interface said there was an error.  However, when I tried to make the same account again, the interface came back saying that the account in error existed.  I was then able to sign in with the account.  Does that make sense?
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
> From: Trish Whetzel [mailto:whetzel at stanford.edu] 
> Sent: Friday, March 08, 2013 11:58 AM
> To: Stephen Granite; Paul R Alexander
> Cc: NCBO User Support; ksr at jhu.edu
> Subject: Re: Issues bringing up the NCBO Virtual Appliance: mgrep does not start
>  
> Mgrep is used by the Annotator, however I do not know if having this start successfully is dependent on having ontologies populated in the Annotator database. Have you also populated ontologies in the Annotator database? As for the dependency, Paul can you comment on this?
>  
> Trish 
>  
>  
> On Mar 8, 2013, at 5:57 AM, Stephen Granite wrote:
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> In the CVRG, we are interested in bringing up an instance of the NCBO applicance, to serve as a sandbox for ontologies we develop, to harden them prior to submission to NCBO.  We downloaded a copy of the version 1.0 OVF and were able to get the BioPortal web interface to come up without much effort.  However, in the startup, one service fails to start properly: mgrep.  mgrep is supposed to start from init.d, but when we execute “service mgrep status,” the response says the service is dead and the subsystem is locked.  We don’t see anything showing up in logs about mgrep.  What would you advise that we do?  Thanks.
>  
> Stephen J. Granite, MS, MBA
> Director of Database and Software Development
> Center for Cardiovascular Bioinformatics and Modeling
> Institute for Computational Medicine
> The Johns Hopkins University
> 3400 North Charles Street
> Hackerman Hall 312
> Baltimore, MD 21218-2686
> phone: 410-516-8849
> fax: 410-516-5294
>  
>  
>  
> _______________________________________________
> bioontology-support mailing list
> bioontology-support at lists.stanford.edu
> https://mailman.stanford.edu/mailman/listinfo/bioontology-support
>  
>  
> <NCBO_Error.txt>
>  
> <NCBO_Error_20130313.txt><Action_Controller_Exception_caught_create.htm><Action_Controller_  Exception_caught_update.htm><Account Information   NCBO BioPortal.htm>
>  
>  
>  

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/bioontology-support/attachments/20130411/f9c0bd3b/attachment-0001.html>


More information about the bioontology-support mailing list