Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] [BioPortal] Feedback from rakesh

Jennifer Leigh Vendetti vendetti at stanford.edu
Thu Aug 13 17:52:18 PDT 2020


Hi Rakesh,

I’m writing with a status report of sorts. Unfortunately, I haven’t found a resolution for this issue yet.

I spoke with our IT person who mentioned a possible Nginx limit on POST size of 1 GB. I examined the log files for the Rails application (/var/log/rails/appliance.log), and did see an indication of such an issue, e.g.:

F, [2020-08-13T00:08:39.434081 #3975] FATAL -- : MultiJson::ParseError (Problem loading json
<html>
<head><title>413 Request Entity Too Large</title></head>
<body>
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>nginx/1.16.1</center>
</body>
</html>
):

To prevent this error, I compressed the SNOMED CT TTL file to a ZIP archive (SNOMED.tt<http://SNOMED.tt>l.zip), which reduced the size to roughly 81 MB.

I made a second attempt to create a new submission for the SNOMED ontology:

1). Logged into the OntoPortal web application
2). Navigated to the ontology summary page (http://192.168.86.40/ontologies/SNOMEDCT)
3). Clicked the plus button at the top of the Submission table to create a new submission
4). Filled the required fields, choosing OWL as the Format, and specifying the compressed ontology source file as the upload

Clicking the Add submission button resulted in a success page displaying instead of the 500 error seen previously. However, when I later checked the submission processing status, the virtual machine showed an out of memory error and the processing failed to complete. I shut down the virtual machine, doubled the memory from 8 to 16 GB, restarted, and attempted the above process a second time. The processing got a bit further the second time, but ran out of memory again. I tried bumping the memory a couple more times, but was still getting failures at 32 GB and my laptop can’t handle experimenting with anything larger.

I’ll need to have another conversation with IT to see if there are any other possibilities.

Kind regards,
Jennifer


On Aug 12, 2020, at 7:05 PM, Rakesh Nagarajan <rakesh at pieriandx.com> wrote:

Jennifer,
     We'd love to upgrade to v3.0 of the appliance as soon as our DevOps has the bandwidth to do so. Thanks for checking on this and I am glad my issue is at least reproducible. I'll await further feedback from you after you've had a chance to connect with your colleagues tomorrow.

Thanks a ton,
Rakesh

On Wed, Aug 12, 2020 at 7:49 PM Jennifer Leigh Vendetti <vendetti at stanford.edu> wrote:
Hi Rakesh,

OK - I’m sorry to say that I don’t have a satisfactory answer for you at the moment. I installed a virtual appliance instance in my local development environment and attempted to upload the SNOMED CT TTL file. I reproduced the 500 error that you reported. Unfortunately, the log files where I would usually look for errors don’t contain anything enlightening (appliance.log, unicorn.stderr.log, unicorn.stdout.log). I’ve pinged our IT person in charge of packaging the appliance to see what other log files might contain error output, and will get back to you with what I find out.

A few things to note:

1). There's a new version of the appliance out (3.0.2). Not sure if this is of interest to you in terms of possibly upgrading. I was testing with 3.0.2, so it won’t help with your immediate issue.

2). The REST API logging is turned off by default in version 2.5 of the appliance. If you want to turn it on in order to examine log output, you’ll need to uncomment stderr_path and stdout_path in /srv/ncbo/ontologies_api/current/config/unicorn.rb. Again, this may not help with your immediate issue since I have REST API logging turned on in my local instance and the log files didn’t reveal anything obvious. If you turn on logging, the files should be located in /srv/ncbo/ontologies_api/current/log.

I should be able to speak with a couple of colleagues tomorrow and hope to have other tips for you.

Kind regards,
Jennifer


On Aug 12, 2020, at 2:32 PM, Rakesh Nagarajan <rakesh at pieriandx.com> wrote:

Jennifer,
     No worries at all. I did track this down last night and tried to use the API and again got an Internal Server Error. The submission itself isn't succeeding so there is no submission ID created on the server and so I can't check logs there. Is there a way to check any other internal application server (nginx, tomcat?) logs to see why the submission is failing? I have tried to restart the VM and flushed the goo cache per the instructions as well. I also validated the TTL file in Protege which was able to successfully load the complete SNOMEDCT file without errors. Any other ideas?

Thanks so much for your help,
Rakesh

On Wed, Aug 12, 2020 at 2:48 PM Jennifer Leigh Vendetti <vendetti at stanford.edu> wrote:
Sorry about that Rakesh - I meant to send a link to this script in our sample code repository:

https://github.com/ncbo/ncbo_rest_sample_code/blob/master/ruby/bioportal_ontologies_import.rb



On Aug 11, 2020, at 6:31 PM, Rakesh Nagarajan <rakesh at pieriandx.com> wrote:

Jennifer,
     Thanks! Can you confirm the last link where you refer to a script but that link is the same as the previous link?

Thanks,
Rakesh

On Tue, Aug 11, 2020 at 6:18 PM Jennifer Leigh Vendetti <vendetti at stanford.edu> wrote:
Hi Rakesh,


On Aug 10, 2020, at 5:18 PM, support at bioontology.org wrote:

Name: rakesh

Email: rakesh at pieriandx.com

Location: https%3A%2F%2Fbioportal.bioontology.org%2F


Feedback:

Hello,
I am using v2.5 of the Ontoportal appliance and have successfully loaded NCIT a few times now. I am now attempting to load SNOMED-CT after conversion from the UMLS using umls2rdf as a .ttl file. I have selected UMLS as the format but I have now tried twice to upload via UI and the server threw an error saying 'something went wrong and administrators have been notified. However I dont know what happened. Are there logs on the server to check?


There are some instructions for locating log files here:

https://ontoportal.github.io/administration/ontologies/submitting_ontologies/#is-there-a-log-file-for-parsing

Any other ideas to help troubleshoot? Finally are there commands to load directly on the server I may use?




I’m not certain this is what you’re asking for, but you can manually re-parse an ontology submission using this methodology:

https://ontoportal.github.io/administration/ontologies/submitting_ontologies/#manually-reparsing-using-the-ruby-console

It’s also possible to upload ontologies programmatically. An appliance user submitted the following script to our code sample repository:

https://ontoportal.github.io/administration/ontologies/submitting_ontologies/#manually-reparsing-using-the-ruby-console

It’s a script that imports ontologies from on appliance instance to another, which is not what you’re doing. However, I thought that some portions of the code might be of interest to you.

Kind regards,
Jennifer



--
Rakesh Nagarajan

Founder, President, and Chief Technology & Visionary Officer

m: 314-504-5620
e: rakesh at pieriandx.com



Wisdom in Every Report™

CONFIDENTIALITY NOTICE: This message and any attachments are solely for the use of the intended recipient and may contain privileged, confidential or other legally protected information. If you are not the intended recipient, please destroy all copies without reading or disclosing their contents and notify the sender of the error by reply email.








--
Rakesh Nagarajan

Founder, President, and Chief Technology & Visionary Officer

m: 314-504-5620
e: rakesh at pieriandx.com



Wisdom in Every Report™

CONFIDENTIALITY NOTICE: This message and any attachments are solely for the use of the intended recipient and may contain privileged, confidential or other legally protected information. If you are not the intended recipient, please destroy all copies without reading or disclosing their contents and notify the sender of the error by reply email.








--
Rakesh Nagarajan

Founder, President, and Chief Technology & Visionary Officer

m: 314-504-5620
e: rakesh at pieriandx.com



Wisdom in Every Report™

CONFIDENTIALITY NOTICE: This message and any attachments are solely for the use of the intended recipient and may contain privileged, confidential or other legally protected information. If you are not the intended recipient, please destroy all copies without reading or disclosing their contents and notify the sender of the error by reply email.





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/bioontology-support/attachments/20200814/f47a777b/attachment-0001.html>


More information about the bioontology-support mailing list