Search Mailing List Archives

Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[bioontology-support] attempting to make a 1.9.3 release of CHEMINF

Jennifer Leigh Vendetti vendetti at
Tue Mar 26 17:06:11 PDT 2019

Hello Egon,

Apologies for the delay in responding. Our support list has been unusually busy for the last couple of weeks.

On Mar 19, 2019, at 12:50 AM, Egon Willighagen <egon.willighagen at<mailto:egon.willighagen at>> wrote:

Dear Jennifer,

Thanks for the detailed update! Very much appreciated.

In my memory, the previous time I had the problem, I could manually update the version (well, not, bc I ran into that problem), and I don't think it was WIKIPATHWAYS; the fact that I'm not even admin on that one (Anders Riutta, our collab in SF is), seems to support that.

Also, it doesn't quite explain to me why it did not pick up the 1.9.2 release that was made in September last year. Many nights have passed since then:

I looked at the commit history for your ontology file. I believe the problem with creation of a corrupt submission object initially occurred in September of last year when the system tried to pull the version that was committed on September 13, 2018. This would have been submission 45 since the one prior was submission 44 (from August 29, 2018).

I can’t confirm that 100% because we only have log files for the ontology pull mechanism going back to Jan. 15th of this year. I do see the error message about the corrupt submission object at the beginning of the log file, so we've been attempting (and failing) to pull 1.9.3 since at least as far back as January 15th.

Anyway, it's not too urgent. I was just hoping the event and info would help you, as much as me.

Yes, it’s helpful in the sense that it alerts us to the need to have monitoring in place for this particular error message in the pull mechanism log files, until we’re able to track down / fix the underlying issue.


PS. do you (BioPortal) do (occasional) invoices, for "Support provided in 2018 for CHEMINF and ENM ontologies", or something like that, for, say, 100$, addressed as specific people (me, in this case)? If so, I would wish and try if I can get my university to pay for that. I know it won't solve the BioPortal sustainability, but I would expect that showing paying customers may help. Then again, just as likely, it may backfire under the statement of funders like: "But you already have paying people”.

I don’t know the answer to that question offhand, but I’ll forward this to our Technical Program Manager (John Graybeal).

Kind regards,

On Tue, Mar 19, 2019 at 12:27 AM Jennifer Leigh Vendetti <vendetti at<mailto:vendetti at>> wrote:

On Mar 16, 2019, at 1:23 AM, Egon Willighagen <egon.willighagen at<mailto:egon.willighagen at>> wrote:

On Fri, Mar 15, 2019 at 8:33 PM Jennifer Leigh Vendetti <vendetti at<mailto:vendetti at>> wrote:
This generally means that at some point we tried to pull and create a new submission for your ontology and something went wrong during the process. This results in a corrupt submission object in our backend triplestore, and subsequent attempts to create new submissions fail. I deleted the corrupt object, and the system should be able to successfully pull your new version overnight tonight. I’ll double check that to make sure everything goes OK.

I have run into the same issue before and back then the answer was basically the same.

I searched our publicly available mailing list archive so that I could refresh my memory regarding our last conversation [1]. In October of 2017 you contacted us about the WIKIPATHWAYS ontology [2]. BioPortal wasn’t able to pull new versions of that ontology because our system was configured to use a pull URL that resulted in a 404 error. You provided us with an updated URL, and I modified the submission information for WIKIPATHWAYS to use the new URL. If BioPortal is unable to fetch new versions of an ontology, the system generates an email notification and sends it to the list of users that have administrative privileges for the ontology. For WIKIPATHWAYS, it looks like the two accounts that have administrative privileges are “ariutta” and “wikipathways”.

The problem with CHEMINF has nothing to do with an invalid pull URL, but rather a bug in our system where corrupt submission objects occasionally occur and prevent newer versions of ontologies from getting loaded on a nightly basis. The first occurrence of the issue is documented in our GitHub repository [3]. We haven’t tracked down the source of this problem yet. It happens infrequently, never with the same ontology, and so far hasn’t been reproducible in our local development environments.

So, I *had* waited with making a manual release (causing the above issue) hoping BioPortal would pick up the 1.9.3 (or 1.9.2 before that) automatically... but it had not. Is there anyway I can see the logs of the attempts of the nightly checks for newer ontology releases to see why it does not normally pick up new releases?

Our system log files aren’t publicly available. I included the error message in my last response. I’ll include it again at the bottom of this message, this time with more of the surrounding lines.

Clearly, bumping the version number of the OWL file did not do the job.

Making any changes, no matter how small, will normally trigger a new version to be pulled and loaded. You can see this in effect for other ontologies in our system, e.g., the Gene Ontology [4]. As I explained above, new versions of CHEMINF weren’t getting pulled due to a corrupt submission object in our backend triplestore.

It did pick it up right now, which is the exact course of events last time. It makes me wonder what you changed (and was changed the last time when a manual release went awry) that does allow the new version to be picked up overnight.

As per my previous message, I had to manually delete CHEMINF submission 45 from our backend triplestore, which was corrupt. After the corrupt submission is removed, the system is able to start loading new versions again without issue. This is a workaround until we’re able to sort out the bug I described above.

This is not the same course of events as our last correspondence. The WIKIPATHWAYS ontology was configured to retrieve new versions from an outdated URL. Once we updated the URL, BioPortal pulled a new version successfully. There was no manual removal of a submission object from the triplestore.




I, [2019-03-14T18:01:43.661721 #2852]  INFO -- : Checking download for CHEMINF
I, [2019-03-14T18:01:43.661827 #2852]  INFO -- : Location:
I, [2019-03-14T18:01:43.702111 #2852]  INFO -- : New file found for CHEMINF
old: b1c6f419a3b596d51bfd30ddd260a045
new: f29aaf2e5c888582118a95c616b07d73
I, [2019-03-14T18:01:43.802705 #2852]  INFO -- : Java call [java -DentityExpansionLimit=2500000 -Xmx10240M -jar /srv/ncbo/ncbo_cron/vendor/bundle/ruby/2.5.0/bundler/gems/ontologies_linked_data-ae81063f1e7a/bin/owlapi-wrapper-1.3.3.jar -m /srv/ncbo/repository/CHEMINF/45/cheminf.owl -o /srv/ncbo/repository/CHEMINF/45 -r false]
I, [2019-03-14T18:01:46.284347 #2852]  INFO -- : 2019-03-14T18:01:44 [main] INFO  o.s.n.o.OntologyParserCommand - Parsing invocation with values: ParserInvocation [inputRepositoryFolder=null, outputRepositoryFolder=/srv/ncbo/repository/CHEMINF/45, masterFileName=/srv/ncbo/repository/CHEMINF/45/cheminf.owl, invocationId=0, parserLog=, userReasoner= false]

2019-03-14T18:01:44 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - executor ...

2019-03-14T18:01:44 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - Input repository folder is null. Unique file being parsed.

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyMetrics - Calculating metrics for /srv/ncbo/repository/CHEMINF/45/cheminf.owl

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyMetrics - Finished metrics calculation for /srv/ncbo/repository/CHEMINF/45/cheminf.owl in 2 milliseconds

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyMetrics - Generated metrics CSV file for /srv/ncbo/repository/CHEMINF/45/cheminf.owl

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - Ontology document format: org.semanticweb.owlapi.formats.RDFXMLDocumentFormat

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - isPrefixOWLOntologyFormat: true

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - isPrefixOWLOntologyFormat: true

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - isPrefixOWLOntologyFormat: true

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - isPrefixOWLOntologyFormat: true

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - isPrefixOWLOntologyFormat: true

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - isPrefixOWLOntologyFormat: true

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - isOBO: false

2019-03-14T18:01:45 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - Serializing ontology in RDF ...

2019-03-14T18:01:46 [main] INFO  o.s.ncbo.oapiwrapper.OntologyParser - Serialization done!

2019-03-14T18:01:46 [main] INFO  o.s.n.o.OntologyParserCommand - Parse result: true

2019-03-14T18:01:46 [main] INFO  o.s.n.o.OntologyParserCommand - Output triples in: {}/srv/ncbo/repository/CHEMINF/45/owlapi.xrdf

2019-03-14T18:01:46 [main] INFO  o.s.n.o.OntologyParserCommand - Finished parsing!

I, [2019-03-14T18:01:46.284446 #2852]  INFO -- : OWLAPI Java command: parsing finished successfully.
I, [2019-03-14T18:01:46.284576 #2852]  INFO -- : Output size 1162063 in `/srv/ncbo/repository/CHEMINF/45/owlapi.xrdf`
E, [2019-03-14T18:01:46.443985 #2852] ERROR -- : Unable to create a new submission in OntologyPull: {:proc_naming=>{:duplicate=>"There is already a persistent resource with id ``<>"}}

Hi, do you like citation networks? Already 51% of all citations are available<> available for innovative new uses<>. Join me in asking the American Chemical Society to join the Initiative for Open Citations too<>. SpringerNature, the RSC and many others already did<>.

E.L. Willighagen
Department of Bioinformatics - BiGCaT
Maastricht University (
ORCID: 0000-0001-7542-0286<>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the bioontology-support mailing list