Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[farmshare-discuss] Git Clone fatal error (sha1 file, timeout)

Nick Henderson nwh at stanford.edu
Fri Sep 16 13:05:11 PDT 2016


Alex, thanks for the reminder about `/farmshare/user_data`.  I was able to
`git clone` a repository there.

A few questions:

1) What are the storage limits on `/farmshare`?  Many of our students are
new to remote computing, programming, and command line tools.  Sometimes,
they inadvertently generate large files, or numerous copies of not so large
files.  I suspect all of this should be less than 500MB.  We can also teach
the students to use `$ du` and clean up files.

2) File permissions on `/farmshare` are not handled via ACLs, right?

Thanks,
Nick

On Fri, Sep 16, 2016 at 10:12 AM Alex Chekholko <chekh at stanford.edu> wrote:

> Hi all,
>
> I recommend you work around AFS problems by not using AFS.  Use your
> user_data directory instead:
>
>
> https://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php/User_Guide#Directory_paths
>
> Regards,
> Alex
>
> On 09/16/2016 09:49 AM, Nick Henderson wrote:
> > I also want to confirm |git| issues on |corn.stanford.edu
> > <http://corn.stanford.edu>|.
> >
> > I was able to clone a small repo directly in my afs home dir:
> >
> > https://github.com/nwh/cme211-test-repo
> >
> > However, the clone times out on my course notes repo:
> >
> > https://github.com/CME211/notes
> >
> > |$ git clone https://github.com/CME211/notes Cloning into 'notes'...
> > remote: Counting objects: 1244, done. remote: Compressing objects: 100%
> > (198/198), done. remote: Total 1244 (delta 97), reused 0 (delta 0),
> > pack-reused 1042 Receiving objects: 100% (1244/1244), 17.75 MiB | 3.61
> > MiB/s, done. Resolving deltas: 100% (506/506), done. Checking
> > connectivity... done. error: unable to create file
> > challenge/cpp_assignment/u.data (Connection timed out) error: unable to
> > create file challenge/cpp_assignment/u.data.short (Connection timed out)
> > fatal: cannot create directory at 'challenge/even_fib': Connection timed
> > out warning: unable to unlink .git/index.lock: Connection timed out
> > warning: Clone succeeded, but checkout failed. You can inspect what was
> > checked out with 'git status' and retry the checkout with 'git checkout
> > -f HEAD' |
> >
> > Very similar behavior when cloning via SSH.
> >
> > This is a new issue. I have been cloning directly into my AFS for quite
> > a long time with no issue.
> >
> > We would really appreciate attention to this issue. I am the instructor
> > for CME211. We teach the students to use |git| and keep track of their
> > work on GitHub. Asking all students to clone to |/tmp| is not a good
> > solution for us. In previous years, all of our students cloned directly
> > to their home folders on AFS with no issue.
> >
> > Thanks so much,
> > Nick
> >
> > On Thu, Sep 15, 2016 at 8:37 PM John Bickar jbickar at stanford.edu
> > <http://mailto:jbickar@stanford.edu> wrote:
> >
> >     Hrm. It's not necessarily AFS, per se, but something with the config
> on
> >     corn and cardinal, plus something in the git repo or git itself that
> >     causes the clone to fail. I (and the rest of my team) clone from
> github
> >     (public and private repos, using primarily the SSH protocol) to AFS
> >     hundreds of times daily, so it's not simply that git and AFS don't
> work
> >     together.
> >
> >     I tested two different public repos cloning to AFS on corn, cardinal,
> >     sites1, and the "my-stanford-vbox" VM*.
> >
> >     https://github.com/SU-SWS/Stanford-Drupal-Profile.git:
> >     corn: success
> >     cardinal: success
> >     sites1: success
> >     my-stanford-vbox: success (but it failed if I tried to clone snorkel
> >     first)
> >
> >     https://github.com/HazyResearch/snorkel.git:
> >     corn: fail
> >     cardinal: fail
> >     sites1: success
> >     my-stanford-vbox: fail
> >
> >     git versions:
> >     cardinal: 1.9.1
> >     corn: 1.9.1
> >     sites1: 1.7.10.4
> >     my-stanford-vbox: 1.9.1
> >
> >     My best guess would be a mismatch in versions of OpenAFS or git among
> >     the different servers, but I couldn't figure out what version of
> OpenAFS
> >     was running on each of the servers I tried (using the instructions at
> >     https://uit.stanford.edu/service/afs/install).
> >
> >     It may also be a difference in the size of the repo: snorkel is 235M,
> >     Stanford-Drupal-Profile is 534K.
> >
> >     Maybe the above will help our friends in Research Computing
> troubleshoot
> >     this issue.
> >
> >     -John Bickar
> >     Stanford Web Services
> >
> >
> >     * https://code.stanford.edu/devops-tools/my-stanford-vbox
> >
> >
> >
> >
> >     On 9/15/16 6:06 PM, Charles E Yates wrote:
> >     > Clone to /tmp then copy to where you want it.  Seems git clone and
> afs
> >     > don't play well together.
> >     >
> >
>  ------------------------------------------------------------------------
> >     > *From:* farmshare-discuss
> >     <farmshare-discuss-bounces at lists.stanford.edu
> >     <mailto:farmshare-discuss-bounces at lists.stanford.edu>>
> >     > on behalf of Andreas Santucci <santucci.andreas at gmail.com
> >     <mailto:santucci.andreas at gmail.com>>
> >     > *Sent:* Thursday, September 15, 2016 1:53:44 PM
> >     > *To:* farmshare-discuss at lists.stanford.edu
> >     <mailto:farmshare-discuss at lists.stanford.edu>
> >     > *Subject:* [farmshare-discuss] Git Clone fatal error (sha1 file,
> >     timeout)
> >     >
> >     > Dear Farmshare Mailing List,
> >     >
> >     > I am quite consistently encountering a connection time-out when
> trying
> >     > to clone git repositories on Corn, regardless whether I attempt to
> >     clone
> >     > via SSH or HTTPS.
> >     >
> >     > _*Using HTTPS:*_
> >     > santucci at corn12:~$ git clone
> >     https://github.com/CME211/cme211-asantucci.git
> >     > Cloning into 'cme211-asantucci'...
> >     > Username for 'https://github.com': asantucci
> >     > Password for 'https://asantucci@github.com':
> >     > remote: Counting objects: 52, done.
> >     > remote: Compressing objects: 100% (24/24), done.
> >     > _*fatal: error when closing sha1 file: Connection timed out
> >     >
> >     > *_
> >     > _*Using * *SSH**:**
> >     > *_
> >     > santucci at corn12:~$ git clone
> >     git at github.com:CME211/cme211-asantucci.git
> >     > Cloning into 'cme211-asantucci'...
> >     > remote: Counting objects: 52, done.
> >     > remote: Compressing objects: 100% (24/24), done.
> >     > remote: Total 52 (delta 9), reused 0 (delta 0), pack-reused 24
> >     > Receiving objects: 100% (52/52), 10.18 MiB | 1.31 MiB/s, done.
> >     > Resolving deltas: 100% (15/15), done.
> >     > Checking connectivity... done.
> >     > error: unable to create file hw1/alignments3.txt (Connection timed
> >     out)
> >     > ... (More connection time-outs here) ...
> >     > _*fatal: unable to checkout working tree
> >     > warning: unable to unlink .git/index.lock: Connection timed out
> >     > warning: Clone succeeded, but checkout failed.
> >     > *_You can inspect what was checked out with 'git status'
> >     > and retry the checkout with 'git checkout -f HEAD'
> >     >
> >     > Attempts to follow the instructions above (retrying the checkout)
> >     leads
> >     > only to more time-outs.
> >     >
> >     > _*Last Ditch Effort: Adding SSH Keys to GitHub Account:
> >     > *_
> >     > I've also tried to follow some of the instructions listed here on
> >     GitHub
> >     > <https://help.github.com/categories/ssh/> pertaining to adding SSH
> >     Keys
> >     > or GPG Keys manually to a GitHub account.
> >     >
> >     > _*Question:*_
> >     > What is the correct way to remedy this error? Executing the same
> >     > commands from a local machine causes no problems. I apologize that
> the
> >     > repository listed above is private. For a public repo which also
> >     causes
> >     > the same error, try:
> >     >
> >     > git clone https://github.com/HazyResearch/snorkel.git
> >     >
> >     > Thank you for your time and attention, please let me know if I can
> add
> >     > any clarifying details.
> >     >
> >     > Andreas Santucci
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     >
> >     > _______________________________________________
> >     > farmshare-discuss mailing list
> >     > farmshare-discuss at lists.stanford.edu
> >     <mailto:farmshare-discuss at lists.stanford.edu>
> >     > https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
> >     >
> >
> >     _______________________________________________
> >     farmshare-discuss mailing list
> >     farmshare-discuss at lists.stanford.edu
> >     <mailto:farmshare-discuss at lists.stanford.edu>
> >     https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
> >
> > ​
> >
> >
> > _______________________________________________
> > farmshare-discuss mailing list
> > farmshare-discuss at lists.stanford.edu
> > https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
> >
>
> --
> Alex Chekholko chekh at stanford.edu 347-401-4860
>
> _______________________________________________
> farmshare-discuss mailing list
> farmshare-discuss at lists.stanford.edu
> https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/farmshare-discuss/attachments/20160916/20c2d6b1/attachment-0001.html>


More information about the farmshare-discuss mailing list