Search Mailing List Archives


Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort
Limit to: All This Week Last Week This Month Last Month
Select Date Range     through    

[farmshare-discuss] Git Clone fatal error (sha1 file, timeout)

Mr. Addis Walter O'Connor addiso at stanford.edu
Fri Sep 16 13:32:38 PDT 2016


Hi Nick,

I would be more than happy to sit down with you in person, and continue this conversation sometime next week? I am available Wednesday morning, or afternoon and can meet you just about any where on campus.


  1.  There are currently no ‘hard’ limits on the /farmshare/user_data storage, but it is intended as a ‘scratch’ space, is not backed up, and should not be more than 1Tb except for a very temporary basis. There should be no issue with the 500MB files you mention.
  2.  They can be handled via acls for shared or group access, but as it set by default, we just use standard NFSv3 permissions. For special shared or group access, please send a request and summary of the project to ‘research-computing-support at stanford.edu’

Please let me know what time is good for you on Wednesday.

Best,
Addis
From: farmshare-discuss <farmshare-discuss-bounces at lists.stanford.edu<mailto:farmshare-discuss-bounces at lists.stanford.edu>> on behalf of Nick Henderson <nwh at stanford.edu<mailto:nwh at stanford.edu>>
Reply-To: Open discussion for users of FarmShare <farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>>
Date: Friday, September 16, 2016 at 1:05 PM
To: Open discussion for users of FarmShare <farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>>
Subject: Re: [farmshare-discuss] Git Clone fatal error (sha1 file, timeout)

Alex, thanks for the reminder about `/farmshare/user_data`.  I was able to `git clone` a repository there.

A few questions:

1) What are the storage limits on `/farmshare`?  Many of our students are new to remote computing, programming, and command line tools.  Sometimes, they inadvertently generate large files, or numerous copies of not so large files.  I suspect all of this should be less than 500MB.  We can also teach the students to use `$ du` and clean up files.

2) File permissions on `/farmshare` are not handled via ACLs, right?

Thanks,
Nick

On Fri, Sep 16, 2016 at 10:12 AM Alex Chekholko <chekh at stanford.edu<mailto:chekh at stanford.edu>> wrote:
Hi all,

I recommend you work around AFS problems by not using AFS.  Use your
user_data directory instead:

https://web.stanford.edu/group/farmshare/cgi-bin/wiki/index.php/User_Guide#Directory_paths

Regards,
Alex

On 09/16/2016 09:49 AM, Nick Henderson wrote:
> I also want to confirm |git| issues on |corn.stanford.edu<http://corn.stanford.edu>
> <http://corn.stanford.edu>|.
>
> I was able to clone a small repo directly in my afs home dir:
>
> https://github.com/nwh/cme211-test-repo
>
> However, the clone times out on my course notes repo:
>
> https://github.com/CME211/notes
>
> |$ git clone https://github.com/CME211/notes Cloning into 'notes'...
> remote: Counting objects: 1244, done. remote: Compressing objects: 100%
> (198/198), done. remote: Total 1244 (delta 97), reused 0 (delta 0),
> pack-reused 1042 Receiving objects: 100% (1244/1244), 17.75 MiB | 3.61
> MiB/s, done. Resolving deltas: 100% (506/506), done. Checking
> connectivity... done. error: unable to create file
> challenge/cpp_assignment/u.data (Connection timed out) error: unable to
> create file challenge/cpp_assignment/u.data.short (Connection timed out)
> fatal: cannot create directory at 'challenge/even_fib': Connection timed
> out warning: unable to unlink .git/index.lock: Connection timed out
> warning: Clone succeeded, but checkout failed. You can inspect what was
> checked out with 'git status' and retry the checkout with 'git checkout
> -f HEAD' |
>
> Very similar behavior when cloning via SSH.
>
> This is a new issue. I have been cloning directly into my AFS for quite
> a long time with no issue.
>
> We would really appreciate attention to this issue. I am the instructor
> for CME211. We teach the students to use |git| and keep track of their
> work on GitHub. Asking all students to clone to |/tmp| is not a good
> solution for us. In previous years, all of our students cloned directly
> to their home folders on AFS with no issue.
>
> Thanks so much,
> Nick
>
> On Thu, Sep 15, 2016 at 8:37 PM John Bickar jbickar at stanford.edu<mailto:jbickar at stanford.edu>
> <http://mailto:jbickar@stanford.edu> wrote:
>
>     Hrm. It's not necessarily AFS, per se, but something with the config on
>     corn and cardinal, plus something in the git repo or git itself that
>     causes the clone to fail. I (and the rest of my team) clone from github
>     (public and private repos, using primarily the SSH protocol) to AFS
>     hundreds of times daily, so it's not simply that git and AFS don't work
>     together.
>
>     I tested two different public repos cloning to AFS on corn, cardinal,
>     sites1, and the "my-stanford-vbox" VM*.
>
>     https://github.com/SU-SWS/Stanford-Drupal-Profile.git:
>     corn: success
>     cardinal: success
>     sites1: success
>     my-stanford-vbox: success (but it failed if I tried to clone snorkel
>     first)
>
>     https://github.com/HazyResearch/snorkel.git:
>     corn: fail
>     cardinal: fail
>     sites1: success
>     my-stanford-vbox: fail
>
>     git versions:
>     cardinal: 1.9.1
>     corn: 1.9.1
>     sites1: 1.7.10.4
>     my-stanford-vbox: 1.9.1
>
>     My best guess would be a mismatch in versions of OpenAFS or git among
>     the different servers, but I couldn't figure out what version of OpenAFS
>     was running on each of the servers I tried (using the instructions at
>     https://uit.stanford.edu/service/afs/install).
>
>     It may also be a difference in the size of the repo: snorkel is 235M,
>     Stanford-Drupal-Profile is 534K.
>
>     Maybe the above will help our friends in Research Computing troubleshoot
>     this issue.
>
>     -John Bickar
>     Stanford Web Services
>
>
>     * https://code.stanford.edu/devops-tools/my-stanford-vbox
>
>
>
>
>     On 9/15/16 6:06 PM, Charles E Yates wrote:
>     > Clone to /tmp then copy to where you want it.  Seems git clone and afs
>     > don't play well together.
>     >
>     ------------------------------------------------------------------------
>     > *From:* farmshare-discuss
>     <farmshare-discuss-bounces at lists.stanford.edu<mailto:farmshare-discuss-bounces at lists.stanford.edu>
>     <mailto:farmshare-discuss-bounces at lists.stanford.edu<mailto:farmshare-discuss-bounces at lists.stanford.edu>>>
>     > on behalf of Andreas Santucci <santucci.andreas at gmail.com<mailto:santucci.andreas at gmail.com>
>     <mailto:santucci.andreas at gmail.com<mailto:santucci.andreas at gmail.com>>>
>     > *Sent:* Thursday, September 15, 2016 1:53:44 PM
>     > *To:* farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>
>     <mailto:farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>>
>     > *Subject:* [farmshare-discuss] Git Clone fatal error (sha1 file,
>     timeout)
>     >
>     > Dear Farmshare Mailing List,
>     >
>     > I am quite consistently encountering a connection time-out when trying
>     > to clone git repositories on Corn, regardless whether I attempt to
>     clone
>     > via SSH or HTTPS.
>     >
>     > _*Using HTTPS:*_
>     > santucci at corn12:~$ git clone
>     https://github.com/CME211/cme211-asantucci.git
>     > Cloning into 'cme211-asantucci'...
>     > Username for 'https://github.com': asantucci
>     > Password for 'https://asantucci@github.com':
>     > remote: Counting objects: 52, done.
>     > remote: Compressing objects: 100% (24/24), done.
>     > _*fatal: error when closing sha1 file: Connection timed out
>     >
>     > *_
>     > _*Using * *SSH**:**
>     > *_
>     > santucci at corn12:~$ git clone
>     git at github.com<mailto:git at github.com>:CME211/cme211-asantucci.git
>     > Cloning into 'cme211-asantucci'...
>     > remote: Counting objects: 52, done.
>     > remote: Compressing objects: 100% (24/24), done.
>     > remote: Total 52 (delta 9), reused 0 (delta 0), pack-reused 24
>     > Receiving objects: 100% (52/52), 10.18 MiB | 1.31 MiB/s, done.
>     > Resolving deltas: 100% (15/15), done.
>     > Checking connectivity... done.
>     > error: unable to create file hw1/alignments3.txt (Connection timed
>     out)
>     > ... (More connection time-outs here) ...
>     > _*fatal: unable to checkout working tree
>     > warning: unable to unlink .git/index.lock: Connection timed out
>     > warning: Clone succeeded, but checkout failed.
>     > *_You can inspect what was checked out with 'git status'
>     > and retry the checkout with 'git checkout -f HEAD'
>     >
>     > Attempts to follow the instructions above (retrying the checkout)
>     leads
>     > only to more time-outs.
>     >
>     > _*Last Ditch Effort: Adding SSH Keys to GitHub Account:
>     > *_
>     > I've also tried to follow some of the instructions listed here on
>     GitHub
>     > <https://help.github.com/categories/ssh/> pertaining to adding SSH
>     Keys
>     > or GPG Keys manually to a GitHub account.
>     >
>     > _*Question:*_
>     > What is the correct way to remedy this error? Executing the same
>     > commands from a local machine causes no problems. I apologize that the
>     > repository listed above is private. For a public repo which also
>     causes
>     > the same error, try:
>     >
>     > git clone https://github.com/HazyResearch/snorkel.git
>     >
>     > Thank you for your time and attention, please let me know if I can add
>     > any clarifying details.
>     >
>     > Andreas Santucci
>     >
>     >
>     >
>     >
>     >
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > farmshare-discuss mailing list
>     > farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>
>     <mailto:farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>>
>     > https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
>     >
>
>     _______________________________________________
>     farmshare-discuss mailing list
>     farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>
>     <mailto:farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>>
>     https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
>
>>
>
> _______________________________________________
> farmshare-discuss mailing list
> farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>
> https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
>

--
Alex Chekholko chekh at stanford.edu<mailto:chekh at stanford.edu> 347-401-4860

_______________________________________________
farmshare-discuss mailing list
farmshare-discuss at lists.stanford.edu<mailto:farmshare-discuss at lists.stanford.edu>
https://mailman.stanford.edu/mailman/listinfo/farmshare-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.stanford.edu/pipermail/farmshare-discuss/attachments/20160916/1af69d72/attachment-0001.html>


More information about the farmshare-discuss mailing list