Example problem 1:🔗

Could not authenticate you from Crowd because "Invalid credentials".

Causes & solutions:🔗

1. Is the user providing the correct PIN & token code (RSA or RZ token, with no re-use)?
2. Is the user providing the correct username? If OUN and LC username aren’t the same, make sure the LC username is provided.
3. Is it possible the user supplied incorrect credentials on a previous login attempt (such as providing an email address instead of a username)? Have the user try to log in from a new browser or incognito window to get a fresh start.
4. Call the LC Hotline to check if they see evidence of failed authentication attempts or Kerberos lockouts.

Note: Historically there was an issue where GitLab would autogenerate and expect OUNs rather than LC usernames for those users whose OUNS & LC usernames differ; this would prevent LC usernames from working upon login. This has been fixed and should no longer be an issue.

Example problem 2:🔗

User cannot successfully git clone <repo>.

Causes & solutions:🔗

The most common reasons a user cannot successfully clone include (A) not having ssh keys set up correctly, (B) not having a config file set up correctly (if cloning from a local machine), or (C) not having repo access.

Check GitLab setup

First, check that the user can connect to gitlab via ssh -p7999 git@czgitlab or ssh -p7999 git@rzgitlab, depending on whether the repo is on the CZ or RZ.

If the user cannot connect to GitLab, the problem is likely (A) or (B) and may present with error messages like

Cloning into <repo name>... 
Permission denied (publickey). 
fatal: Could not read from remote repository. 

Confirm that the user 1. has a ssh private key on the machine from which ssh -p7999 git@czgitlab is issued, 2. has copied the corresponding public key to the user profile at lc.llnl.gov/gitlab or rzlc.llnl.gov/gitlab, 3. has added the corresponding public key to the $HOME/.ssh/authorized_keys file on the corresponding *slic machine, 4. has correctly set up a $HOME/.ssh/config file if connecting to GitLab from a laptop or desktop (but not from an LC machine).

If the user has set up ssh keys correctly, try to gain more info with the -vvvv flag, for example, ssh -vvvv -p7999 git@czgitlab.

Second, after confirming the user can connect go GitLab, confirm the user can clone a dummy public repo. Try via

git clone ssh://git@czgitlab.llnl.gov:7999/janeh/jane_gitlab_demo.git # if testing CZ

git clone ssh://git@rzgitlab.llnl.gov:7999/janeh/jane_gitlab_demo.git # if testing RZ

Check repo permissions

If the user can connect to GitLab, confirm that the user has access rights to the relevant repository. Check that the user can see the desired project from the browser. If the project is private, is the user listed as a member in “Project Information” -> “Members” under the relevant project on GitLab?

Check repo/remote address

Check that the user is trying to clone using the correct ssh:git@….. path (including the port!) to the repo listed under “Clone with SSH”:

For a repository that has already been cloned, the user can check the full path to the repository by running git remote -v at the command line from within the local copy of the repository.

Example problem 3:🔗

You see a RPC error message similar to the following:

error: RPC failed; HTTP 401 curl 22 The requested URL returned error: 401
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly

Causes & solutions:🔗

Git-over-https has a number of issues, including having to repeatedly enter passwords, and timeouts that occur when performing operations on some larger repos. We recommend setting up SSH keys and switching to git-over-ssh, as described above.

If you are already using git-over-ssh and continue to see RPC errors with git push, the commit size may simply be too large. Please try breaking your most recent commit into sets of smaller changes.