Posted on

#StackBounty: #functions #users #forms #permissions Add new user : make the fields First Name and Last name required

Bounty: 50

I’m searching how to make First Name and Last name fields required when we add a new user. Right now only username and Email fields are required.

I found a way by adding class="form-required" for the first and last name fields on the file user-new.php.

But I’m looking for a method with adding code on function.php and not touch to the WordPress Core.

Thanks.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

Posted on

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

  • I know how git works.
  • I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

  1. Edit code on the local copy.
  2. Commit the changes to the local copy.
  3. Push to the remote copy.
  4. SSH to the remote machine (or switch terminals).
  5. Test the code on the remote machine.
  6. Checkout another branch (to allow force-pushing).
  7. End the SSH sessions (or switch [back] terminals).
  8. Edit the code.
  9. Amend the previous commit on the local copy.
  10. Force-push to the remote copy.
  11. SSH to the remote machine (or switch terminals).
  12. Checkout the force-pushed branch.
  13. Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. End the SSH sessions (or switch [back] terminals).
  7. Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

  1. SSH to the remote machine (or switch terminals).
  2. Edit code on the remote copy from the remote machine.
  3. Test the code on the remote machine.
  4. Edit code on the remote copy from the remote machine.
  5. Repeat steps 3 – 4 (two steps!) until I’m happy.
  6. Commit the changes to the remote copy from the remote machine.
  7. Checkout another branch (to allow force-pushing).
  8. End the SSH sessions (or switch [back] terminals).
  9. Pull from the remote copy.
  10. Amend (sign) the previous commit on the local copy.
  11. Force-push to the remote copy.
  12. SSH to the remote machine (or switch terminals).
  13. Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

  1. Mount remote with debug output:
 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0
  1. In a different terminal, access the mounted share:
cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)
  1. Verify regular writing works:
echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)
  1. Trigger error by attempting to append:
echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!