users – TechUtils.in

September 5, 2021

#StackBounty: #8 #users #migration Map profile_id based on previously mapped uid when migrating user profiles from CSV file

Bounty: 50

I’m trying to migrate users and user profiles to site, where both entities might be already present. In this case I would like to overwrite specified fields and leave others in the original state. I’m trying to map profile_id like this:

  uid:
    plugin: migration_lookup
    no_stub: true
    # previous user migration
    migration: mayors_users
    # property in the source data
    source: lau
  profile_id:
    plugin: entity_lookup
    entity_type: profile
    bundle: profile
    bundle_key: type
    value_key: uid
    source: '@uid'

However, this fails, so profile_id is null.

Do you happen to know, how can I use entity_lookup (or another) plugin for this?

EDIT: It looks like this might be a Drupal Core issue. I debugged the code of EntityLookup.php plugin and printed the actual DB query generated from the entity query:

SELECT base_table.revision_id AS revision_id, base_table.profile_id AS profile_id
FROM profile base_table
         INNER JOIN profile profile ON profile.profile_id = base_table.profile_id
WHERE (profile.uid = '153')
  AND (profile.type = 'profile')

This works fine and return following result when used directly against MySQL server:

revision_id	profile_id
87	87

However, $query->execute() still returns an empty array. Why?

EDIT: It looks like this might be a bug: https://www.drupal.org/project/migrate_plus/issues/3230477

Get this bounty!!!

August 30, 2021

#StackBounty: #users #migration Map profile_id based on previously mapped uid when migrating user profiles from CSV file

Bounty: 50

  uid:
    plugin: migration_lookup
    no_stub: true
    # previous user migration
    migration: mayors_users
    # property in the source data
    source: lau
  profile_id:
    plugin: entity_lookup
    entity_type: profile
    bundle: profile
    bundle_key: type
    value_key: uid
    source: '@uid'

However, this fails, so profile_id is null.

Do you happen to know, how can I use entity_lookup (or another) plugin for this?

EDIT: It looks like this might be a Drupal Core issue. I debugged the code of EntityLookup.php plugin and printed the actual DB query generated from the entity query:

SELECT base_table.revision_id AS revision_id, base_table.profile_id AS profile_id
FROM profile base_table
         INNER JOIN profile profile ON profile.profile_id = base_table.profile_id
WHERE (profile.uid = '153')
  AND (profile.type = 'profile')

This works fine and return following result when used directly against MySQL server:

revision_id	profile_id
87	87

However, $query->execute() still returns an empty array. Why?

EDIT: It looks like this might be a bug: https://www.drupal.org/project/migrate_plus/issues/3230477

Get this bounty!!!

August 26, 2021

#StackBounty: #8 #users #flags How can I limit flagging per user?

Bounty: 50

Node compare is awesome plugin but it’s just only in D7.

I’m using Drupal 8.

I have a views page which integrated with flag(compare it) and flipped table module to compare contents. It has limit to show max. 4 content. Everything works until this point.

But how can limit user to flagging max. 4 times. Flag limit is nice in D7 but it’s not ported in D8.And I couldn’t achieved it via rules integration.
Any help with this would be awesome . Thanks

Get this bounty!!!

February 17, 2021

#StackBounty: #users #8 How to implement decoupled user registration with React / jsonapi?

Bounty: 200

I have built a React app that uses JSON:API to pull in data, but now I want to implement user registration.

JSON:API doesn’t do user registration because it isn’t a CRUD operation.

However, the links above explain that core provides a REST endpoint for handling user registration.

The documentation gives this example:

curl 
  --header "Content-type: application/json" 
  --request POST http://drupal.d8/user/register

But this example seems incomplete– for example, where is the username?

My website is configured with the Email Registration module and only an email address is required to register, but where do I submit the email address?

Here’s the code I tried in React:

UserRegistrationForm.tsx

import * as React from 'react';
import { useForm } from 'react-hook-form';
import { IonButton } from '@ionic/react';
import { baseUrl, basicHeaders } from '../utils/globals';
import handleErrors from '../utils/fetchHandleErrors';

interface InterfaceRegistrationFormInput {
  email: string;
}

const UserRegistrationForm: React.VFC = () => {
  const { register, handleSubmit, errors } = useForm<InterfaceRegistrationFormInput>();

  const onSubmit = (formData: InterfaceRegistrationFormInput) => {
    fetch(`${baseUrl}/user/register`, {
      method: 'POST',
      headers: basicHeaders,
      body: formData,
    })
      .then(handleErrors)
      .catch((error) => Promise.reject(new Error(error)));
  };

  return (
    <>
      <form onSubmit={handleSubmit(onSubmit)}>
        <label htmlFor="email">
          Email address
          <br />
          <input
            name="email"
            type="email"
            ref={register({ required: true })}
          />
        </label>
        <IonButton type="submit">Register</IonButton>
      </form>
    </>
  );
};

Get this bounty!!!

August 12, 2020

#StackBounty: #functions #users #forms #permissions Add new user : make the fields First Name and Last name required

Bounty: 50

I’m searching how to make First Name and Last name fields required when we add a new user. Right now only username and Email fields are required.

I found a way by adding class="form-required" for the first and last name fields on the file user-new.php.

But I’m looking for a method with adding code on function.php and not touch to the WordPress Core.

Thanks.

Get this bounty!!!

June 28, 2020

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

Given the above symptoms, I assume the issue lies with appending to a file.
I found Git repository on SSHFS: unable to append to '.git/logs/HEAD': Invalid argument which refers to https://github.com/libfuse/sshfs/issues/82 suggesting the issue (note the slighty differerent error message) could be solved by mounting the remote file system with writeback_cache=no. The latter source quotes the man page referencing the following caveat/workaround:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

I found that the writeback-cache feature that I tried to disable, was actually removed (after being disabled and re-enabled more than once before).
So I guess I should be good but clearly there (still) is a problem.

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

I know how git works.
I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Why I don’t do it? – Because I track code that can only be tested on the remote machine and I do want to test it before committing.
So to some extent this is ‘just’ a convenience issue to avoid having to:

Edit code on the local copy.
Commit the changes to the local copy.
Push to the remote copy.
SSH to the remote machine (or switch terminals).
Test the code on the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Edit the code.
Amend the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.
Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
End the SSH sessions (or switch [back] terminals).
Commit the changes to the remote copy from the local machine.

Why don’t I simply commit from the remote machine? – Because I want to sign my commits but can’t entrust the remote machine with the private key.
So the best alternative I could come up with is:

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
Commit the changes to the remote copy from the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Pull from the remote copy.
Amend (sign) the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.

So on the one hand, I’d like to get rid of these extra steps (things get more complicated when adding feature branches as those need to be properly checked out on both copies and configured for proper tracking), on the other I want to understand why it doesn’t ‘just work'(tm).

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

Mount remote with debug output:

 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0

In a different terminal, access the mounted share:

cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)

Verify regular writing works:

echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)

Trigger error by attempting to append:

echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

June 28, 2020

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

I know how git works.
I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Edit code on the local copy.
Commit the changes to the local copy.
Push to the remote copy.
SSH to the remote machine (or switch terminals).
Test the code on the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Edit the code.
Amend the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.
Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
End the SSH sessions (or switch [back] terminals).
Commit the changes to the remote copy from the local machine.

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
Commit the changes to the remote copy from the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Pull from the remote copy.
Amend (sign) the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

Mount remote with debug output:

 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0

In a different terminal, access the mounted share:

cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)

Verify regular writing works:

echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)

Trigger error by attempting to append:

echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

June 28, 2020

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

I know how git works.
I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Edit code on the local copy.
Commit the changes to the local copy.
Push to the remote copy.
SSH to the remote machine (or switch terminals).
Test the code on the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Edit the code.
Amend the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.
Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
End the SSH sessions (or switch [back] terminals).
Commit the changes to the remote copy from the local machine.

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
Commit the changes to the remote copy from the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Pull from the remote copy.
Amend (sign) the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

Mount remote with debug output:

 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0

In a different terminal, access the mounted share:

cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)

Verify regular writing works:

echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)

Trigger error by attempting to append:

echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

June 28, 2020

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

I know how git works.
I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Edit code on the local copy.
Commit the changes to the local copy.
Push to the remote copy.
SSH to the remote machine (or switch terminals).
Test the code on the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Edit the code.
Amend the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.
Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
End the SSH sessions (or switch [back] terminals).
Commit the changes to the remote copy from the local machine.

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
Commit the changes to the remote copy from the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Pull from the remote copy.
Amend (sign) the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

Mount remote with debug output:

 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0

In a different terminal, access the mounted share:

cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)

Verify regular writing works:

echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)

Trigger error by attempting to append:

echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!

June 28, 2020

#StackBounty: #linux #permissions #users #sshfs git over sshfs (with idmap): unable to append to '.git/logs/HEAD': Permission d…

Bounty: 100

Problem:

I have a git repository mounted via sshfs and cannot commit changes with the following error message:

fatal: cannot update the ref 'HEAD': unable to append to '.git/logs/HEAD': Permission denied

Note that I can

cp -a .git/logs/HEAD .git/logs/HEAD.bu
printf foo > .git/logs/HEAD
mv .git/logs/HEAD.bu .git/logs/HEAD

without a problem, but

printf foo >> .git/logs/HEAD

gives me the ‘Permission denied’ as well.

Question:

What do I need to change about my configuration to be able to commit from my local machine to the remote repository?

What I tried:

CAVEATS / WORKAROUNDS
[...]
   O_APPEND
       When  writeback  caching is enabled, SSHFS cannot reliably support the O_APPEND open
       flag and thus signals an error on open.  To enable support for  unreliable  O_APPEND
       (which  may  overwrite  data if the file changes on the server at a bad time), mount
       the file system with -o unreliable_append.

However, this section is not in my man page:

sshfs -V

SSHFS version 3.7.0
FUSE library version 3.9.1
using FUSE kernel interface version 7.31
fusermount3 version: 3.9.1

A further complication that I should probably mention, is that my user name and ID on the remote system do not match the local one, so I need to used the idmap feature.

Here is the corresponding fstab entry:

<remote-user>@<remote-machine>: /mnt/ssh/<remote-machine>  sshfs  _netdev,user,idmap=user,allow_other  0 0

Also, my /etc/fuse.conf contains

user_allow_other

Background:

To avoid answer just telling me not to do this:

I know how git works.
I know I can clone the repository locally, commit there, and push to the remote one over ssh.

Edit code on the local copy.
Commit the changes to the local copy.
Push to the remote copy.
SSH to the remote machine (or switch terminals).
Test the code on the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Edit the code.
Amend the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.
Repeat steps 5 – 11 (seven steps!) until I’m happy.

Instead, I want to:

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
End the SSH sessions (or switch [back] terminals).
Commit the changes to the remote copy from the local machine.

SSH to the remote machine (or switch terminals).
Edit code on the remote copy from the remote machine.
Test the code on the remote machine.
Edit code on the remote copy from the remote machine.
Repeat steps 3 – 4 (two steps!) until I’m happy.
Commit the changes to the remote copy from the remote machine.
Checkout another branch (to allow force-pushing).
End the SSH sessions (or switch [back] terminals).
Pull from the remote copy.
Amend (sign) the previous commit on the local copy.
Force-push to the remote copy.
SSH to the remote machine (or switch terminals).
Checkout the force-pushed branch.

Update:

Following up on a comment by @tukan, I reproduced the error with debug output:

Mount remote with debug output:

 mount -o sshfs_debug MOUNTPOINT

SSHFS version 3.7.0
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-2> <USER@SERVER> <-s> <sftp>
USER@SERVER's password:
Server version: 3
Extension: versions <2,3,4,5,6>
Extension: fsync@openssh.com <1>
Extension: posix-rename@openssh.com <1>
Extension: statvfs@openssh.com <2>
Extension: fstatvfs@openssh.com <2>
Extension: hardlink@openssh.com <1>
remote_uid = 0

In a different terminal, access the mounted share:

cd MOUNTPOINT/DIR_WITH_WRITE_PERMISSIONS

[00002] LSTAT
  [00002]          ATTRS       45bytes (188ms)

Verify regular writing works:

echo foo > foobar

[00003] LSTAT
  [00003]         STATUS       38bytes (46ms)
[00004] LSTAT
  [00004]         STATUS       38bytes (32ms)
[00005] LSTAT
  [00005]          ATTRS       45bytes (242ms)
[00006] OPENDIR
  [00006]         HANDLE       29bytes (31ms)
[00007] READDIR
[00008] READDIR
  [00007]           NAME      668bytes (58ms)
[00009] READDIR
[00010] READDIR
  [00008]           NAME      483bytes (65ms)
[00011] READDIR
[00012] READDIR
  [00009]         STATUS       37bytes (27ms)
  [00010]         STATUS       37bytes (27ms)
[00013] CLOSE
[00014] LSTAT
  [00011]         STATUS       37bytes (27ms)
  [00012]         STATUS       37bytes (27ms)
  [00013]         STATUS       28bytes (26ms)
  [00014]         STATUS       38bytes (31ms)
[00015] OPEN
[00016] LSTAT
  [00015]         HANDLE       29bytes (153ms)
  [00016]          ATTRS       45bytes (158ms)
[00017] FSTAT
  [00017]          ATTRS       45bytes (29ms)
[00018] WRITE
  [00018]         STATUS       28bytes (28ms)
[00019] CLOSE
  [00019]         STATUS       28bytes (28ms)

Trigger error by attempting to append:

echo bar >> foobar

[00020] LSTAT
  [00020]         STATUS       38bytes (74ms)
[00021] LSTAT
  [00021]         STATUS       38bytes (57ms)
[00022] LSTAT
  [00022]          ATTRS       45bytes (52ms)
[00023] OPENDIR
  [00023]         HANDLE       29bytes (53ms)
[00024] READDIR
[00025] READDIR
  [00024]           NAME      668bytes (68ms)
[00026] READDIR
[00027] READDIR
  [00025]           NAME      597bytes (77ms)
[00028] READDIR
[00029] READDIR
  [00026]         STATUS       37bytes (47ms)
[00030] CLOSE
  [00027]         STATUS       37bytes (47ms)
[00031] OPEN
[00032] LSTAT
  [00028]         STATUS       37bytes (47ms)
  [00029]         STATUS       37bytes (47ms)
  [00030]         STATUS       28bytes (26ms)
  [00031]         STATUS       43bytes (28ms)
  [00032]          ATTRS       45bytes (29ms)

zsh: permission denied: foobar

Hope this helps to find the root cause of my problem.

Get this bounty!!!