#StackBounty: #csom #access Fine grained permissions applied to SharePoint Online App-Auth?

Bounty: 50

We have written an application that access SharePoint CSOM apis to do the following:

  1. Get sharepoint users and groups in a site collection.
  2. Get sites in a site colleciton
  3. Get lists in a site collection
  4. Get role assignments/definitions
  5. Get list items in a site collection.
  6. Get changes since a given time in a site collection.

When we access SharePoint online using app-auth (oauth or azure private key), we are forced to give Full admin access to the app in order to do these things.

But when we user a normal service account (username/password) we have access to the fine-grained permissions as you would expect. We do this by creating a custom SharePoint permission level and give it

  • View Items – View items in lists and documents in document libraries.
  • Open Items – View the source of documents with server-side file handlers.
  • View Versions – View past versions of a list item or document.
  • View Application Pages – View forms, views, and application pages. Enumerate lists. Site Permissions
  • View Web Analytics Data – View reports on Web site usage.
  • Browse Directories – Enumerate files and folders in a Web site using SharePoint Designer and Web DAV interfaces.
  • View Pages – View pages in a Web site.
  • numerate Permissions – Enumerate permissions on the Web site, list, folder, document, or list item.
  • Browse User Information – View information about users of the Web site.
  • Use Remote Interfaces – Use SOAP, Web DAV, the Client Object Model or SharePoint Designer interfaces to access the Web site.
  • Open – Allows users to open a Web site, list, or folder in order to access items inside that container.

Is Microsoft ever going to fix this so that app-auth can be given fine grained permission?


Get this bounty!!!

#StackBounty: #sharepoint-online #development #csom #remote-event-receiver #reverse-proxy I am no longer able to debug/run my SharePoin…

Bounty: 150

Last month the below steps were working well for me to debug and test a SharePoint online remote event receiver locally:-

  1. Open Ngrok.exe >> run the following command inside ngrok:-
 ngrok authtoken 3***e
 ngrok http --host-header=rewrite  57269
  1. register a new app:- @ https://.sharepoint.com/sites//_layouts/15/AppRegNew.aspx >> enter the above Ngrok urls inside the App Redirect URL & App Domain.

  2. Inside the _layouts/15/appinv.aspx >> I search for the above app using Client ID >> and enter the following:-

 <AppPermissionRequests AllowAppOnlyPolicy="true">
   <AppPermissionRequest Scope="http://sharepoint/content/tenant" Right="FullControl" />
 </AppPermissionRequests>
  1. Update the service’s web config with the above ClientID and ClientSecret

  2. Register the new remove event receiver as follow:-

 Add-PnPEventReceiver -List "Order Management" -Name "TasksRER" -Url http://cc6e945e82f6.ngrok.io/service1.svc -EventReceiverType ItemUpdated -Synchronization Asynchronous

But today when I tried the above steps it failed >> where inside my event receiver when I tried to get the SharePoint context >> I will get that the Context is null:-

  public void ProcessOneWayEvent(SPRemoteEventProperties properties)
         {
             var prop = properties;
             var listItemID = properties.ItemEventProperties.ListItemId;
             var listTitle = properties.ItemEventProperties.ListTitle;
             using (ClientContext context = Helpers.GetAppOnlyContext(properties.ItemEventProperties.WebUrl))
             {
                 context.Load(context.Web);
                 context.ExecuteQuery();

Here is a screen shot from Visual Studio with the error i am getting when trying to get the context:-

enter image description here

Any advice if anything has been changed which is preventing me from running the above steps? which were working well last month?
Thanks

here is the code for the GetAppOnlyContext

     public class Helpers
         {
             public static ClientContext GetAppOnlyContext(string siteUrl)
             {
                 try
                 {
                     Uri siteUri = new Uri(siteUrl);
                     string realm = TokenHelper.GetRealmFromTargetUrl(siteUri);
                     string accessToken = TokenHelper.GetAppOnlyAccessToken(TokenHelper.SharePointPrincipal, siteUri.Authority, realm).AccessToken;
        
                     return TokenHelper.GetClientContextWithAccessToken(siteUri.ToString(), accessToken);
                 }
        
                 catch (Exception ex)
                 {
                     Trace.TraceInformation("GetAppOnlyContext failed. {0}", ex.Message);
                 }
                 return null;
             }
        
             public static ClientContext GetAuthenticatedContext(string siteUrl)
             {
                 string userName = WebConfigurationManager.AppSettings.Get("AuthenticatedUserName");
                 string password = WebConfigurationManager.AppSettings.Get("AuthenticatedUserPassword");
                 return GetAuthenticatedContext(siteUrl, userName, password);
             }
        
             public static ClientContext GetAuthenticatedContext(string siteUrl, string userName, SecureString password)
             {
                 ClientContext ctx = new ClientContext(siteUrl);
                 ctx.Credentials = new SharePointOnlineCredentials(userName, password);
                 return ctx;
             }
        
             public static ClientContext GetAuthenticatedContext(string siteUrl, string userName, string password)
             {
                 SecureString securePassword = GetPassword(password);
                 return GetAuthenticatedContext(siteUrl, userName, securePassword);
             }
        
             private static SecureString GetPassword(string passwd)
             {
                 var secure = new SecureString();
                 foreach (char c in passwd)
                 {
                     secure.AppendChar(c);
                 }
                 return secure;
             }
        
             public static string EmptyIfNull(object obj)
             {
                 return obj == null ? "" : obj.ToString();
             }
         }
     }  

seem this statement string accessToken = TokenHelper.GetAppOnlyAccessToken(TokenHelper.SharePointPrincipal, siteUri.Authority, realm).AccessToken; inside my Helpers.GetAppOnlyContext will raise this error token request failed, as follow:-

enter image description here


Get this bounty!!!

#StackBounty: #sharepoint-online #csom #caml-query #large-lists #list-view-paging Difference between camlQuery.AllowIncrementalResults=…

Bounty: 50

My aim is to get all items from a SharePoint List (>5000 items). I have used the standard pagination based approach and it is working fine.

ListItemCollectionPosition position=null;

do{
   CamlQuery caml=new CamlQuery(){ViewXml="<View><RowLimit Scope='RecursiveAll'>5000</RowLimit></View>"};
 caml.ListItemCollectionPosition = position;
   ListItemsCollection items=list.getItems(caml);
   context.Load(items);
   context.executeQuery();
   foreach(var item in items){
   //do stuff
   }
   position=items.ListItemCollectionPosition;
}while(position!=null);

However, I recently came across a code sample where it also includes the line:
caml.AllowIncrementalResults=true; in the article:
https://gp23.com.au/2019/01/16/sharepoint-online-list-view-thresholds/

I would like to know the impact for setting the AllowIncrementalResults=true.
The documentation in msdn is very sparse.


Get this bounty!!!

#StackBounty: #sharepoint-online #csom #caml-query #large-lists #list-view-paging DIfference between camlQuery.AllowIncrementalResuts=t…

Bounty: 50

My aim is to get all items from a SharePoint List (>5000 items). I have used the standard pagination based approach and it is working fine.

ListItemCollectionPosition position=null;

do{
   CamlQuery caml=new CamlQuery(){ViewXml="<View><RowLimit Scope='RecursiveAll'>5000</RowLimit></View>"};
 caml.ListItemCollectionPosition = position;
   ListItemsCollection items=list.getItems(caml);
   context.Load(items);
   context.executeQuery();
   foreach(var item in items){
   //do stuff
   }
   position=items.ListItemCollectionPosition;
}while(position!=null);

However, I recently came across a code sample where it also includes the line:
caml.AllowIncrementalResults=true; in the article:
https://gp23.com.au/2019/01/16/sharepoint-online-list-view-thresholds/

I would like to know the impact for setting the AllowIncrementalResults=true.
The documentation in msdn is very sparse.


Get this bounty!!!

#StackBounty: #sharepoint-online #development #csom #remote-event-receiver Reference/Update the current item inside ItemAdded/ing and I…

Bounty: 50

I am working on remote event receivers, and let say I want to update a field named ProjectID inside my ItemAdded, currently, I am executing a CAML query to get the current item >> update the ProjectID, as follow:-

public void ProcessOneWayEvent(SPRemoteEventProperties properties)
        {
    var listItemID = properties.ItemEventProperties.ListItemId;
    var listTitle = properties.ItemEventProperties.ListTitle;
    using (ClientContext context = Helpers.GetAppOnlyContext(properties.ItemEventProperties.WebUrl))
            {

                CamlQuery camlQuery = new CamlQuery();
                context.Load(context.Web);
                context.ExecuteQuery();

                string webrelativeurl = context.Web.ServerRelativeUrl;

                camlQuery.ViewXml = string.Format("<View Scope="RecursiveAll"><Query><Where><Eq><FieldRef Name='ID' /><Value Type='Number'>{0}</Value></Eq></Where></Query></View>", listItemID);
                ListItemCollection collListItem = context.Web.GetList(webrelativeurl + "/lists/" + "Projects").GetItems(camlQuery);
                context.Load(collListItem, items => items.Include(
                            item => item.Id,
                             item => item["ProjectID"],
                             item => item["ID"],
                             item => item.RoleAssignments.Include(roleAssigned => roleAssigned.Member.Title, roleAssigned => roleAssigned.RoleDefinitionBindings)



                              ));
                context.ExecuteQuery();
                //get the template id based on the entered Purpose

   foreach (ListItem currentFilingSystemItem in collListItem)
      {

      ListItemCreationInformation listItemCreationInformation = new ListItemCreationInformation();
      ListItem listItem = context.Web.GetList(webrelativeurl + "/lists/" + "Projects").AddItem(listItemCreationInformation);

      listItem["ProjectID"] = "ARQ-Project-" + listItem["ID"];
      listItem.Update();

So is my approach correct? or I can directly reference the current item instead of getting it by CAML? the issue is that I can not reference properties.ListItem as in the server-side event receivers.


Get this bounty!!!

#StackBounty: #document-library #csom #sharepoint-on-prem #versioning #flow Combining five files with same content but different names …

Bounty: 50

I am working with SP2019, on the file share there is a Legal folder added which gets lot of files dumped into it. Once users work on files, they save it there and send it to external party for review.

  • For example a user could save a file named ‘General Accounting Assessment’.
  • The external party reviews it, then the file comes back named ‘General Accounting Assessment_V1’ and sits on the same shared folder as the original file.

This review process happens multiple times and that results into 4-5 files on the file share with below names:
1. ‘General Accounting Assessment’
2. ‘General Accounting Assessment_v1’
3. ‘General Accounting Assessment_v3’
4. ‘General Accounting Assessment_v4’

Users initially save a file ‘General Accounting Assessment’ to SharePoint prior to sending it out for review to external party so when v1, v2, v3 and v4 version files arrive, they manually rename the files to match it to name ‘General Accounting Assessment’ and drop into SharePoint so it can add up as a new version to the existing file.

I would like to automate this process and for which I am wondering if there is a way maybe by Flow or SharePoint to check whenever a new file is added, strip off the portion _v1 or _v2 or _v3 or _v4 dynamically appended at the end to file name ‘General Accounting Assessment’ and save it to SharePoint

Can someone please help on the proposed solution, thanks in advance.


Get this bounty!!!

#StackBounty: #document-library #csom #sharepoint-on-prem #versioning #flow Combing five files with same content but different names in…

Bounty: 50

I am working with SP2019 and On the file share, there is a Legal folder added which gets lot of files dumped into it. Once users work on files, they save it there and send it to external party for review, this file as an example be named as ‘General Accounting Assessment’ and post review it comes back as ‘General Accounting Assessment_V1’ and sits on the same shared folder as the original file.

This review process happen multiple times and that results into 4-5 files on the file share with below names:
1. ‘General Accounting Assessment’
2. ‘General Accounting Assessment_v1’
3. ‘General Accounting Assessment_v3’
4. ‘General Accounting Assessment_v4’

Users initially save file ‘General Accounting Assessment’to SharePoint prior to sending out for review to external party so when v1, v2,v3 and v4 version file arrives, they manually rename the files to match it to name ‘General Accounting Assessment’ and drop into SharePoint so it can add up as a new version to the existing file.

I would like to automate this process and for which I am wondering if there is a way maybe by Flow or SharePoint to check whenever a new file is added, strip off the portion _v1 or _v2 or _v3 or _v4 dynamically appended at the end to file name ‘General Accounting Assessment’ and save it to SharePoint

Can someone please help on the proposed solution, thanks in advance.


Get this bounty!!!

#StackBounty: #development #csom #getchanges Why does Changes API return changes for Lists that do not exist?

Bounty: 50

I am using the Changes API to fetch changes:

ChangeQuery cq = new ChangeQuery(true, true);
cq.ChangeTokenStart = new ChangeToken();
cq.ChangeTokenStart.StringValue = string.Format("1;1;{0};{1};-1", site.Id, DateTime.Now.AddDays(-1).ToUniversalTime().Ticks.ToString());
cq.FetchLimit = 1000;
cq.RecursiveAll = false;
cq.GroupMembershipAdd = false;
cq.GroupMembershipDelete = false;
cq.RoleAssignmentAdd = false;
cq.RoleAssignmentDelete = false;
cq.RoleDefinitionUpdate = false;
cq.RoleDefinitionAdd = false;
cq.RoleDefinitionDelete = false;
cq.User = false;
cq.SystemUpdate = false;
cq.Group = false;
cq.SecurityPolicy = false;
cq.Update = false;
cq.Add = true;
cq.DeleteObject = false;
cq.Item = true;
cq.File = false;
cq.Folder = false;
cq.Alert = false;
cq.Web = false;
cq.LatestFirst = false;//Sorting

while (cq.ChangeTokenStart != null)
{
    var changes = context.Site.GetChanges(cq);
    context.Load(changes);
    context.ExecuteQuery();
    Console.WriteLine("Page");
    Change lastChangeItem = null;


    foreach (Microsoft.SharePoint.Client.Change change in changes)
    {
        Console.WriteLine(string.Format("Chnage is {0}", change.ChangeType));


        if (change is ChangeItem)
        {
            ChangeItem changeItem = (ChangeItem)change;
            List list = context.Web.Lists.GetById(changeItem.ListId);
            context.Load(list);
            context.Load(list.Fields);
            context.ExecuteQuery();
            Console.Write("List {0}", list.Id);
        }

        lastChangeItem = change;
    }

    if (lastChangeItem == null)
    {
        cq.ChangeTokenStart = null;
    }
    else
    {
        cq.ChangeTokenStart = lastChangeItem.ChangeToken;
    }
}

But I’m getting errors when trying to do this. Some of the ChangeItem values refer to lists that do not exist:

The page you selected contains a list that does not exist.  It may have been deleted by another user.

I did not delete any lists during my test. Just added new site and some new documents to a document library.

So in summary, I’m getting “ghost” lists and list items that do not actually exist as far as I can tell.

What are these “ghost ChangeItem” values? where are they coming from and should I be doing something to filter them?


Get this bounty!!!

#StackBounty: #sharepoint-online #csom Is there a way to get all paging info for a list up front so that very large lists can be fetche…

Bounty: 50

I am using SharePoint online and SharePoint 2013+ onprem.

I have written a CSOM rest program that fetches all the items in a list and fetches list items in 5000 item batches, the maximum allowed by the list item view threshold.

Here is an example of the XML payload of the CSOM request to get one of the pages.

<Request AddExpandoFieldTypeSuffix="true" SchemaVersion="14.0.0.0" LibraryVersion="16.0.0.0"
         ApplicationName=".NET Library" xmlns="http://schemas.microsoft.com/sharepoint/clientquery/2009">
    <Actions>
        <ObjectPath Id="10019" ObjectPathId="10018"/>
        <Query Id="10020" ObjectPathId="10018">
            <Query SelectAllProperties="true">
                <Properties/>
            </Query>
            <ChildItemQuery SelectAllProperties="true">
                <Properties>
                    <Property Name="Id" ScalarProperty="true"/>
                    <Property Name="DisplayName" ScalarProperty="true"/>
                    <Property Name="HasUniqueRoleAssignments" ScalarProperty="true"/>
                    <Property Name="File" SelectAll="true">
                        <Query SelectAllProperties="false">
                            <Properties/>
                        </Query>
                    </Property>
                    <Property Name="ContentType" SelectAll="false">
                        <Query SelectAllProperties="false">
                            <Properties>
                                <Property Name="Name" ScalarProperty="true"/>
                            </Properties>
                        </Query>
                    </Property>
                </Properties>
            </ChildItemQuery>
        </Query>
    </Actions>
    <ObjectPaths>
        <Method Id="10018" ParentId="9" Name="GetItems">
            <Parameters>
                <Parameter TypeId="{3d248d7b-fc86-40a3-aa97-02a75d69fb8a}">
                    <Property Name="AllowIncrementalResults" Type="Boolean">false</Property>
                    <Property Name="DatesInUtc" Type="Boolean">true</Property>
                    <Property Name="FolderServerRelativePath" Type="Null"/>
                    <Property Name="FolderServerRelativeUrl" Type="Null"/>
                    <Property Name="ListItemCollectionPosition" TypeId="{922354eb-c56a-4d88-ad59-67496854efe1}">
                        <Property Name="PagingInfo" Type="String">Paged=TRUE&amp;p_ID=10000</Property>
                    </Property>
                    <Property Name="ViewXml" Type="String">&lt;View Scope="RecursiveAll"&gt;&lt;RowLimit&gt;5000&lt;/RowLimit&gt;&lt;/View&gt;</Property>
                </Parameter>
            </Parameters>
        </Method>
        <Identity Id="9"
                  Name="740c6a0b-85e2-48a0-a494-e0f1759d4aa7:site:2386a403-8d76-4737-b774-dabad52201e3:web:7a2f544f-e3ed-444e-8de3-178c2c9b5848:list:63029d5e-0588-4406-89f4-9b5a69ae5d81"/>
    </ObjectPaths>
</Request>

Notice the Paged=TRUE&amp;p_ID=10000 is specified which means get me the next page of results starting at p_ID 10000.

So the simple algorithm to fetch all the items is this:

next_page_id = 0
while (next_page_id is not null)
   get next page of list items starting at next_page_id.
   if there is another page
      set next_page_id to that page id.
   else
      set next_page_id to null.

But the main flaw with this is that now a list with 1,000,000 items can only be fetched with a single thread. Which means that this can potentially fetch for hours and hours if a single list has millions of records in it.

To get past this, I tried to use the List.ItemCount to create a blocking collection of pages that we need to fetch. For example, if you have 150000 records in a list, create Paged=TRUE&amp;p_ID=0, Paged=TRUE&amp;p_ID=5000, and Paged=TRUE&amp;p_ID=10000 and a thread pool can grab pages to fetch until there is no more work to do.

But this isn’t correct. You will end up with incomplete lists, because the p_ID is not the “index of result.” It has some other meaning that I do fully understand.

Is there a way to do what I am trying to do?


Get this bounty!!!

#StackBounty: #powershell #csom #memory CSOM Powershell eating up memory

Bounty: 50

What is the proper way to free up memory in Powershell when looping trough list items?
This loop is slowly eating up the memory:

for ($ID= 94368; $ID -le 98970 ; $ID++)
# foreach ($ID in $Ids)
{

try
{

    $item = $list.GetItemById($ID)
    $ctx.Load($item)
    $ctx.ExecuteQuery()

    if ($item["TaskStatus"] -eq "Nezačaté" )
    {

        Write-Host "Nastavim jedinecne prava na polozku $ID" -NoNewline
        changePermissions -listName $listName -itemId $ID
        Write-Host " OK" -ForegroundColor Green

    } else {

        Write-Host "Nastavim dedenie prav na polozku $ID" -NoNewline
        resetPermissions -listName $listName -itemId $ID
        Write-Host " OK" -ForegroundColor Green

    }
} 
catch 
{
  write-host "$($_.Exception.Message)" -foregroundcolor red
} 
finally
{
    $item.Dispose()  # Not existing method
    [GC]::Collect()  # Not helping

}


}

I think the $ctx (ClientContext) variable is getting bigger in each loop, but it is just my opinion. Is there something like $ctx.unload($item)? Or I must dispose the $ctx in each loop and retrieve it again? (it will impact on speed)
Appreciate any help.


Get this bounty!!!