#StackBounty: #c# #wpf #kinect #gesture #kinect-v2 Kinect Swipe Conflict between left hand and right hand

Bounty: 50

I have written a Kinect swipe testing program from scratch that read gbd files produced by Visual gesture builder Kinect Swipe Test Project here.

The Main 4 gestures I want to focus in this questions is Left & right Hands up, Left & right hand Swipe up.

In the github link i posted above it works fine integrated together.

However, when i port the code over to my main project, Left hand Hands
Up and Right hand Swipe up gestures both malfunctioned. Both gesture
are unable to be detected.

I cannot post my Main project here so i will be posting the code snippets of how i implemented on the 2 Main project files, Mainly GestureDetector.cs and GestureResultsView.cs

Here are the respective snippet of the i had issue with files.

GestureDetector.cs :

    public class GestureDetector : IDisposable
    {
        //stores an array of paths that leads to the gbd files
        private string[] databaseArray = {//Left Hand
                                           @"DatabaseLeftHandDownToTop.gbd"
                                         //, @"DatabaseLeftHandHandsUp.gbd"
                                         //, @"DatabaseLeftHandLeftToRight.gbd"
                                         //, @"DatabaseLeftHandRightToLeft.gbd"
                                          //Right Hand
                                         , @"DatabaseRightHandBottomToUp.gbd"
                                         , @"DatabaseRightHandHandsUp.gbd"
                                         //, @"DatabaseRightHandLeftToRight.gbd"
                                         //, @"DatabaseRightHandRightToLeft.gbd"
                                         };

        //stores an array of the names of the gesture found inside the gbd file
        private string[] databaseGestureNameArray = { //Swipe Up
                                                      "SwipeUp_Left",
                                                     "SwipeUp_Right",
                                                    //  //Swipe Right
                                                    // "SwipeRight_Left",
                                                    // "SwipeRight_Right",
                                                    //  //Swipe Left
                                                    // "SwipeLeft_Left",
                                                    // "SwipeLeft_Right",
                                                      //Hands Up
                                                     //"HandsUp_Left",
                                                     "HandsUp_Right"
                                                    };


            for (int i = 0; i < databaseArray.Length; i++)
            {
                // load the 'Seated' gesture from the gesture database
                using (VisualGestureBuilderDatabase database = new VisualGestureBuilderDatabase(databaseArray[i]))
                {
                    // we could load all available gestures in the database with a call to vgbFrameSource.AddGestures(database.AvailableGestures), 
                    // but for this program, we only want to track one discrete gesture from the database, so we'll load it by name
                    foreach (Gesture gesture in database.AvailableGestures)
                    {
                        if (gesture.Name.Equals(databaseGestureNameArray[i]))
                        {
                            this.vgbFrameSource.AddGesture(gesture);

                        }
                    }
                }
            }
        }


        /// <summary>
        /// Handles gesture detection results arriving from the sensor for the associated body tracking Id
        /// </summary>
        /// <param name="sender">object sending the event</param>
        /// <param name="e">event arguments</param>
        private void Reader_GestureFrameArrived(object sender, VisualGestureBuilderFrameArrivedEventArgs e)
        {
            VisualGestureBuilderFrameReference frameReference = e.FrameReference;
            using (VisualGestureBuilderFrame frame = frameReference.AcquireFrame())
            {
                if (frame != null)
                {
                    // get the discrete gesture results which arrived with the latest frame
                    IReadOnlyDictionary<Gesture, DiscreteGestureResult> discreteResults = frame.DiscreteGestureResults;
                    bool foundGesture = false;
                    if (discreteResults != null)
                    {
                        // we only have one gesture in this source object, but you can get multiple gestures
                        foreach (Gesture gesture in this.vgbFrameSource.Gestures)
                        {
                            for (int i = 0; i < databaseGestureNameArray.Length; i++)
                            {
                                if (gesture.Name.Equals(databaseGestureNameArray[i]) && gesture.GestureType == GestureType.Discrete)
                                {
                                    DiscreteGestureResult result = null;
                                    discreteResults.TryGetValue(gesture, out result);

                                    if (result != null && !foundGesture)
                                    {

                                        // update the GestureResultView object with new gesture result values & update the label
                                        foundGesture = this.GestureResultView.UpdateGestureResult(true, result.Detected, result.Confidence, databaseGestureNameArray[i]);


                                    }
                                }
                            }
                        }
                    }
                }
            }
        }

GestureResultView.cs :

        public bool UpdateGestureResult(bool isBodyTrackingIdValid, bool isGestureDetected, float detectionConfidence, string gestureName)
        {
            this.IsTracked = isBodyTrackingIdValid;
            this.Confidence = 0.0f;
            bool gestureFound = false;

            if (!this.IsTracked)
            {
                this.ImageSource = this.notTrackedImage;
                this.Detected = false;
                this.BodyColor = Brushes.Gray;
            }
            else
            {
                this.Detected = isGestureDetected;
                this.BodyColor = this.trackedColors[this.BodyIndex];

                if (this.Detected)
                {
                    this.Confidence = detectionConfidence;
                    if (this.Confidence > 0.4)
                    {
                        this.ImageSource = this.seatedImage;
                        //https://stackoverflow.com/questions/15425495/change-wpf-mainwindow-label-from-another-class-and-separate-thread
                        //to change label in other class wpf
                        //http://the--semicolon.blogspot.co.id/p/change-wpf-window-label-content-from.html

                        string state = App.Current.Properties["state"].ToString();
                        switch (gestureName)
                        {
                            case "SwipeUp_Right":
                                SwipeUp(state);
                                break;

                            case "SwipeUp_Left":
                                SwipeUp(state);
                                break;

                            case "SwipeDown_Right":
                                break;

                            case "SwipeDown_Left":
                                break;

                            case "SwipeLeft_Right":
                                break;

                            case "SwipeLeft_Left":
                                break;

                            case "HandsUp_Right":
                                if (state.Equals("GoBackHome"))
                                {

                                }
                                else
                                {
                                    Thread.Sleep(350);
                                    MainWindow.handsUp();
                                }

                                break;

                            case "HandsUp_Left":
                                if (state.Equals("GoBackHome"))
                                {

                                }
                                else
                                {
                                    Thread.Sleep(350);
                                    MainWindow.handsUp();
                                }
                                break;




                        }
                        //"HandsUp_Right"
                        //                            , "SwipeRight_Right"
                        //                            , "SwipeUp_Right"
                        //                            , "SwipeLeft_Right"
                        //                            , "HandsUp_Left"
                        //                            , "SwipeRight_Left"


                        gestureFound = true;
                    }

                }
                else
                {
                    this.ImageSource = this.notSeatedImage;
                }
            }
            return gestureFound;
        }


        //Routing for gesture start
        /// <summary>
        /// Take in the current screen, filtered swipe up gesture
        /// </summary>
        /// <param name="state"></param>
        private void SwipeUp(string state)
        {
            if (state.Equals("Home"))
            {
                int milliseconds = 350;
                Thread.Sleep(milliseconds);
                //await Task.Delay(500);
                Home.swipeUp();
            }
            else if (state.Equals("ItemChoice"))
            {
                int milliseconds = 350;
                Thread.Sleep(milliseconds);
                //await Task.Delay(500);
                ItemChoice.swipeUp();

            }
            else if (state.Equals("ItemParentChoice"))
            {
                int milliseconds = 350;
                Thread.Sleep(milliseconds);
                //await Task.Delay(500);
                ItemParentChoice.swipeUp();
            }
            else if (state.Equals("LoanSummary"))
            {
                int milliseconds = 350;
                Thread.Sleep(milliseconds);
                //await Task.Delay(500);
                LoanSummary.swipeUp();
            }
            else if (state.Equals("Timeslot"))
            {
                int milliseconds = 350;
                Thread.Sleep(milliseconds);
                //await Task.Delay(500);
                Timeslot.swipeUp();
            }
            else if (state.Equals("ViewLoans"))
            {
                int milliseconds = 350;
                Thread.Sleep(milliseconds);
                //await Task.Delay(500);
                ViewLoans.swipeUp();
            }
            else if (state.Equals("WhatToDo"))
            {
                int milliseconds = 350;
                Thread.Sleep(milliseconds);
                //await Task.Delay(500);
                //WhatToDo.swipeUp();
            }
        }

        /// <summary>
        /// Take in the current screen, filtered swipe right gesture
        /// </summary>
        /// <param name="state"></param>
        private void swipeRight(string state)
        {
            if (state.Equals("Home"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //Home.swipeUp();
            }
            else if (state.Equals("ItemChoice"))
            {
                int milliseconds = 500;
                Thread.Sleep(milliseconds);
                ItemChoice.swipeRight();
            }
            else if (state.Equals("ItemParentChoice"))
            {
                int milliseconds = 500;
                Thread.Sleep(milliseconds);
                ItemParentChoice.swipeRight();
            }
            else if (state.Equals("LoanSummary"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //LoanSummary.swipeUp();
            }
            else if (state.Equals("Timeslot"))
            {
                int milliseconds = 500;
                Thread.Sleep(milliseconds);
                Timeslot.swipeRight();
            }
            else if (state.Equals("ViewLoans"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //ViewLoans.swipeUp();
            }
            else if (state.Equals("WhatToDo"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //Home.swipeUp();
            }
        }

        /// <summary>
        /// Take in the current screen, filtered swipe right gesture
        /// </summary>
        /// <param name="state"></param>
        private void swipeLeft(string state)
        {
            if (state.Equals("Home"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //Home.swipeUp();
            }
            else if (state.Equals("ItemChoice"))
            {
                int milliseconds = 500;
                Thread.Sleep(milliseconds);
                ItemChoice.swipeLeft();
            }
            else if (state.Equals("ItemParentChoice"))
            {
                int milliseconds = 500;
                Thread.Sleep(milliseconds);
                ItemParentChoice.swipeLeft();
            }
            else if (state.Equals("LoanSummary"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //LoanSummary.swipeUp();
            }
            else if (state.Equals("Timeslot"))
            {
                int milliseconds = 500;
                Thread.Sleep(milliseconds);
                Timeslot.swipeLeft();
            }
            else if (state.Equals("ViewLoans"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //ViewLoans.swipeUp();
            }
            else if (state.Equals("WhatToDo"))
            {
                //int milliseconds = 500;
                //Thread.Sleep(milliseconds);
                //Home.swipeUp();
            }
        }





        //routing for gesture end



    }
}


Get this bounty!!!

#StackBounty: #c# #authentication #oauth #asp.net-core OAuth2 login for ASP.NET Core without local accounts

Bounty: 50

I am using OAuth2 for users to sign in with their Google Account and authenticate Google API requests.

I have not used the built-in ASP.NET Identity provider because this requires an app database of local user accounts (passwords, etc). I just want Google to tell us who the user is, keep hold of their auth/refresh tokens, and set an auth cookie to remember them.

Helper class:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Claims;
using System.Threading;
using System.Threading.Tasks;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Auth.OAuth2.Flows;
using Google.Apis.Auth.OAuth2.Responses;
using Google.Apis.Auth.OAuth2.Web;
using Google.Apis.Oauth2.v2;
using Google.Apis.Requests;
using Google.Apis.Services;
using Google.Apis.Util.Store;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.Authentication.Cookies;
using Microsoft.AspNetCore.Http.Extensions;
using Microsoft.AspNetCore.Mvc;

public static class GoogleOAuth2
{
  private static string AppName { get; set; }
  private static GoogleAuthorizationCodeFlow Flow { get; set; }
  private static GoogleAuthorizationCodeFlow FlowPromptConsent { get; set; }

  public static void Configure(string appName, IEnumerable<string> scopes, string clientId, string clientSecret, IDataStore dataStore)
  {
    AppName = appName;
    Flow = new GoogleAuthorizationCodeFlow(new GoogleAuthorizationCodeFlow.Initializer {
      DataStore = dataStore,
      ClientSecrets = new ClientSecrets {
        ClientId = clientId,
        ClientSecret = clientSecret
      },
      Scopes = scopes
    });
    FlowPromptConsent = new GoogleAuthorizationCodeFlow(new GoogleAuthorizationCodeFlow.Initializer {
      DataStore = dataStore,
      ClientSecrets = new ClientSecrets {
        ClientId = clientId,
        ClientSecret = clientSecret
      },
      Scopes = scopes,
      UserDefinedQueryParams = new[] { KeyValuePair.Create("prompt", "consent") }
    });
  }

  public static string GetEmail(this ClaimsPrincipal user)
  {
    var claims = (ClaimsIdentity)user.Identity;
    return claims.FindFirst(ClaimTypes.Email)?.Value;
  }

  public static async Task<BaseClientService.Initializer> GetInitializerAsync(string email)
  {
    var token = await Flow.DataStore.GetAsync<TokenResponse>(email) ?? throw new KeyNotFoundException("User token not found in the data store.");
    return new BaseClientService.Initializer {
      ApplicationName = AppName,
      HttpClientInitializer = new UserCredential(Flow, email, token)
    };
  }

  public static async Task<ActionResult> LoginWithGoogleAsync(this Controller controller)
  {
    var userId = controller.HttpContext.Session.Id;
    var uri = controller.Request.GetDisplayUrl();
    var redirectUri = uri.Substring(0, uri.Length - controller.Request.QueryString.Value.Length); // Current URI without query
    var code = controller.Request.Query["code"].ToString();
    string appReturnUrl = null;

    UserCredential credential = null;

    if (code == string.Empty) {
      // First pass: 'code' is empty
      if (controller.Request.Query.ContainsKey("ReturnUrl")) {
        controller.TempData["ReturnUrl"] = controller.Request.Query["ReturnUrl"].ToString();
      }
      var auth = await new AuthorizationCodeWebApp(Flow, redirectUri, redirectUri).AuthorizeAsync(userId, CancellationToken.None);
      if (auth.RedirectUri != null) {
        return controller.Redirect(auth.RedirectUri); // To Google login page
      }
      credential = auth.Credential;
    } else {
      // On return from the Google login page:
      var token = await Flow.ExchangeCodeForTokenAsync(userId, code, redirectUri, CancellationToken.None);
      credential = new UserCredential(Flow, userId, token);
    }

    var oauthService = new Oauth2Service(new BaseClientService.Initializer { HttpClientInitializer = credential });
    var userInfo = await oauthService.Userinfo.Get().ExecuteAsync();

    // Check if we already have a token for this email address

    var store = Flow.DataStore;
    var oldToken = await store.GetAsync<TokenResponse>(userInfo.Email);
    var newToken = await store.GetAsync<TokenResponse>(userId);
    await store.DeleteAsync<TokenResponse>(userId); // Clean up token store
    await store.DeleteAsync<string>("oauth_" + userId); // Clean up token store

    if (newToken.RefreshToken == null) {

      // If Google didn't give us a refresh token, take the previously saved one if
      // it exists, or redirect back to Google and force a consent screen

      if (oldToken?.RefreshToken != null) {
        newToken.RefreshToken = oldToken.RefreshToken;
      } else {
        controller.TempData.Keep("ReturnUrl");
        var auth = await new AuthorizationCodeWebApp(FlowPromptConsent, redirectUri, redirectUri).AuthorizeAsync(userId, CancellationToken.None);
        return controller.Redirect(auth.RedirectUri);
      }

    }
    await store.StoreAsync(userInfo.Email, newToken); // Save new token, using user's email as key

    var identity = new ClaimsIdentity(CookieAuthenticationDefaults.AuthenticationScheme);
    identity.AddClaim(new Claim(ClaimTypes.Name, $"{userInfo.GivenName} {userInfo.FamilyName}"));
    identity.AddClaim(new Claim(ClaimTypes.Email, userInfo.Email));
    await controller.HttpContext.SignInAsync(CookieAuthenticationDefaults.AuthenticationScheme, new ClaimsPrincipal(identity), new AuthenticationProperties { IsPersistent = true });

    appReturnUrl = controller.TempData["ReturnUrl"]?.ToString() ?? "/";
    return controller.Redirect(appReturnUrl);
  }
}

Note that session state is used to hold a temporary User Id while communicating with Google, but once we’ve got an email address for the user this is used as the permanent key in the token store.

Login:

[AllowAnonymous]
[Route("/login")]
public async Task<IActionResult> Login()
{
  if (User.Identity.IsAuthenticated) return Redirect("/");
  return await this.LoginWithGoogleAsync();
}

Google API calls:

var service = new CalendarService(await GoogleOAuth2.GetInitializerAsync(User.GetEmail()));
var calendars = await service.CalendarList.List().ExecuteAsync();


Get this bounty!!!

#StackBounty: #c# #windows-10 #visual-studio-2017 #excel-interop #excel-2016 WIndows 10 Excel 2016 C# hresult 0x800a03ec error on save as

Bounty: 50

I have Windows 10 OS, Excel 2016 and working on Visual Studio 2017.

I’m using Microsoft Excel 16.0 Object Library, version 1.9.

I want to save 50 rows having 2 columns in Excel file.

I get error hresult ‘0x800a03ec’ on saveas method.

I have seen more than 25+ links till now, which are previously associated with the same error, but no clue.

Here is my code:

            object misValue = System.Reflection.Missing.Value;
            var excel = new Application();

            // for making Excel visible
            excel.Visible = false;
            excel.DisplayAlerts = false;

            // Creation a new Workbook
            var excelWorkBook = excel.Workbooks.Add(Type.Missing);
            // Work sheet
            var excelSheet = (Worksheet)excelWorkBook.ActiveSheet;
            excelSheet.Name = "xxxxx";

            excelSheet.Cells[1][1] = "head1";
            excelSheet.Cells[1][2] = "head2";

            for (int i = 0; i < list.Count; i++)
            {
                excelSheet.Cells[i + 2][1] = list[i].xxx;
                excelSheet.Cells[i + 2][2] = list[i].yyy;
            }

            excelSheet.SaveAs(@"E:data.xlsx", Microsoft.Office.Interop.Excel.XlFileFormat.xlWorkbookNormal, misValue, misValue, misValue, misValue,
                Microsoft.Office.Interop.Excel.XlSaveAsAccessMode.xlExclusive, misValue, misValue, misValue);
            excel.Quit();

            Marshal.ReleaseComObject(excelSheet);
            Marshal.ReleaseComObject(excelWorkBook);
            Marshal.ReleaseComObject(excel);

Need help.

Edit:

1) xls or xlsx, none of them works.

2) Based on some previous answers found, I tried this too:
Console Root -> Computers -> DCOM Config -> Microsoft Excel Application -> Not found, triple checked.

Someone please try to apply the same scenario in your machine & let me know if you can find the answers.


Get this bounty!!!

#StackBounty: #c# #asp.net #.net #iis #odp.net How can I get unmanaged ODP.NET (64-bit) working in ASP.NET on 64-bit O/S?

Bounty: 100

I have to get a legacy application that uses unmanaged ODP.NET working on a 64-bit machine.

  1. I have installed 64-bit unmanaged ODP.NET
  2. I have verified that the 64-bit unmanaged ODP.NET is working fine by testing it in a console application
  3. I created a test web application with the same data access code as the working console application and a reference to the same Oracle.Data.Access DLL in the GAC that the console application uses. The web application is unable to connect to the database. The code snippet isn’t relevant, as the code works and is very simple (see point 2.).

It fails at conn.Open();

Exception.Message is empty.

Exception.StackTrace is not useful:

 at Oracle.DataAccess.Client.OracleException.HandleErrorHelper(Int32 errCode, 
    OracleConnection conn, IntPtr opsErrCtx, OpoSqlValCtx* pOpoSqlValCtx, Object 
    src, String procedure, Boolean bCheck, Int32 isRecoverable) at 
    Oracle.DataAccess.Client.OracleException.HandleError(Int32 errCode, 
    OracleConnection conn, IntPtr opsErrCtx, Object src) at 
    Oracle.DataAccess.Client.OracleConnection.Open() at 
    TestODPNet.ODPNetTester.Test() in 
    D:ProjectsTestODPNetTestODPNetODPNetTester.cs:line 27

The application is running on Local IIS, not IIS Express.

In IIS, “Enable 32-bit applications” is set to false.

The application is targeting x64-bit platform, not Any CPU.

Locally, I’m using 64-bit Windows 10 and on the server, I’m using 64-bit Windows Server 2012. I’m not able to get unmanaged ODP.NET working on either. So, it doesn’t seem like this has to do with 32-bit binaries as far as I can tell since ODP.NET and both OSes are all 64-bit.

What am I missing? How can I get unmanaged ODP.NET (64-bit) working in ASP.NET on 64-bit O/S?

Note: This question looks a bit similar to other questions, but it is not as far as I can tell

Note 2: Yes, I know that the recommended way is to use managed ODP.NET, but I do not have the source code and thus cannot make that change.


Get this bounty!!!

#StackBounty: #c# #optimization #powerset How can I limit my powerset function?

Bounty: 50

I found a powerset function from a StackOverflow Post:

    public static T[][] FastPowerSet<T>(T[] seq)
    {
        var powerSet = new T[1 << seq.Length][];
        powerSet[0] = new T[0]; // starting only with empty set
        for (int i = 0; i < seq.Length; i++)
        {
            var cur = seq[i];
            int count = 1 << i; // doubling list each time
            for (int j = 0; j < count; j++)
            {
                var source = powerSet[j];
                var destination = powerSet[count + j] = new T[source.Length + 1];
                for (int q = 0; q < source.Length; q++)
                    destination[q] = source[q];
                destination[source.Length] = cur;
            }
        }
        return powerSet;
    }

But I am not interested in the entire powerset. It uses too much memory, is limited to the maxvalue of integer, and takes too long. Imagine I have an unlimited length string that I’m taking the powerset of, I couldn’t store it. I only want 10 values that are located 25% of the way through the powerset.

For instance, if I send an array of 22 values to the powerset function:

var toTakePowerSetOf = {3,-3,39,-39,392,-392,3920,-3920, 9, -9, 92, -92, 920, -920, 9203, -9203, 2, -2, 20, -20, 203, -203}

I normally get a powerset of 4194304 elements. But I only care about the values at index @ 1049089/4194304 which is {3, -9, 203} . How can I edit the fast powerset function to use unlimited precision, go to a specific index, and only store ~10 values.

The index is calculated as 2^n/4 where n is the number of elements in the toTakePowerSetOf. For my example there are 22 elements: 2^22=4194304 . 4194304/4 = 1048576 . The actual value I was seeking is at 1049089 which is 513 away. In my question I said I would like to check the surrounding 10 values, but I guess I should really say I’d like to check about 513 values (.0122 %) of numbers around it.

This algorithm I’m trying to use may lead to faster factoring, but I need to find out if its the case that factors always appear near 25% of the way through the powerset. It may not make much sense, but if it works I’ll share it with you and we can solve P=NP lol


Get this bounty!!!

#StackBounty: #c# #asp.net #.net #web-services #asp.net-core .NET Core 2.0 & old WebService error

Bounty: 50

I’ve been trying to rewrite an older web application in the latest version of .NET Core. The project started out in .NET Core 1.1 but we quickly realized we’d have to move to .NET 2.0 (currently in preview 2) so that we could get access to DataSets/DataTables since our middleware service uses them and we’re not looking to upgrade them as of right now.

From what I’m seeing everything is correct and connecting to the service works as expected when using the “HelloWorldAsync” test. The problem comes when we use our execute method that contains a DataSet as one of the parameters. I’ve literally boiled down the issue to that (the DataSet Paramenter) specific parameter.

Helper Method

    public async static Task<DataSet> CheckTIN(int TIN) {
        ParameterArrayBuilder pab = new ParameterArrayBuilder();
        pab.Add("@TIN", SqlDbType.Int, TIN);

        DataSet ds = await GetData("usp_CheckTIN", pab);
        if (ds == null) {
            ds.Tables.Add();
        }
        return ds;
    }

    public async static Task<string> HellowWorldAsync() {
        BusLogic40.FactorySoapClient factory = new BusLogic40.FactorySoapClient(BusLogic40.FactorySoapClient.EndpointConfiguration.FactorySoap, "http://localhost/BusLogic40/Request.asmx");
        var result = await factory.HelloWorldAsync();

        return result;
    }

    private async static Task<DataSet> GetData(string procedure, ParameterArrayBuilder pab) {
        string ProviderName = "";
        string Service = "";
        string AuthKey = "";
        string CmdName = procedure;

        BusLogic40.FactorySoapClient factory = new BusLogic40.FactorySoapClient(BusLogic40.FactorySoapClient.EndpointConfiguration.FactorySoap, "http://localhost/BusLogic40/Request.asmx");

        var theDataSet = await factory.ExecuteAsync(ProviderName, Service, AuthKey, CmdName, pab.AsDataSet());

        if (theDataSet.Tables.Count > 0) {
            return theDataSet;
        } else {
            return null;
        }
    }

Results

Application version: 1.0.0.0
AspNetCoreEnvironment: Development
Client IP address: ::1
DeveloperMode: true
Event time: 7/5/2017 4:50:58 PM
Exception type: System.ServiceModel.FaultException
handledAt: Platform
Message: System.Web.Services.Protocols.SoapException: Server was unable to read request. ---> System.InvalidOperationException: There is an error in XML document (1, 1269). ---> System.ArgumentException: Column requires a valid DataType.
   at System.Data.DataColumn.set_DataType(Type value)
   at System.Data.XSDSchema.SetProperties(Object instance, XmlAttribute[] attrs)
   at System.Data.XSDSchema.HandleElementColumn(XmlSchemaElement elem, DataTable table, Boolean isBase)
   at System.Data.XSDSchema.HandleParticle(XmlSchemaParticle pt, DataTable table, ArrayList tableChildren, Boolean isBase)
   at System.Data.XSDSchema.HandleComplexType(XmlSchemaComplexType ct, DataTable table, ArrayList tableChildren, Boolean isNillable)
   at System.Data.XSDSchema.InstantiateTable(XmlSchemaElement node, XmlSchemaComplexType typeNode, Boolean isRef)
   at System.Data.XSDSchema.HandleTable(XmlSchemaElement node)
   at System.Data.XSDSchema.HandleDataSet(XmlSchemaElement node, Boolean isNewDataSet)
   at System.Data.XSDSchema.LoadSchema(XmlSchemaSet schemaSet, DataSet ds)
   at System.Data.DataSet.ReadXml(XmlReader reader, XmlReadMode mode, Boolean denyResolving)
   at System.Data.DataSet.System.Xml.Serialization.IXmlSerializable.ReadXml(XmlReader reader)
   at System.Xml.Serialization.XmlSerializationReader.ReadSerializable(IXmlSerializable serializable, Boolean wrappedAny)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read3_Execute()
   at Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer4.Deserialize(XmlSerializationReader reader)
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   --- End of inner exception stack trace ---
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters()
   --- End of inner exception stack trace ---
   at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters()
   at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()
Node name:
Operation ID: 0HL63ONV9T29O
Operation name: GET Home/Index
Problem ID: System.ServiceModel.FaultException at Infx.FIDM.WebUI.SecureWebUI.Helpers.Utilities.GetData
Role instance:
SDK version: aspnet5c:2.0.0

Stack Trace
System.ServiceModel.FaultException: System.Web.Services.Protocols.SoapException: Server was unable to read request. ---> System.InvalidOperationException: There is an error in XML document (1, 1269). ---> System.ArgumentException: Column requires a valid DataType.
   at System.Data.DataColumn.set_DataType(Type value)
   at System.Data.XSDSchema.SetProperties(Object instance, XmlAttribute[] attrs)
   at System.Data.XSDSchema.HandleElementColumn(XmlSchemaElement elem, DataTable table, Boolean isBase)
   at System.Data.XSDSchema.HandleParticle(XmlSchemaParticle pt, DataTable table, ArrayList tableChildren, Boolean isBase)
   at System.Data.XSDSchema.HandleComplexType(XmlSchemaComplexType ct, DataTable table, ArrayList tableChildren, Boolean isNillable)
   at System.Data.XSDSchema.InstantiateTable(XmlSchemaElement node, XmlSchemaComplexType typeNode, Boolean isRef)
   at System.Data.XSDSchema.HandleTable(XmlSchemaElement node)
   at System.Data.XSDSchema.HandleDataSet(XmlSchemaElement node, Boolean isNewDataSet)
   at System.Data.XSDSchema.LoadSchema(XmlSchemaSet schemaSet, DataSet ds)
   at System.Data.DataSet.ReadXml(XmlReader reader, XmlReadMode mode, Boolean denyResolving)
   at System.Data.DataSet.System.Xml.Serialization.IXmlSerializable.ReadXml(XmlReader reader)
   at System.Xml.Serialization.XmlSerializationReader.ReadSerializable(IXmlSerializable serializable, Boolean wrappedAny)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read3_Execute()
   at Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer4.Deserialize(XmlSerializationReader reader)
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   --- End of inner exception stack trace ---
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters()
   --- End of inner exception stack trace ---
   at System.Web.Services.Protocols.SoapServerProtocol.ReadParameters()
   at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()
    at  System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
    at  System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
    at  System.ServiceModel.Channels.ServiceChannelProxy.TaskCreator.<>c__DisplayClass1_0.<CreateGenericTask>b__0(IAsyncResult asyncResult)
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
    at  Infx.FIDM.WebUI.SecureWebUI.Helpers.Utilities.<GetData>d__2.MoveNext() in C:ProjectsFIDMWebInfx.FIDM.WebUI.SecureWebUIHelpersUtilities.cs:line 32
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
    at  Infx.FIDM.WebUI.SecureWebUI.Helpers.Utilities.<CheckTIN>d__0.MoveNext() in C:ProjectsFIDMWebInfx.FIDM.WebUI.SecureWebUIHelpersUtilities.cs:line 11
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
    at  Infx.FIDM.WebUI.SecureWebUI.Controllers.HomeController.<Index>d__0.MoveNext() in C:ProjectsFIDMWebInfx.FIDM.WebUI.SecureWebUIControllersHomeController.cs:line 8
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeActionMethodAsync>d__27.MoveNext()
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeNextActionFilterAsync>d__25.MoveNext()
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ActionExecutedContext context)
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeNextResourceFilter>d__22.MoveNext()
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Rethrow(ResourceExecutedContext context)
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
    at  Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeAsync>d__20.MoveNext()
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  Microsoft.AspNetCore.Builder.RouterMiddleware.<Invoke>d__4.MoveNext()
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  Microsoft.VisualStudio.Web.BrowserLink.BrowserLinkMiddleware.<ExecuteWithFilter>d__7.MoveNext()
    at  End of stack trace from previous location where exception was thrown ---
    at  System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at  System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at  Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.<Invoke>d__7.MoveNext()


Get this bounty!!!

#StackBounty: #python #c++ #c++11 #memory-leaks #swig C++ Memory Leak in Swig Python Module

Bounty: 50

Background

I have created a python module that wraps a c++ program using SWIG. It works just fine, but it has a pretty serious memory leak issue that I think is a result of poorly handled pointers to large map objects. I have very little experience with c++, and I have questions as to whether delete[] can be used on an object created with new in a different function or method.

The program was written in 2007, so excuse the lack of useful c++11 tricks.


The swig extension basically just wraps a single c++ class (Matrix) and a few functions.

Matrix.h

#ifndef __MATRIX__
#define __MATRIX__

#include <string>
#include <vector>
#include <map>
#include <cmath>
#include <fstream>
#include <cstdlib>
#include <stdio.h>
#include <unistd.h>

#include "FileException.h"
#include "ParseException.h"

#define ROUND_TO_INT(n) ((long long)floor(n))
#define MIN(a,b) ((a)<(b)?(a):(b))
#define MAX(a,b) ((a)>(b)?(a):(b))

using namespace std;

class Matrix {

private:


  /**
  * Split a string following delimiters
   */
  void tokenize(const string& str, vector<string>& tokens, const string& delimiters) {

    // Skip delimiters at beginning.
    string::size_type lastPos = str.find_first_not_of(delimiters, 0);
    // Find first "non-delimiter".
    string::size_type pos     = str.find_first_of(delimiters, lastPos);

    while (string::npos != pos || string::npos != lastPos)
    {
      // Found a token, add it to the vector.
      tokens.push_back(str.substr(lastPos, pos - lastPos));
      // Skip delimiters.  Note the "not_of"
      lastPos = str.find_first_not_of(delimiters, pos);
      // Find next "non-delimiter"
      pos = str.find_first_of(delimiters, lastPos);
    }
  }


public:


  // used for efficiency tests
  long long totalMapSize;
  long long totalOp;

  double ** mat; // the matrix as it is stored in the matrix file
  int length;
  double granularity; // the real granularity used, greater than 1
  long long ** matInt; // the discrete matrix with offset
  double errorMax;
  long long *offsets; // offset of each column
  long long offset; // sum of offsets
  long long *minScoreColumn; // min discrete score at each column
  long long *maxScoreColumn; // max discrete score at each column
  long long *sum;
  long long minScore;  // min total discrete score (normally 0)
  long long maxScore;  // max total discrete score
  long long scoreRange;  // score range = max - min + 1
  long long *bestScore;
  long long *worstScore;
  double background[4];

  Matrix() {
    granularity = 1.0;
    offset = 0;
    background[0] = background[1] = background[2] = background[3] = 0.25;
  }

  Matrix(double pA, double pC, double pG, double pT) {
    granularity = 1.0;
    offset = 0;
    background[0] = pA;
    background[1] = pC;
    background[2] = pG;
    background[3] = pT;  
  }

  ~Matrix() {
      for (int k = 0; k < 4; k++ ) {
        delete[] matInt[k];
      }
      delete[] matInt;
      delete[] mat;
      delete[] offsets;
      delete[] minScoreColumn;
      delete[] maxScoreColumn;
      delete[] sum;
      delete[] bestScore;
      delete[] worstScore;
  }


  void toLogOddRatio () {
    for (int p = 0; p < length; p++) {
      double sum = mat[0][p] + mat[1][p] + mat[2][p] + mat[3][p];
      for (int k = 0; k < 4; k++) {
        mat[k][p] = log((mat[k][p] + 0.25) /(sum + 1)) - log (background[k]); 
      }
    }
  }

  void toLog2OddRatio () {
    for (int p = 0; p < length; p++) {
      double sum = mat[0][p] + mat[1][p] + mat[2][p] + mat[3][p];
      for (int k = 0; k < 4; k++) {
        mat[k][p] = log2((mat[k][p] + 0.25) /(sum + 1)) - log2 (background[k]); 
      }
    }
  }

  /**
    * Transforms the initial matrix into an integer and offseted matrix.
   */
  void computesIntegerMatrix (double granularity, bool sortColumns = true);

  // computes the complete score distribution between score min and max
  void showDistrib (long long min, long long max) {
    map<long long, double> *nbocc = calcDistribWithMapMinMax(min,max); 
    map<long long, double>::iterator iter;

    // computes p values and stores them in nbocc[length] 
    double sum = 0;
    map<long long, double>::reverse_iterator riter = nbocc[length-1].rbegin();
    while (riter != nbocc[length-1].rend()) {
      sum += riter->second;
      nbocc[length][riter->first] = sum;
      riter++;      
    }

    iter = nbocc[length].begin();
    while (iter != nbocc[length].end() && iter->first <= max) {
      //cout << (((iter->first)-offset)/granularity) << " " << (iter->second) << " " << nbocc[length-1][iter->first] << endl;
      iter ++;
    }
  }

  /**
    * Computes the pvalue associated with the threshold score requestedScore.
    */
  void lookForPvalue (long long requestedScore, long long min, long long max, double *pmin, double *pmax);

  /**
    * Computes the score associated with the pvalue requestedPvalue.
    */
  long long lookForScore (long long min, long long max, double requestedPvalue, double *rpv, double *rppv);

  /** 
    * Computes the distribution of scores between score min and max as the DP algrithm proceeds 
    * but instead of using a table we use a map to avoid computations for scores that cannot be reached
    */
  map<long long, double> *calcDistribWithMapMinMax (long long min, long long max); 

  void readMatrix (string matrix) {

    vector<string> str;
    tokenize(matrix, str, " t|");
    this->length = 0;
    this->length = str.size() / 4;
    mat = new double*[4];
    int idx = 0;

    for (int j = 0; j < 4; j++) {
      this->mat[j] = new double[this->length];
      for (int i = 0; i < this->length; i++) {
        mat[j][i] = atof(str.at(idx).data());
        idx++;
      }
    }

    str.clear();

  }

}; /* Matrix */

#endif

Matrix.cpp

#include "Matrix.h"

#define MEMORYCOUNT

void Matrix::computesIntegerMatrix (double granularity, bool sortColumns) {
  double minS = 0, maxS = 0;
  double scoreRange;

  // computes precision
  for (int i = 0; i < length; i++) {
    double min = mat[0][i];
    double max = min;
    for (int k = 1; k < 4; k++ )  {
      min = ((min < mat[k][i])?min:(mat[k][i]));
      max = ((max > mat[k][i])?max:(mat[k][i]));
    }
    minS += min;
    maxS += max;
  } 

  // score range
  scoreRange = maxS - minS + 1;

  if (granularity > 1.0) {
    this->granularity = granularity / scoreRange;
  } else if (granularity < 1.0) {
    this->granularity = 1.0 / granularity;
  } else {
    this->granularity = 1.0;
  }

  matInt = new long long *[length];
  for (int k = 0; k < 4; k++ ) {
    matInt[k] = new long long[length];
    for (int p = 0 ; p < length; p++) {
      matInt[k][p] = ROUND_TO_INT((double)(mat[k][p]*this->granularity)); 
    }
  }

  this->errorMax = 0.0;
  for (int i = 1; i < length; i++) {
    double maxE = mat[0][i] * this->granularity - (matInt[0][i]);
    for (int k = 1; k < 4; k++) {
      maxE = ((maxE < mat[k][i] * this->granularity - matInt[k][i])?(mat[k][i] * this->granularity - (matInt[k][i])):(maxE));
    }
    this->errorMax += maxE;
  }

  if (sortColumns) {
    // sort the columns : the first column is the one with the greatest value
    long long min = 0;
    for (int i = 0; i < length; i++) {
      for (int k = 0; k < 4; k++) {
        min = MIN(min,matInt[k][i]);
      }
    }
    min --;
    long long *maxs = new long long [length];
    for (int i = 0; i < length; i++) {
      maxs[i] = matInt[0][i];
      for (int k = 1; k < 4; k++) {
        if (maxs[i] < matInt[k][i]) {
          maxs[i] = matInt[k][i];
        }
      }
    }
    long long **mattemp = new long long *[4];
    for (int k = 0; k < 4; k++) {        
      mattemp[k] = new long long [length];
    }
    for (int i = 0; i < length; i++) {
      long long max = maxs[0];
      int p = 0;
      for (int j = 1; j < length; j++) {
        if (max < maxs[j]) {
          max = maxs[j];
          p = j;
        }
      }
      maxs[p] = min;
      for (int k = 0; k < 4; k++) {        
        mattemp[k][i] = matInt[k][p];
      }
    }

    for (int k = 0; k < 4; k++)  {
      for (int i = 0; i < length; i++) {
        matInt[k][i] = mattemp[k][i];
      }
    }

    for (int k = 0; k < 4; k++) {        
      delete[] mattemp[k];
    }
    delete[] mattemp;
    delete[] maxs;
  }

  // computes offsets
  this->offset = 0;
  offsets = new long long [length];
  for (int i = 0; i < length; i++) {
    long long min = matInt[0][i];
    for (int k = 1; k < 4; k++ )  {
      min = ((min < matInt[k][i])?min:(matInt[k][i]));
    }
    offsets[i] = -min;
    for (int k = 0; k < 4; k++ )  {
      matInt[k][i] += offsets[i];  
    }
    this->offset += offsets[i];
  }

  // look for the minimum score of the matrix for each column
  minScoreColumn = new long long [length];
  maxScoreColumn = new long long [length];
  sum            = new long long [length];
  minScore = 0;
  maxScore = 0;
  for (int i = 0; i < length; i++) {
    minScoreColumn[i] = matInt[0][i];
    maxScoreColumn[i] = matInt[0][i];
    sum[i] = 0;
    for (int k = 1; k < 4; k++ )  {
      sum[i] = sum[i] + matInt[k][i];
      if (minScoreColumn[i] > matInt[k][i]) {
        minScoreColumn[i] = matInt[k][i];
      }
      if (maxScoreColumn[i] < matInt[k][i]) {
        maxScoreColumn[i] = matInt[k][i];
      }
    }
    minScore = minScore + minScoreColumn[i];
    maxScore = maxScore + maxScoreColumn[i];
    //cout << "minScoreColumn[" << i << "] = " << minScoreColumn[i] << endl;
    //cout << "maxScoreColumn[" << i << "] = " << maxScoreColumn[i] << endl;
  }
  this->scoreRange = maxScore - minScore + 1;

  bestScore = new long long[length];
  worstScore = new long long[length];
  bestScore[length-1] = maxScore;
  worstScore[length-1] = minScore;
  for (int i = length - 2; i >= 0; i--) {
    bestScore[i]  = bestScore[i+1]  - maxScoreColumn[i+1];
    worstScore[i] = worstScore[i+1] - minScoreColumn[i+1];
  }


}




/**
* Computes the pvalue associated with the threshold score requestedScore.
 */
void Matrix::lookForPvalue (long long requestedScore, long long min, long long max, double *pmin, double *pmax) {

  map<long long, double> nbocc = calcDistribWithMapMinMax(min,max); 
  map<long long, double>::iterator iter;


  // computes p values and stores them in nbocc[length] 
  double sum = nbocc[length][max+1];
  long long s = max + 1;
  map<long long, double>::reverse_iterator riter = nbocc[length-1].rbegin();
  while (riter != nbocc[length-1].rend()) {
    sum += riter->second;
    if (riter->first >= requestedScore) s = riter->first;
    nbocc[length][riter->first] = sum;
    riter++;      
  }
  //cout << "   s found : " << s << endl;

  iter = nbocc[length].find(s);
  while (iter != nbocc[length].begin() && iter->first >= s - errorMax) {
    iter--;      
  }
  //cout << "   s - E found : " << iter->first << endl;

#ifdef MEMORYCOUNT
  // for tests, store the number of memory bloc necessary
  for (int pos = 0; pos <= length; pos++) {
    totalMapSize += nbocc[pos].size();
  }
#endif

  *pmax = nbocc[length][s];
  *pmin = iter->second;

}



/**
* Computes the score associated with the pvalue requestedPvalue.
 */
long long Matrix::lookForScore (long long min, long long max, double requestedPvalue, double *rpv, double *rppv) {

  map<long long, double> *nbocc = calcDistribWithMapMinMax(min,max); 
  map<long long, double>::iterator iter;

  // computes p values and stores them in nbocc[length] 
  double sum = 0.0;
  map<long long, double>::reverse_iterator riter = nbocc[length-1].rbegin();
  long long alpha = riter->first+1;
  long long alpha_E = alpha;
  nbocc[length][alpha] = 0.0;
  while (riter != nbocc[length-1].rend()) {
    sum += riter->second;
    nbocc[length][riter->first] = sum;
    if (sum >= requestedPvalue) { 
      break;
    }
    riter++;      
  }
  if (sum > requestedPvalue) {
    alpha_E = riter->first;
    riter--;
    alpha = riter->first; 
  } else {
    if (riter == nbocc[length-1].rend()) { // path following the remark of the mail
      riter--;
      alpha = alpha_E = riter->first;
    } else {
      alpha = riter->first;
      riter++;
      sum += riter->second;
      alpha_E = riter->first;
    }
    nbocc[length][alpha_E] = sum;  
    //cout << "Pv(S) " << riter->first << " " << sum << endl;   
  } 

#ifdef MEMORYCOUNT
  // for tests, store the number of memory bloc necessary
  for (int pos = 0; pos <= length; pos++) {
    totalMapSize += nbocc[pos].size();
  }
#endif

  if (alpha - alpha_E > errorMax) alpha_E = alpha;

  *rpv = nbocc[length][alpha];
  *rppv = nbocc[length][alpha_E];   

  delete[] nbocc;
  return alpha;

}


// computes the distribution of scores between score min and max as the DP algrithm proceeds 
// but instead of using a table we use a map to avoid computations for scores that cannot be reached
map<long long, double> *Matrix::calcDistribWithMapMinMax (long long min, long long max) { 

  // maps for each step of the computation
  // nbocc[length] stores the pvalue
  // nbocc[pos] for pos < length stores the qvalue
  map<long long, double> *nbocc = new map<long long, double> [length+1];
  map<long long, double>::iterator iter;

  long long *maxs = new long long[length+1]; // @ pos i maximum score reachable with the suffix matrix from i to length-1

  maxs[length] = 0;
  for (int i = length-1; i >= 0; i--) {
    maxs[i] = maxs[i+1] + maxScoreColumn[i];
  }

  // initializes the map at position 0
  for (int k = 0; k < 4; k++) {
    if (matInt[k][0]+maxs[1] >= min) {
      nbocc[0][matInt[k][0]] += background[k];
    }
  }

  // computes q values for scores greater or equal than min
  nbocc[length-1][max+1] = 0.0;
  for (int pos = 1; pos < length; pos++) {
    iter = nbocc[pos-1].begin();
    while (iter != nbocc[pos-1].end()) {
      for (int k = 0; k < 4; k++) {
        long long sc = iter->first + matInt[k][pos];
        if (sc+maxs[pos+1] >= min) {
          // the score min can be reached
          if (sc > max) {
            // the score will be greater than max for all suffixes
            nbocc[length-1][max+1] += nbocc[pos-1][iter->first] * background[k]; //pow(4,length-pos-1) ;
            totalOp++;
          } else {              
            nbocc[pos][sc] += nbocc[pos-1][iter->first] * background[k];
            totalOp++;
          }
        } 
      }
      iter++;      
    }      
    //cerr << "        map size for " << pos << " " << nbocc[pos].size() << endl;
  }

  delete[] maxs;

  return nbocc;


}

pytfmpval.i

%module pytfmpval
%{
#include "../src/Matrix.h"
#define SWIG_FILE_WITH_INIT
%}

%include "cpointer.i"
%include "std_string.i"
%include "std_vector.i"
%include "typemaps.i"
%include "../src/Matrix.h"

%pointer_class(double, doublep)
%pointer_class(int, intp)

%nodefaultdtor Matrix;

The c++ functions are called in this python module.

tfmp.py

from __future__ import absolute_import, division, print_function, unicode_literals
import pytfmpval.pytfmpval as tfm
from math import ceil

def read_matrix(matrix, bg=[0.25, 0.25, 0.25, 0.25], mat_type="counts", log_type="nat"):
    """
    From a string of space-delimited counts create a Matrix object.

    Break the string into 4 rows corresponding to A, C, G, and T.
    This function also converts it to a log-odds (position weight) matrix if necessary.

    Args:
        matrix_file (str): White-space delimited string of row-concatenated motif matrix.
        bg (list of floats): Background nucleotide frequencies for [A, C, G, T].
        mat_type (str): Type of motif matrix provided. Options are: "counts", "pfm", "pwm".
            "counts" is for raw count matrices for each base at each position.
            "pfm" is for position frequency matrices (frequencies already calculated).
            "pwm" is for position weight matrices (also referred to as position-specific scoring matrices.)
        log_type (str): Base to use for log. Default is to use the natural log. "log2" is the other option.
            This will affect the scores and p-values.

    Returns:
        m (pytfmpval Matrix): Matrix in pwm format.
    """

    try:
        a, c, g, t = bg[0], bg[1], bg[2], bg[3]
        if (len(matrix.split()) % 4) != 0:
            raise ValueError("Uneven rows in motif matrix. Ensure rows of equal length in input.")

        m = tfm.Matrix(a, c, g, t)
        m.readMatrix(matrix)

        if mat_type.upper() == "COUNTS":
            if log_type.upper() == "NAT":
                m.toLogOddRatio()
            elif log_type.upper() == "LOG2":
                m.toLog2OddRatio()
            else:
                print("Improper log type argument, using natural log.")
                m.toLogOddRatio()

        return m

    except ValueError as error:
        print(repr(error))


def score2pval(matrix, req_score):
    """
    Determine the p-value for a given score for a specific motif PWM.

    Args:
        matrix (pytfmpval Matrix): Matrix in pwm format.
        req_score (float): Requested score for which to determine the p-value.

    Returns:
        ppv (float): The calculated p-value corresponding to the score.
    """

    granularity = 0.1
    max_granularity = 1e-10
    decrgr = 10  # Factor to increase granularity by after each iteration.

    pv = tfm.doublep()
    ppv = tfm.doublep()

    while granularity > max_granularity:
        matrix.computesIntegerMatrix(granularity)
        max_s = int(req_score * matrix.granularity + matrix.offset + matrix.errorMax + 1)
        min_s = int(req_score * matrix.granularity + matrix.offset - matrix.errorMax - 1)
        score = int(req_score * matrix.granularity + matrix.offset)

        matrix.lookForPvalue(score, min_s, max_s, ppv, pv)

        if ppv.value() == pv.value():
            return ppv.value()

        granularity = granularity / decrgr

    print("Max granularity exceeded. Returning closest approximation.")
    return ppv.value()


def pval2score(matrix, pval):
    """
    Determine the score for a given p-value for a specific motif PWM.

    Args:
        matrix (pytfmpval Matrix): Matrix in pwm format.
        pval (float): p-value for which to determine the score.

    Returns:
        score (float): The calculated score corresponding to the p-value.
    """

    init_granularity = 0.1
    max_granularity = 1e-10
    decrgr = 10  # Factor to increase granularity by after each iteration.

    pv = tfm.doublep()  # Initialize as a c++ double.
    ppv = tfm.doublep()
    matrix.computesIntegerMatrix(init_granularity)
    max_s = int(matrix.maxScore + ceil(matrix.errorMax + 0.5))
    min_s = int(matrix.minScore)
    granularity = init_granularity

    while granularity > max_granularity:
        matrix.computesIntegerMatrix(granularity)

        score = matrix.lookForScore(min_s, max_s, pval, pv, ppv)

        min_s = int((score - ceil(matrix.errorMax + 0.5)) * decrgr)
        max_s = int((score + ceil(matrix.errorMax + 0.5)) * decrgr)

        if ppv.value() == pv.value():
            break

        granularity = granularity / decrgr

    if granularity <= max_granularity:
        print("Max granularity exceeded. Returning closest score approximation.")

    final_score = (score - matrix.offset) / matrix.granularity
    return final_score

The script I’ve been using for testing.

thresholds.py

#!/usr/bin/env python
"""
Calculate motif thresholds for a given motif file and print to a new file.

Usage: thresholds.py -m <motifs.txt> -o <output.txt> [OPTIONS]

Args:
    -m (str): Input filename of a file containing PWMs.
    -o (str): Output filename.
    -a (float, optional) <0.25>: Background probability for A nucleotides. If not
        provided then all are assumed to be equally likely (all are 0.25).
    -t (float, optional) <0.25>: Background probability for T nucleotides.
    -c (float, optional) <0.25>: Background probability for C nucleotides.
    -g (float, optional) <0.25>: Background probability for G nucleotides.
    -p (int, optinal) <1>: Processor cores to utilize. Will decrease computation time linearly.
    -pc (float, optional) <0.1>: Pseudocounts value to be added to all positions of the motif frequency matrix
        before calculating the probability matrix.
    -pv (float, optional) <0.00001>: P-value to be used for defining thresholds for each motifs.
        I don't recommend changing this.
    -ow (flag, optional): OverWrite: If present, thresholds already present in the input file will be
        replaced in the output file.
"""

import sys
import argparse
from multiprocessing.dummy import Pool as ThreadPool
import itertools
from pytfmpval import tfmp
from timeit import default_timer as timer

from Bio import motifs

# from memory_profiler import profile
import pdb


# @profile
def find_thresh(combo):
    """
    Calculate the detection thresholds for each pwm matrix given the nucleotide
    background frequencies and desired p-value.

    Args:
        combo (tuple): Three element tuple as defined below.
            matrix (tuple): The PWM for which a detection threshold will be found.
                (motif name, rows of matrix concatenated to single string with spaces between positions).
            pval (float): P-value to which threshold will be calculated.
            bg (list of floats): List containing background nucleotide frequencies [A, C, G, T]
    """

    matrix, pval, bg = (combo[0], combo[1], combo[2])

    start = timer()
    mat = tfmp.read_matrix(matrix[1], bg=bg, mat_type="pwm")
    thresh = tfmp.pval2score(mat, pval)
    pdb.set_trace()
    del mat
    end = timer()

    print(matrix[0] + ": " + str(end - start))

    return (matrix[0], thresh)


def main(motif_file, motif_outfile, pc, bp, ow, pv, p):
    matrices = []
    background = {'A': bp[0], 'C': bp[1], 'G': bp[2], 'T': bp[3]}
    print(("Baseline nucleotide frequencies:nt" + str(background)))

    # Calculate thresholds using pytfmpval.
    print(("Reading in motifs."))
    fh = open(motif_file)
    for m in motifs.parse(fh, "jaspar"):
        pfm = m.counts.normalize(pseudocounts=pc)    # Create frequency matrix.
        pwm = pfm.log_odds(background)              # Calculate to log likelihoods vs background.

        # Create matrix string from motif pwm.
        mat = pwm[0] + pwm[1] + pwm[2] + pwm[3]
        mat = [str(x) for x in mat]
        mat = " ".join(mat)
        matrices.append((m.name, mat))

    fh.close()

    # Multiprocessing to use multiple processing cores.
    thresholds = []
    with ThreadPool(p) as pool:
        for x in pool.imap_unordered(find_thresh, zip(matrices, itertools.repeat(pv),
                                                      itertools.repeat(bp)), chunksize=8):
            thresholds.append(x)

    print(("Total motifs read: " + str(len(thresholds))))
    print("Writing output file.")

    return


if __name__ == '__main__':
    parser = argparse.ArgumentParser(usage=__doc__)

    parser.add_argument("-m", "--motif", dest="motif_file", required=True)
    parser.add_argument("-o", "--outfile", dest="motif_outfile", required=True)
    parser.add_argument("-a", "--a_freq", dest="a_freq", required=False, default=0.25, type=float)
    parser.add_argument("-c", "--c_freq", dest="c_freq", required=False, default=0.25, type=float)
    parser.add_argument("-g", "--g_freq", dest="g_freq", required=False, default=0.25, type=float)
    parser.add_argument("-t", "--t_freq", dest="t_freq", required=False, default=0.25, type=float)
    parser.add_argument("-p", "--processors", dest="processors", required=False, default=1, type=int)
    parser.add_argument("-pc", "--pseudocounts", dest="pseudocounts",
                        required=False, default=0.1, type=float)
    parser.add_argument("-pv", "--pval", dest="p_val",
                        required=False, default=0.00001, type=float)
    parser.add_argument("-ow", "--overwrite", action="store_true", required=False)
    args = parser.parse_args()

    if sum([args.a_freq, args.c_freq, args.g_freq, args.t_freq]) == 1:
        bp = [args.a_freq, args.c_freq, args.g_freq, args.t_freq]
    else:
        print("Background frequencies must equal 1. Check input parameters, exiting.")
        sys.exit()

    main(args.motif_file, args.motif_outfile, args.pseudocounts, bp, args.overwrite, args.p_val, args.processors)

Sample input file can be accessed here.

setup.py can be accessed here.


Setup

To create the swig python module for the Matrix class:
swig -python -c++ pytfmpval.i

To build and install the pytfmpval python module:
pip install .

Python module file structure can be seen here, as can other source code I’ve removed. You’ll likely need the MANIFEST.in for the python module to build properly.

Command to test:
python thresholds.py -m sample_motifs.txt -o sample_motifs.results.txt -ow

That will pop you into the python debugger, and can continue using c. You’ll see that most of the Matrix objects created are processed quickly and use little memory. I’ve included one of the more troublesome ones (ARID3A) in the sample input. It takes much longer (~90s) to run and uses much more memory than the others (~2.7 gb), which is fine and expected. But the memory isn’t ever released, so processing a large file of these quickly becomes an issue.


I worry that nbocc in Matrix.cpp is not being properly dereferenced or deleted. Is this use valid?

I have tried using gc.collect() and I am using the multiprocessing module as recommended in this question to call these functions from my python program. I’ve also tried deleting the Matrix object from within python to no avail.

I’m out of characters, but will provide any additional needed info in the comments as well as I can.


Get this bounty!!!

#StackBounty: #c# #asp.net #file-upload #xmlhttprequest FileUploader error when uploading images

Bounty: 50

I’m using nopcommerce and I’ve added code for photo upload using file uploader but I’m getting this error:

Reason: XHR returned response code 404

@model ProductDetailsModel
@using Nop.Core.Domain.Seo;
@using Nop.Core.Infrastructure;
@using Nop.Web.Models.Catalog;
@{
    Layout = "~/Views/Shared/_ColumnsOne.cshtml";

    //title
    Html.AddTitleParts(!String.IsNullOrEmpty(Model.MetaTitle) ? Model.MetaTitle : Model.Name);
    //meta
    Html.AddMetaDescriptionParts(Model.MetaDescription);
    Html.AddMetaKeywordParts(Model.MetaKeywords);
    //page class
    Html.AppendPageCssClassParts("html-product-details-page");

    var seoSettings = EngineContext.Current.Resolve<SeoSettings>();

    //canonical URL
    if (seoSettings.CanonicalUrlsEnabled)
    {
        var productUrl = Url.RouteUrl("Product", new { SeName = Model.SeName }, this.Request.Url.Scheme);
        Html.AddCanonicalUrlParts(productUrl);
    }

    //open graph META tags
    if (seoSettings.OpenGraphMetaTags)
    {
        Html.AddHeadCustomParts("<meta property="og:type" content="product" />");
        Html.AddHeadCustomParts("<meta property="og:title" content="" + Html.Encode(Model.Name) + "" />");
        Html.AddHeadCustomParts("<meta property="og:description" content="" + Html.Encode(Nop.Core.Html.HtmlHelper.StripTags(Model.MetaDescription)) + "" />");
        Html.AddHeadCustomParts("<meta property="og:image" content="" + Model.DefaultPictureModel.ImageUrl + "" />");
        Html.AddHeadCustomParts("<meta property="og:url" content="" + Request.Url.AbsoluteUri + "" />");
        Html.AddHeadCustomParts("<meta property="og:site_name" content="" + Html.Encode(Model.CurrentStoreName) + "" />");
    }

    //Twitter META tags
    if (seoSettings.TwitterMetaTags)
    {
        Html.AddHeadCustomParts("<meta property="twitter:card" content="summary" />");
        Html.AddHeadCustomParts("<meta property="twitter:site" content="" + Html.Encode(Model.CurrentStoreName) + "" />");
        Html.AddHeadCustomParts("<meta property="twitter:title" content="" + Html.Encode(Model.Name) + "" />");
        Html.AddHeadCustomParts("<meta property="twitter:description" content="" + Html.Encode(Nop.Core.Html.HtmlHelper.StripTags(Model.MetaDescription)) + "" />");
        Html.AddHeadCustomParts("<meta property="twitter:image" content="" + Model.DefaultPictureModel.ImageUrl + "" />");
        Html.AddHeadCustomParts("<meta property="twitter:url" content="" + Request.Url.AbsoluteUri + "" />");
    }


    Html.AddCssFileParts("~/Scripts/fineuploader/fineuploader-4.2.2.min.css");
    Html.AddScriptParts("~/Scripts/fineuploader/jquery.fineuploader-4.2.2.min.js");


    

        
    
}
<!--product breadcrumb-->
@section Breadcrumb
{
    @Html.Partial("_ProductBreadcrumb", Model.Breadcrumb)
}
@Html.Widget("productdetails_after_breadcrumb", Model.Id)
@Html.Widget("productdetails_top", Model.Id) @using (Html.BeginRouteForm("Product", new { SeName = Model.SeName }, FormMethod.Post, new { id = "product-details-form" })) {
@Html.Widget("productdetails_before_pictures", Model.Id) @Html.Partial("_ProductDetailsPictures", Model) @Html.Widget("productdetails_after_pictures", Model.Id)
@Html.Partial("_Discontinued", Model)

@Model.Name

@if (!String.IsNullOrEmpty(Model.ShortDescription)) {
@Html.Raw(Model.ShortDescription)
} @Html.Widget("productdetails_overview_top", Model.Id) <!--product reviews--> @Html.Partial("_ProductReviewOverview", Model.ProductReviewOverview) <!--manufacturers--> @Html.Partial("_ProductManufacturers", Model.ProductManufacturers) <!--availability--> @Html.Partial("_Availability", Model) <!--SKU, MAN, GTIN, vendor--> @Html.Partial("_SKU_Man_GTIN_Ven", Model) <!--delivery--> @Html.Partial("_DeliveryInfo", Model) <!--sample download--> @Html.Partial("_DownloadSample", Model) <!--attributes--> @{ var dataDictAttributes = new ViewDataDictionary(); dataDictAttributes.TemplateInfo.HtmlFieldPrefix = string.Format("attributes_{0}", Model.Id); @Html.Partial("_ProductAttributes", Model.ProductAttributes, dataDictAttributes) } <!--gift card--> @{ var dataDictGiftCard = new ViewDataDictionary(); dataDictGiftCard.TemplateInfo.HtmlFieldPrefix = string.Format("giftcard_{0}", Model.Id); @Html.Partial("_GiftCardInfo", Model.GiftCard, dataDictGiftCard) } <!--rental info--> @{ var dataDictRental = new ViewDataDictionary(); dataDictRental.TemplateInfo.HtmlFieldPrefix = string.Format("rental_{0}", Model.Id); @Html.Partial("_RentalInfo", Model, dataDictRental) } <!--price & add to cart--> @{ var dataDictPrice = new ViewDataDictionary(); dataDictPrice.TemplateInfo.HtmlFieldPrefix = string.Format("price_{0}", Model.Id); @Html.Partial("_ProductPrice", Model.ProductPrice, dataDictPrice) @Html.Partial("_ProductTierPrices", Model.TierPrices) var dataDictAddToCart = new ViewDataDictionary(); dataDictAddToCart.TemplateInfo.HtmlFieldPrefix = string.Format("addtocart_{0}", Model.Id); @Html.Partial("_AddToCart", Model.AddToCart, dataDictAddToCart) } <!--wishlist, compare, email a friend-->
@Html.Widget("productdetails_inside_overview_buttons_before", Model.Id) @{ var dataDictAddToWishlist = new ViewDataDictionary(); dataDictAddToWishlist.TemplateInfo.HtmlFieldPrefix = string.Format("addtocart_{0}", Model.Id); @Html.Partial("_AddToWishlist", Model.AddToCart, dataDictAddToWishlist) } @Html.Partial("_CompareProductsButton", Model) @Html.Partial("_ProductEmailAFriendButton", Model) @Html.Widget("productdetails_inside_overview_buttons_after", Model.Id)
@Html.Partial("_ShareButton", Model) @Html.Widget("productdetails_overview_bottom", Model.Id) </div> @if (!String.IsNullOrEmpty(Model.FullDescription)) {
@Html.Raw(Model.FullDescription)
} </div> @Html.Widget("productdetails_before_collateral", Model.Id)
@Html.Partial("_ProductSpecifications", Model.ProductSpecifications) @Html.Partial("_ProductTags", Model.ProductTags)
@Html.Action("ProductsAlsoPurchased", "Product", new { productId = Model.Id }) @Html.Action("RelatedProducts", "Product", new { productId = Model.Id }) </div> } @Html.Widget("productdetails_bottom", Model.Id) @Html.Partial("Picture") var galleryUploader = new qq.FineUploader({ element: document.getElementById("fine-uploader-gallery"), template: 'qq-template-gallery', request: { endpoint: '/server/uploads' }, thumbnails: { placeholders: { waitingPath: '/source/placeholders/waiting-generic.png', notAvailablePath: '/source/placeholders/not_available-generic.png' } }, validation: { allowedExtensions: ['jpeg', 'jpg', 'gif', 'png'] }, callbacks: { onError: function (id, name, errorReason, xhrOrXdr) { alert(qq.format("Error on file number {} - {}. Reason: {}", id, name, errorReason)); } }, cors: { //all requests are expected to be cross-domain requests expected: false, //if you want cookies to be sent along with the request sendCredentials: false }, xhrFields: { withCredentials: true } }); </div> </div>


Get this bounty!!!

#StackBounty: #c++ #visual-studio-2015 #linker #visual-studio-2005 #unresolved-external Link errors when trying to compile against an o…

Bounty: 50

I have an old project that is compiled in VS2005 (Sadly). It has to remain in VS2005 so it can link properly to another process which has the VS2005 CRT,MFC, etc.

Now I need to compile this project in VS2015, using the old VS2005 toolset.
I’ve changed the project’s VC++ directories to the old folders for all the STD and Windows SDK headers/libs (Include directories, Reference Directories, Library Directories, Source Directories).

This trick used to work fine while working with VS2010, but on VS2015 I’m getting some weird link errors:

1>Project1.obj : error LNK2019: unresolved external symbol "void __stdcall `eh vector destructor iterator'(void *,unsigned int,unsigned int,void (__thiscall*)(void *))" (??_M@YGXPAXIIP6EX0@Z@Z) referenced in function "public: virtual void * __thiscall PluginInterface::`vector deleting destructor'(unsigned int)" (??_EPluginInterface@@UAEPAXI@Z)
1>     1>
1>StdAfx.obj : error LNK2001: unresolved external symbol "void __stdcall `eh vector destructor iterator'(void *,unsigned int,unsigned int,void (__thiscall*)(void *))" (??_M@YGXPAXIIP6EX0@Z@Z)
1>     1>
1>Project1.obj : error LNK2019: unresolved external symbol "void __cdecl operator delete(void *,unsigned int)" (??3@YAXPAXI@Z) referenced in function __unwindfunclet$?getInstance@Project1@@SAPAV1@XZ$0
1>     1>
1>Project1.obj : error LNK2019: unresolved external symbol "void __cdecl operator delete[](void *,unsigned int)" (??_V@YAXPAXI@Z) referenced in function "public: virtual void * __thiscall PluginInterface::`vector deleting destructor'(unsigned int)" (??_EPluginInterface@@UAEPAXI@Z)

Why is it looking for this inner implementation of the deleter ? Should it be getting the implementation from the headers? Why would it work in VS2010 and not VS2015?

How can I fix this properly ?


Get this bounty!!!

#StackBounty: #c# #dapper Mapper made specifically to work with Dapper

Bounty: 200

I needed a mapper for a project I have using Dapper. I tried Automapper and other tools that comes with Dapper (Contrib, Rainbow, etc.), but I couldn’t make Automapper work properly and the rest doesn’t have what I need, so I did one.
Take in mind that I’m an amateur and not native english speaker 😉

The whole thing is in GitHub, I’ll write down here what I can, but it’s large, it won’t fit the code limit so there’s the link to the whole thing.
It works and in my tests it maps each object in about 2 milliseconds… it worry me, it’s too much good.
To be honest, given that I’m an amateur and after reading lots about reflection performance overhead, I was “mentally prepared” to lose performance for the sake of having it working the way I wanted and what not, so now I’m worried if I’m overlooking something and will appreciate any advice.

I’m sorry, I tried to summarize the post but I can’t summarize it any more, remember english is not my native language, and again the whole thing doesn’t fit here. If you want to help (thanks you) or it interests you, you can just take a peek at the Github link I provided earlier and come back to read what you need from the rest, if you need it.

Just to be sure, both in code and here I’ll refer as “Dapper’s dynamic result” to the result of the query Dapper returns. I don’t know if it’s really suitable, but it’s easier to me to name it that way.
Also, there’s a general diagram I did to have things clearer myself: in google drive.

The mapper works given some static dictionaries that stores information about the types that will be mapped, using a configuration the user have to make before map any object (at the app beginning… similar to Automapper I think).
I divided the different dictionaries in different usual public classes, so each class of the mapper can inherit only what’s needed, f.i.:

public class DMStatic_Dictionary : MQBStatic_IEnumerable
{
    public DMStatic_Dictionary(string[] customNamespaces = null) : base(customNamespaces)
    {
        InitStaticDDictionary();
    }

    #region fields
    protected static Dictionary<Type, Dictionary<string, string[]>> _Dictionaries;
    #endregion

    #region helpers
    private static void InitStaticDDictionary()
    {
        if (_Dictionaries == null)
        {
            lock (_LockObject)
            {
                if (_Dictionaries == null)
                    _Dictionaries = new Dictionary<Type, Dictionary<string, string[]>>();
            }
        }
    }
    #endregion
}

They store all the information, all the members, if each member is built-in, nested or a dictionary, prefixes and suffixes for searching later through the Dapper’s dynamic result, members to ignore, etc. Lets say they are just static dictionaries with the “reflection information” of the types (nested types, enumerables, dictionaries, etc.), so I don’t have to spend resources to retrieve that information at run time.


MAPPERCONFIG

As I said, the dictionaries should be filled at the very start of the app. To do that I made a fluent type class called MapperConfig.
This class do several things.

  • First it populates some static dictionaries through the public methods, f.i.:
    public MapperConfig AddNestedProperty<T>(bool isAnInterface, string memberName)
    {
        Type destination = typeof(T);
    
        if (_OnlyConstructor.Contains(destination))
            throw new CustomException_MapperConfig(
                @"MapperConfig.AddNestedProperty: You can't any configuration if MapOnlyConstructor have been setted.");
        if (_MembersCreators.ContainsKey(destination) && _MembersCreators[destination].ContainsKey(memberName))
            throw new CustomException_MapperConfig(
                $@"MapperConfig.AddNestedProperty: One member({memberName}) can not have a creator expression AND be setted as a nested type at same type");
        if (_Ignores.ContainsKey(destination) && _Ignores[destination].Contains(memberName))
            throw new CustomException_MapperConfig(
                $@"MapperConfig.AddNestedProperty: One member({memberName}) can not be setted as nested property if it was setted for ignore.");
    
        if (isAnInterface)
        {
            lock (_LockObject)
            {
                if (!_Interfaces.ContainsKey(destination))
                    _Interfaces.Add(destination, new List<string>() { memberName });
                else if (!_Interfaces[destination].Contains(memberName))
                    _Interfaces[destination].Add(memberName);
            }
        }
    
        lock (_LockObject)
        {
            if (!_NestedProperties.ContainsKey(destination))
            {
                _NestedProperties.Add(destination, new List<string>() { memberName });
                return this;
            }
            else
            {
                if (!_NestedProperties[destination].Contains(memberName))
                    _NestedProperties[destination].Add(memberName);
                return this;
            }
        }
    }
    

    This public methods are how one tell the mapper how to map each type member, if it’s a nested type (AddNestedProperty), if it’s a dictionary (AddDictionary), if it has to run some Func with the Dapper’s dynamic result to create the member (AddMemberCreator), if it has to use a constructor (AddConstructor, works via a Func again), etc.
    I wrote the ones I think I’ll need and others more general. Some notes about them:

    IEnumerable members are automated, the mapper will identify and map them automatically therefore there’s no need to configure them and no method in this class for them, but I didn’t find a way to automate dictionaries so I did a specific method to declare an object member as a dictionary to map it later: one need to pass the names that keys and values will have at the Dapper’s dynamic result, so the mapper can find them at runtime. If keys or values (or both) are custom, nested types, one have to configure the member as a nested property too:

    .AddNestedProperty<Finca>(false, "_Cuotas") 
    .AddDictionary<Finca>("_Cuotas", new string[2] { "cuotaId", "Cuota" })
    

    The mapper will find what’s nested (keys, values or both), map the built-in types using the name provided, ignore the name of the nested type and map the nested type recursively with a new mapper.

    Other note: as you probably have noted, a single member can be declared as various things, for instance (again) if a member is of type Dictionary<MyClass, int>, not only it can be declared as dictionary and nested, it MUST be declared as both of them or later the mapper will try to map keys as a built-in type and throw an exception.

  • Second it populates more static dictionaries with reflection. I didn’t want to search and retrieve all member of all objects-to-map at run time, so I store every member of the objects configured in the dictionaries:
    protected static Dictionary<Type, PropertyInfo[]> _QBPropertyInfos;
    protected static Dictionary<Type, FieldInfo[]> _QBFieldInfos;
    

    This is done in the method private void SetMembersInformation(Type t) of the MapperConfig class:

    private void SetMembersInformation(Type t)
    {
        if (_OnlyConstructor.Contains(t))
            RemoveAllConfigExceptConstructor(t);
    
        var pInfos = t.GetProperties(BindingFlags.Public | BindingFlags.Instance | BindingFlags.FlattenHierarchy)
            .Where(pInfo => pInfo.GetSetMethod() != null)
            .ToList();
        var fInfos = t.GetFields(BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.FlattenHierarchy)
            //http://stackoverflow.com/questions/40820102/reflection-returns-backing-fields-of-read-only-properties
            .Where(fInfo => fInfo.GetCustomAttribute<CompilerGeneratedAttribute>() == null)
            .ToList();
    
        //Get all inherited fields up the hierarchy => BindingFlags.FlattenHierarchy only works with public members
        bool inheritance = t.BaseType != null;
        Type inheritedT = t;
        Type baseT;
        while (inheritance)
        {
            //inherited fields
            baseT = inheritedT.BaseType;
            var baseFInfos = baseT.GetFields(BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.FlattenHierarchy)
                .Where(fInfo => fInfo.GetCustomAttribute<CompilerGeneratedAttribute>() == null)
                .ToList();
            fInfos = fInfos.Union(baseFInfos).ToList();
    
            //inherit mapper configurations
            MapperStore store = new MapperStore();
            if (store.GetMapper(baseT) != null)
                CopyConfigurations(baseT, t);
    
            inheritance = baseT.BaseType != null;
            inheritedT = baseT;
        }
    
        IEnumerable<MemberInfo> mInfos = pInfos.Union((IEnumerable<MemberInfo>)fInfos);
        Dictionary<MemberInfo, MemberTypeInfo> preMTInfos = (Dictionary<MemberInfo, MemberTypeInfo>)mInfos
            .Select(x =>
            {
                if (_Ignores.ContainsKey(t) && _Ignores[t].Contains(x.Name))
                    return new KeyValuePair<MemberInfo, MemberTypeInfo>(x, MemberTypeInfo.Ignore);
                else return new KeyValuePair<MemberInfo, MemberTypeInfo>(x, MemberTypeInfo.BuiltIn);
            })
            .ToDictionary(x => x.Key, x => x.Value);
    
        //prevent collection was modified exception
        Dictionary<MemberInfo, MemberTypeInfo> changes = new Dictionary<MemberInfo, MemberTypeInfo>(preMTInfos);
    
        IEnumerable<string> preNamesList = pInfos
            .Where(pInfo => preMTInfos[pInfo] != MemberTypeInfo.IEnumerable)
            .Select(pInfo => pInfo.Name)
            .Union(fInfos
                .Where(pInfo => preMTInfos[pInfo] != MemberTypeInfo.IEnumerable)
                .Select(fInfo => fInfo.Name));
    
        //Store all MemberTypeInfo
        //Trying to save iterations doing first if dictionary.contains(type)
        if (_MembersCreators.ContainsKey(t) && _NestedProperties.ContainsKey(t))
        {
            //Set members type dictionary
            foreach (KeyValuePair<MemberInfo, MemberTypeInfo> kvp in preMTInfos.Where(kvp => kvp.Value != MemberTypeInfo.Ignore))
            {
                if (_MembersCreators[t].ContainsKey(kvp.Key.Name))
                {
                    if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = MemberTypeInfo.Creator;//_preMTInfos[kvp.Key] = MemberTypeInfo.Creator;
                    RemoveMember(kvp.Key, pInfos, fInfos);
                }
                else
                {
                    if (_NestedProperties[t].Contains(kvp.Key.Name))
                    {
                        if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = MemberTypeInfo.Nested;//_preMTInfos[kvp.Key] = MemberTypeInfo.Nested;
                        RemoveMember(kvp.Key, pInfos, fInfos);
                    }
    
                    Type mType = GetMemberType(kvp.Key);
                    if (typeof(IEnumerable).IsAssignableFrom(mType) && !typeof(string).IsAssignableFrom(mType))
                    {
                        if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = changes[kvp.Key] | MemberTypeInfo.IEnumerable; //_preMTInfos[kvp.Key] = _preMTInfos[kvp.Key] | MemberTypeInfo.IEnumerable;
                        RemoveMember(kvp.Key, pInfos, fInfos);
                    }
                }
            }
        }
        else if (_MembersCreators.ContainsKey(t))
        {
            //Set members type dictionary
            foreach (KeyValuePair<MemberInfo, MemberTypeInfo> kvp in preMTInfos.Where(kvp => kvp.Value != MemberTypeInfo.Ignore))
            {
                if (_MembersCreators[t].ContainsKey(kvp.Key.Name))
                {
                    if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = MemberTypeInfo.Creator; //_preMTInfos[kvp.Key] = MemberTypeInfo.Creator;
                    RemoveMember(kvp.Key, pInfos, fInfos);
                }
                else
                {
                    Type mType = GetMemberType(kvp.Key);
                    if (typeof(IEnumerable).IsAssignableFrom(mType) && !typeof(string).IsAssignableFrom(mType))
                    {
                        if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = changes[kvp.Key] | MemberTypeInfo.IEnumerable; //_preMTInfos[kvp.Key] = _preMTInfos[kvp.Key] | MemberTypeInfo.IEnumerable;
                        RemoveMember(kvp.Key, pInfos, fInfos);
                    }
                }
            }
        }
        else if (_NestedProperties.ContainsKey(t))
        {
            //Add to members names list
            preNamesList = preNamesList.Union(_NestedProperties[t]);
    
            //Set members type dictionary
            foreach (KeyValuePair<MemberInfo, MemberTypeInfo> kvp in preMTInfos.Where(kvp => kvp.Value != MemberTypeInfo.Ignore))
            {
                if (_NestedProperties[t].Contains(kvp.Key.Name))
                {
                    if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = MemberTypeInfo.Nested; //_preMTInfos[kvp.Key] = MemberTypeInfo.Nested;
                    RemoveMember(kvp.Key, pInfos, fInfos);
                }
    
                Type mType = GetMemberType(kvp.Key);
                if (typeof(IEnumerable).IsAssignableFrom(mType) && !typeof(string).IsAssignableFrom(mType))
                {
                    if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = changes[kvp.Key] | MemberTypeInfo.IEnumerable; //_preMTInfos[kvp.Key] = _preMTInfos[kvp.Key] | MemberTypeInfo.IEnumerable;
                    RemoveMember(kvp.Key, pInfos, fInfos);
                }
            }
        }
        //built-in
        else if (!_OnlyConstructor.Contains(t))
        {
            //Set members type dictionary
            foreach (KeyValuePair<MemberInfo, MemberTypeInfo> kvp in preMTInfos.Where(kvp => kvp.Value != MemberTypeInfo.Ignore))
            {
                Type mType = GetMemberType(kvp.Key);
                if (typeof(IEnumerable).IsAssignableFrom(mType) && !typeof(string).IsAssignableFrom(mType))
                    changes[kvp.Key] = MemberTypeInfo.IEnumerable; //_preMTInfos[kvp.Key] = MemberTypeInfo.IEnumerable;
            }
        }
    
        if (_Interfaces.ContainsKey(t))
        {
            //Set members type dictionary
            foreach (KeyValuePair<MemberInfo, MemberTypeInfo> kvp in preMTInfos.Where(kvp => kvp.Value != MemberTypeInfo.Ignore))
            {
                if (_Interfaces[t].Contains(kvp.Key.Name))
                {
                    if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = changes[kvp.Key] | MemberTypeInfo.Interface;
                    RemoveMember(kvp.Key, pInfos, fInfos);
                }
    
                Type mType = GetMemberType(kvp.Key);
                if (typeof(IEnumerable).IsAssignableFrom(mType) && !typeof(string).IsAssignableFrom(mType))
                {
                    if (!_OnlyConstructor.Contains(t)) changes[kvp.Key] = changes[kvp.Key] | MemberTypeInfo.IEnumerable; //_preMTInfos[kvp.Key] = _preMTInfos[kvp.Key] | MemberTypeInfo.IEnumerable;
                    RemoveMember(kvp.Key, pInfos, fInfos);
                }
            }
        }
    
        //Lock-static dictionaries
        lock (_LockObject)
        {
            if (!_mtInfos.ContainsKey(t)) _mtInfos.Add(t, changes);
            else _mtInfos[t] = changes;
    
            if (!_NamesList.ContainsKey(t)) _NamesList.Add(t, preNamesList);
            else _NamesList[t] = preNamesList;
    
            if (!_QBPropertyInfos.ContainsKey(t)) _QBPropertyInfos.Add(t, pInfos.ToArray());
            else _QBPropertyInfos[t] = pInfos.ToArray();
    
            if (!_QBFieldInfos.ContainsKey(t)) _QBFieldInfos.Add(t, fInfos.ToArray());
            else _QBFieldInfos[t] = fInfos.ToArray();
    
            string columns = "";
            var orderedMembersNames = _QBPropertyInfos[t].Select(x => x.Name)
                .Union(_QBFieldInfos[t].Select(x => QBuilder.StringSQLBuilder.RemoveFieldsUnderscore(x.Name)))
                .OrderBy(x => x);
            columns = string.Join(",", orderedMembersNames);
    
            if (!_Columns.ContainsKey(t)) _Columns.Add(t, columns);
            else _Columns[t] = columns;
        }
    }
    

    It iterates through all members of the given type (and parent classes), storing them, determining who are an IEnumerable and filling another important dictionary:

    protected static Dictionary<Type, Dictionary<MemberInfo, MemberTypeInfo>> _mtInfos;
    

    Which is just a fast way to determine at runtime what have to do the mapper.
    MemberTypeInfo is an Enum:

    [Flags]
    public enum MemberTypeInfo { BuiltIn = 1, Nested = 2, Creator = 4, IEnumerable = 8, Dictionary = 16, Interface = 32, Ignore = 64 }
    

    So, at runtime the mapper looks at this dictionary for every member, and does whatever it has to do, e.g. if the member is a nested type it calls another mapper recursively to map the member, and if it’s built-in type it just search the value in the given Dapper dynamic result by the member’s name and sets it.
    The configurations of the parent classes are inherited too through the private void CopyConfigurations(Type baseT, Type nestedT) method, so one can configure a parent class completely (even an abstract class), and then configure all the child class without having to repeat the parent’s things, since all child will automatically inherit parent configurations.

There’re some obvious configurations like AddNestedProperty or IgnoreProperty, and others not so obvious like AddInterfaceToObjectCondition but I think that the methods comments are enough to understand what they all do.
If that’s not the case I’ll explain anything, just ask it.

Just to note a pair of rules here (or flaws…):

  1. The method EndConfig() must be called for every type that will have to be mapped, even nested types, interfaces, structs, everything. Even if the type have only built in types and needs no other configuration. The mapper only “see” what’s configured, if EndConfig() doesn’t get called for a type, that type doesn’t exist for the mapper.
  2. The same method must be called after all configurations for that type have been called. Given that it’s fluent type, the last method for a type configuration should be always EndConfig(). Take in mind that the method SetMembersInformation compares members with what the user stored in dictionaries, so if that method is called various times, I have no idea what could happen, it’s not prepared for that.
  3. I wouldn’t make configurations at run time. The system haven’t been made for that, I haven’t tested it and I have no ides what would happen. Maybe it’d work flawlessly, or maybe not.

A pair of example of some configurations of my own project:

//A pair of easy ones
            mConfig
                .AddNestedProperty<Apunte>(false, "_Asiento")
                .AddMemberCreator<Apunte>("_DebeHaber", x => (DebitCredit)x.DebeHaber)
                .AddPrefixes<Apunte>(new string[] { "apu" })
                .EndConfig<Apunte>();
            mConfig
                .AddConstructor<ApunteDLO>(x => new ApunteDLO(x.Id, x.IdOwnerComunidad, x.OrdenEnAsiento, x.Asiento, x.Concepto, x.DebeHaber,
                    x.Importe, x.IdCuenta, x.Punteo, x.Factura))
                .MapOnlyConstructor<ApunteDLO>()
                .EndConfig<ApunteDLO>();

...

//More complicated
            mConfig
                .AddConstructor<Finca>(x => new Finca(x.Id, x.IdOwnerComunidad, x.Baja, x.Nombre, x.Coeficiente, x.Codigo, this.ACData))
                .AddNestedProperty<Finca>(false, "_PropietarioActual")
                .AddDictionary<Finca>("_HistoricoPropietarios", new string[2] { "FechaFinal", "IdPropietario" })
                .AddNestedProperty<Finca>(false, "_Cuotas")
                .AddDictionary<Finca>("_Cuotas", new string[2] { "cuotaId", "Cuota" })
                .AddNestedProperty<Finca>(false, "_EntregasACuenta")
                .AddNestedProperty<Finca>(false, "_Devoluciones")
                .AddNestedProperty<Finca, DireccionPostal>(false, x => x.Direccion)
                .AddMemberCreator<Finca, DireccionPostal>(x => x.Direccion2,
                    x => new DireccionPostal(x.TipoVia2, x.Direccion2, x.CP2, x.Localidad2, x.Provincia2))
                .AddMemberCreator<Finca, TipoPagoCuotas>(x => x.DefaultTipoPagoCuotas, x => (TipoPagoCuotas)x.DefaultTipoPagoCuotas)
                .AddMemberCreator<Finca, sTelefono>(x => x.Telefono1, x => new sTelefono(x.Telefono, x.TipoTelefono))
                .AddMemberCreator<Finca, sTelefono>(x => x.Telefono2, x => new sTelefono(x.Telefono2, x.TipoTelefono2))
                .AddMemberCreator<Finca, sTelefono>(x => x.Telefono3, x => new sTelefono(x.Telefono3, x.TipoTelefono3))
                .AddMemberCreator<Finca, sTelefono>(x => x.Fax, x => new sTelefono(x.Fax, TipoTelefono.Fax))
                .AddMemberCreator<Finca, int[]>(x=>x.IdAsociadas, x=>
                {
                    IEnumerable<dynamic> ex = (IEnumerable<dynamic>)x;
                    return ex
                        .Select(dyn => dyn.IdFincaAsociada)
                        .Distinct()
                        .ToArray();
                })
                .AddMemberCreator<Finca>("_IdCopropietarios", x =>
                {
                    IEnumerable<dynamic> ex = (IEnumerable<dynamic>)x;
                    return ex
                        .Select(dyn => dyn.IdPersonaCoProp)
                        .Distinct()
                        .ToArray();
                })
                .AddMemberCreator<Finca>("_IdPagadores", x =>
                {
                    IEnumerable<dynamic> ex = (IEnumerable<dynamic>)x;
                    return ex
                        .Select(dyn => new Tuple<int, TipoPagoCuotas>(dyn.IdPersonaCoPag, (TipoPagoCuotas)dyn.TipoPagoCuotas))
                        .Distinct()
                        .ToArray();
                })
                .EndConfig<Finca>();

MAPPERSTORE

I thought it would be a waste to create mappers all around the app, so I just created a little class called MapperStore to store and retrieve mappers already created, using a static dictionary to store mappers as interfaces.

/// <summary>
/// Use this to get DapperMapper objects.
/// </summary>
public class MapperStore : DMStatic_Store
{
    /// <summary>
    /// Store t as a type that can be, and has been configured for mapping by DapperMapper. If you are configuring a type, use 
    /// MapperConfig.EndCongfig<type>() instead.
    /// Don't use it before configuring the mapper first.
    /// </summary>
    /// <param name="t"></param>
    public void StoreType(Type t)
    {
        if (!_TypesToMap.Contains(t))
            _TypesToMap.Add(t);
    }
    /// <summary>
    /// Store a mapper object in static dictionary.
    /// </summary>
    /// <param name="t"></param>
    /// <param name="mapper"></param>
    public void StoreMapper(Type t, iDapperMapper mapper)
    {
        if (!_Mappers.ContainsKey(t)) _Mappers.Add(t, mapper);
    }
    /// <summary>
    /// Get mapper of type t. If type t hasn't been stored by StoreType, returns null. If type t has been stored and there are no mappers
    /// created yet, it creates a new one, stores it, and return it.
    /// </summary>
    /// <param name="t"></param>
    /// <returns></returns>
    public iDapperMapper GetMapper(Type t)
    {
        if (_Mappers.ContainsKey(t))
            return _Mappers[t];

        if (!_TypesToMap.Contains(t))
            return null;

        iDapperMapper mapper = (iDapperMapper)Activator.CreateInstance(typeof(DapperMapper<>).MakeGenericType(t), this);
        StoreMapper(t, mapper);
        return mapper;
    }
    /// <summary>
    /// Returns true if a mapper exists or can be created, and set it as iDapperMapper.
    /// If type t hasn't been stored by StoreType, returns false. If type t has been stored and there are no mappers
    /// created yet, it creates a new one, stores it, set it as iDapperMapper and returns true.
    /// </summary>
    /// <param name="t"></param>
    /// <param name="mapper"></param>
    /// <returns></returns>
    public bool GetMapper(Type t, out iDapperMapper mapper)
    {
        if (_Mappers.ContainsKey(t))
        {
            mapper = _Mappers[t];
            return true;
        }

        if (!_TypesToMap.Contains(t))
        {
            mapper = null;
            return false;
        }

        mapper = (iDapperMapper)Activator.CreateInstance(typeof(DapperMapper<>).MakeGenericType(t), this);
        StoreMapper(t, mapper);
        return false;
    }
    /// <summary>
    /// Remove mapper previously stored.
    /// </summary>
    /// <param name="t"></param>
    public void RemoveMapper(Type t)
    {
        if (_Mappers.ContainsKey(t))
            _Mappers.Remove(t);
    }
}

And the expected way to get a mapper:

MapperStore store = new MapperStore();
DapperMapper<Foo> mapper = (DapperMapper<Foo>)store.GetMapper(typeof(Foo));

DAPPERMAPPER

And here’s the thing finally, it’s the class that actually map the objects, usually through this method:

        public T Map(IEnumerable<dynamic> dapperResult, bool cleanResult = false)
        {
            var parser = new PrePostFixesParser(this);
            T mapped = this.NewObject(dapperResult.First());
            if (_OnlyConstructor.Contains(this.TType)) return mapped;

            //TODO: divide el siguiente foreach en dos con dos nuevos diccionarios estáticos, uno para pInfos y otro para fInfos, 
            //aunque se repita código: hacer métodos para cada parte del código del tipo:
            //private T PreMapCreator(KeyValuePair<PropertyInfo, MemberTypeInfo> kvp, IEnumerable<dynamic> dapperResult, bool cleanResult = false)
            //private T PreMapIEnumerable(KeyValuePair<PropertyInfo, MemberTypeInfo> kvp, IEnumerable<dynamic> dapperResult, bool cleanResult = false)
            //...

            //Loop through all members
            foreach (KeyValuePair<MemberInfo, MemberTypeInfo> kvp in mtInfos)
            {
                if (kvp.Value == MemberTypeInfo.Ignore)
                    continue;
                //Member have a creator
                else if ((kvp.Value & MemberTypeInfo.Creator) == MemberTypeInfo.Creator)
                {
                    //MemberDelegate mDel = (MemberDelegate)_MembersCreators[this.TType][kvp.Key.Name];
                    Func<dynamic, object> mDel = (Func<dynamic, object>)_MembersCreators[this.TType][kvp.Key.Name];

                    if (kvp.Key.MemberType == MemberTypes.Property) ((PropertyInfo)kvp.Key).SetValue(mapped, mDel(dapperResult));
                    else ((FieldInfo)kvp.Key).SetValue(mapped, mDel(dapperResult));
                }
                //Member is IDictionary or IEnumerable
                else if ((kvp.Value & MemberTypeInfo.IEnumerable) == MemberTypeInfo.IEnumerable)
                {
                    Type t = GetMemberType(kvp.Key);
                    //if ((kvp.Value & MemberTypeInfo.Interface) == MemberTypeInfo.Interface) t = ResolveInterface(kvp.Key, dapperResult);
                    //else t = GetMemberType(kvp.Key);
                    /*
                    {
                        //Type of property or field
                        if (kvp.Key.MemberType == MemberTypes.Property) t = ((PropertyInfo)kvp.Key).PropertyType;
                        else t = ((FieldInfo)kvp.Key).FieldType;
                    }*/
                    bool isAnInterface = (kvp.Value & MemberTypeInfo.Interface) == MemberTypeInfo.Interface;
                    bool isNested = (kvp.Value & MemberTypeInfo.Nested) == MemberTypeInfo.Nested;

                    //If member is a dictionary
                    if (typeof(IDictionary).IsAssignableFrom(t))
                    {
                        //Create a dummy dictionary with the dapper's dynamic result which should be equal to the final one
                        DictionaryMapper dictMapper = new DictionaryMapper(dapperResult, kvp.Key.Name, isNested, isAnInterface, cleanResult, t, this);

                        try
                        {
                            if (kvp.Key.MemberType == MemberTypes.Property) ((PropertyInfo)kvp.Key).SetValue(mapped, dictMapper.DummyDictionary);
                            else ((FieldInfo)kvp.Key).SetValue(mapped, dictMapper.DummyDictionary);
                        }
                        catch (Exception err)
                        {
                            throw new CustomException_DapperMapper(
                                $@"DapperMapper.Map: Couldn't map IDictionary member {kvp.Key.Name} with value contained by dynamic object.
Incorrect type of value?: {kvp.Value.ToString()}",
                                err);
                        }
                    }
                    //Rest of enumerables
                    else
                    {
                        IEnumerable<dynamic> iEnumDapperResult;
                        //Select current member's values from dynamic
                        if (isNested && !cleanResult)
                        {
                            //Type mType = t; // GetMemberType(kvp.Key);//IEnumerable<T>
                            Type genericType = t.GenericTypeArguments[0];//mType.GenericTypeArguments[0];//T
                            if ((kvp.Value & MemberTypeInfo.Interface) == MemberTypeInfo.Interface)
                            {
                                bool genericIsInterfaceNotIEnumerable =
                                    genericType.IsInterface &&
                                    !typeof(IDictionary).IsAssignableFrom(genericType) &&
                                    !(typeof(IEnumerable).IsAssignableFrom(genericType) && !typeof(string).IsAssignableFrom(genericType));

                                if (genericIsInterfaceNotIEnumerable) genericType = ResolveInterface(genericType, dapperResult);
                            }

                            iDapperMapper nestedMapper = MappersStore.GetMapper(genericType);
                            var nestedParser = new PrePostFixesParser(nestedMapper);

                            iEnumDapperResult = dapperResult
                                .Select(dyn => nestedParser.GetTypeMembersWithoutPrePostFixes(dyn, nestedMapper.NamesList));
                        }
                        else if (!cleanResult) iEnumDapperResult = dapperResult.Select(dyn => parser.RemovePrePostFixesFromDictionary(dyn));
                        else iEnumDapperResult = dapperResult;

                        //Create dummy IEnumerable
                        EnumerableMapper enumMapper = new EnumerableMapper(iEnumDapperResult, kvp.Key.Name, isNested, t, this.TType); ;
                        var dummy = Activator.CreateInstance(t, enumMapper.DummyEnumerable);

                        try
                        {
                            if (kvp.Key.MemberType == MemberTypes.Property) ((PropertyInfo)kvp.Key).SetValue(mapped, dummy);
                            else ((FieldInfo)kvp.Key).SetValue(mapped, dummy);
                        }
                        catch (Exception err)
                        {
                            throw new CustomException_DapperMapper(
                                $@"DapperMapper.Map: Couldn't map IEnumerable member {kvp.Key.Name} with value contained by dynamic object.
Incorrect type of value?: {kvp.Value.ToString()}",
                                err);
                        }
                    }
                }//End IDictionary/IEnumerable
                //If Built-in
                else if ((kvp.Value & MemberTypeInfo.BuiltIn) == MemberTypeInfo.BuiltIn)
                {
                    string name = parser.RemoveFieldsUnderscore(kvp.Key.Name);
                    IDictionary<string, object> dapperDict;
                    if (!cleanResult)
                        dapperDict = parser.GetTypeMembersWithoutPrePostFixes(dapperResult.First(), NamesList) as IDictionary<string, object>;
                    else
                        dapperDict = dapperResult.First() as IDictionary<string, object>;

                    if (!dapperDict.ContainsKey(name))
                        throw new CustomException_DapperMapper(
                            $@"DapperMapper.Map: There's no member in dynamic dapper result with name {kvp.Key.Name}. Cannot Map object.");

                    try
                    {
                        if (kvp.Key.MemberType == MemberTypes.Property) ((PropertyInfo)kvp.Key).SetValue(mapped, dapperDict[name]);
                        else ((FieldInfo)kvp.Key).SetValue(mapped, dapperDict[name]);
                    }
                    catch (Exception err)
                    {
                        throw new CustomException_DapperMapper(
                            $@"DapperMapper.Map: Couldn't map BuiltIn-type member {kvp.Key.Name} with value contained by dynamic object.
Incorrect type of value?: {kvp.Value.ToString()}",
                            err);
                    }
                }
                //if nested
                else if ((kvp.Value & MemberTypeInfo.Nested) == MemberTypeInfo.Nested)
                {
                    Type mType = GetMemberType(kvp.Key);

                    if ((kvp.Value & MemberTypeInfo.Interface) == MemberTypeInfo.Interface)
                        mType = ResolveInterface(mType, dapperResult);

                    //access generic Map method through nongeneric interface method
                    iDapperMapper nestedMapper = MappersStore.GetMapper(mType);

                    if (nestedMapper == null)
                        throw new CustomException_DapperMapper(
                            $@"DapperMapper.Map: No Mapper found at store for property {kvp.Key.Name} of type {mType.ToString()}.
If you want to map a nested property you have to create a mapper for that property type.");

                    if (kvp.Key.MemberType == MemberTypes.Property)
                        ((PropertyInfo)kvp.Key).SetValue(mapped, nestedMapper.NoGenericMap(dapperResult, cleanResult));
                    else ((FieldInfo)kvp.Key).SetValue(mapped, nestedMapper.NoGenericMap(dapperResult, cleanResult));
                }
            }

            return mapped;
        }

The idea is to use:

MapperStore store = new MapperStore();
DapperMapper<Persons> mapper = (DapperMapper<Persons>)store.GetMapper(typeof(Persons));
Persons p = mapper.Map(dynamicDapperResult);

Due to it being generic and that the method has to recursively call other DapperMapper.Map methods, I use a non-generic interface to call the mappers while mapping at run time, using the NoGenericMap methods.
The generic DapperMapper<T> class actually maps the objects and is called at runtime. It uses the dictionaries to know if a member is nested or built-in, if it’s an interface, if it’s an IEnumerable or IDictionary, if it have to use a Func to create a member, etc. It actually does very little with reflection, the minimum I could manage to achieve.
Basically it first create an instance of the object (via reflection or with the configured constructor), then loops through all class members, search in the dictionaries for adequate type and member using the MemberTypeInfo Enum I spoke about above and fill the member according to what was configured following the priority showed in the diagram. If a member is a nested type, just create a new mapper and again and again.
IEnumerableand IDictionary members have a their own classes to map them, EnumerableMapper and DictionaryMapper. They simply create an instance of the adequate object and search the data to map all the collection members through the dynamic Dapper’s result, which will call again a normal DapperMapper if they are configured as nested types.

There’s an overloaded method to map enumerables:

public R Map<R>(IEnumerable<dynamic> dapperResult, string splitOn = "Id", bool cleanResult = false)
        where R : IEnumerable<T>

One must call this method if want to map a whole IEnumerable of objects:

List<Persons> myList = mapper.Map<List<Persons>>(dapperResult, "Id");

I didn’t find other way to do this except with an overloaded method.

In general the Dapper result dynamic members must have the same name as the object to be mapped members. I wrote the code to support prefixes and sufixes, that can be added via MapperConfig, and a parser to “decode” them at runtime.
The parser:

public class PrePostFixesParser
{
    public PrePostFixesParser(iDapperMapper mapper)
    {
        Tuple<string[], bool> prefixes = mapper != null ? mapper.Prefixes : null;
        Tuple<string[], bool> postfixes = mapper != null ? mapper.Postfixes : null;

        if (prefixes != null)
        {
            this.Prefixes = prefixes.Item1;
            this.PrefixesExclusive = prefixes.Item2;
            this.PrefixesCount = prefixes.Item1.Select(x => x.Count()).Distinct();
        }
        else
        {
            this.Prefixes = null;
            this.PrefixesExclusive = false;
            this.PrefixesCount = new int[] { 0 };
        }

        if (postfixes != null)
        {
            this.Postfixes = postfixes.Item1;
            this.PostfixesExclusive = postfixes.Item2;
            this.PostfixesCount = postfixes.Item1.Select(x => x.Count()).Distinct();
        }
    }

    #region properties
    public string[] Prefixes { get; private set; }
    public string[] Postfixes { get; private set; }
    public bool PrefixesExclusive { get; private set; }
    public bool PostfixesExclusive { get; private set; }
    public IEnumerable<int> PrefixesCount { get; private set; }
    public IEnumerable<int> PostfixesCount { get; private set; }
    #endregion

    #region helpers
    private bool RemovePrefixIfStringContains(ref string str)
    {
        if (Prefixes == null)
            return false;

        bool contain = false;
        foreach (int count in PrefixesCount)
        {
            //contains some of the configurated prefixes
            contain = Prefixes.Contains(str.Substring(0, count));
            if (contain)
            {
                str = str.Remove(0, count);
                return contain;
            }
        }

        return contain;
    }
    private bool RemovePostfixIfStringContains(ref string str)
    {
        if (Postfixes == null)//(PostfixesCount.Count() == 1 && PostfixesCount.First() == 0)
            return false;

        bool contain = false;
        foreach (int count in PostfixesCount)
        {
            contain = Postfixes.Contains(str.Substring(str.Length - count, count));
            if (contain)
            {
                str = str.Remove(str.Length - count, count);
                return contain;
            }
        }
        return contain;
    }
    private bool KeysContainsSomePrePostfix(IDictionary<string, object> dict)
    {
        if (Prefixes != null && Postfixes != null)
        {
            return dict.Any(kvp =>
            {
                foreach (int count in PrefixesCount)
                {
                    //contains some of the configurated prefixes or postfixes
                    if (Prefixes.Contains(kvp.Key.Substring(0, count))
                        || Postfixes.Contains(kvp.Key.Substring(kvp.Key.Length - count, count)))
                        return true;
                    else return false;
                }
                return false;
            });
        }
        else if (Prefixes != null)
        {
            return dict.Any(kvp =>
            {
                foreach (int count in PrefixesCount)
                {
                    //contains some of the configurated prefixes
                    if (Prefixes.Contains(kvp.Key.Substring(0, count)))
                        return true;
                    else return false;
                }
                return false;
            });
        }
        else if (Postfixes != null)
        {
            return dict.Any(kvp =>
            {
                foreach (int count in PrefixesCount)
                {
                    //contains some of the configurated postfixes
                    if (Postfixes.Contains(kvp.Key.Substring(kvp.Key.Length - count, count)))
                        return true;
                    else return false;
                }
                return false;
            });
        }
        return false;
        //return dict.Any(kvp => (Prefixes.Contains(kvp.Key.Substring(0,)
    }
    #endregion

    #region public methods
    public string RemoveFieldsUnderscore(string str)
    {
        if (str.StartsWith("_")) return str.Remove(0, 1);
        return str;
    }
    public IEnumerable<string> GetCleanNamesList(IEnumerable<string> namesList)
    {
        return namesList.Select(str => RemoveFieldsUnderscore(str));
    }
    public IEnumerable<IDictionary<string, object>> RemovePrePostFixesFromDictionary(IEnumerable<dynamic> dyn)
    {
        List<Dictionary<string, object>> ienum = new List<Dictionary<string, object>>();
        foreach (dynamic d in dyn)
        {
            Dictionary<string, object> dict = new Dictionary<string, object>();
            foreach (KeyValuePair<string, object> kvp in (d as IDictionary<string, object>))
            {
                string keyWithoutPrePostfixes = kvp.Key;
                if (!RemovePrefixIfStringContains(ref keyWithoutPrePostfixes) && !RemovePostfixIfStringContains(ref keyWithoutPrePostfixes))
                    dict.Add(kvp.Key, kvp.Value);
                else
                    dict.Add(keyWithoutPrePostfixes, kvp.Value);//return new KeyValuePair<string, object>(keyWithoutPrePostfixes, kvp.Value);
            }
            ienum.Add(dict);
        }

        return ienum;
    }
    public dynamic GetTypeMembersWithoutPrePostFixes(dynamic dyn, IEnumerable<string> namesList)
    {
        var typeMembers = new ExpandoObject() as IDictionary<string, object>;
        IDictionary<string, object> dict = dyn as IDictionary<string, object>;
        IEnumerable<string> names = namesList.Select(str => RemoveFieldsUnderscore(str));

        int i = 0;
        if (!PrefixesExclusive && !PostfixesExclusive)
        {
            foreach (KeyValuePair<string, object> kvp in dict)
            {
                string keyWithoutPrePostfixes = kvp.Key;
                bool namesContainsKey = names.Contains(kvp.Key);

                if (!RemovePrefixIfStringContains(ref keyWithoutPrePostfixes)
                    && !RemovePostfixIfStringContains(ref keyWithoutPrePostfixes)
                    && namesContainsKey)
                {
                    typeMembers.Add(kvp.Key, kvp.Value);
                    i++;
                }
                else if (names.Contains(keyWithoutPrePostfixes))
                {
                    typeMembers.Add(keyWithoutPrePostfixes, kvp.Value);
                    i++;
                }
            }
        }
        else if (PrefixesExclusive && PostfixesExclusive)
        {
            foreach (KeyValuePair<string, object> kvp in dict)
            {
                string keyWithoutPrePostfixes = kvp.Key;
                if (RemovePrefixIfStringContains(ref keyWithoutPrePostfixes)
                    && RemovePostfixIfStringContains(ref keyWithoutPrePostfixes)
                    && names.Contains(keyWithoutPrePostfixes))
                    typeMembers.Add(keyWithoutPrePostfixes, kvp.Value);
            }
        }
        else if (PrefixesExclusive)
        {
            foreach (KeyValuePair<string, object> kvp in dict)
            {
                string keyWithoutPrePostfixes = kvp.Key;
                if (RemovePrefixIfStringContains(ref keyWithoutPrePostfixes))
                {
                    RemovePostfixIfStringContains(ref keyWithoutPrePostfixes);

                    if (names.Contains(keyWithoutPrePostfixes))
                        typeMembers.Add(keyWithoutPrePostfixes, kvp.Value);
                }
            }
        }
        else
        {
            foreach (KeyValuePair<string, object> kvp in dict)
            {
                string keyWithoutPrePostfixes = kvp.Key;
                if (RemovePostfixIfStringContains(ref keyWithoutPrePostfixes))
                {
                    RemovePrefixIfStringContains(ref keyWithoutPrePostfixes);

                    if (names.Contains(keyWithoutPrePostfixes))
                        typeMembers.Add(keyWithoutPrePostfixes, kvp.Value);
                }
            }
        }

        return typeMembers;
    }
    #endregion
}

And that’s all, I think. If you are still reading this I believe I have to thank you and congratulate you 😛
Besides that, I’ll just post the test I did that I mentioned at the beginning (and the results with a simple object and with an object with dictionary members), and that will be all:

        public static void MapperTest<T>(int testsTotal, int desde, int cada, int idAlias = 0, bool deleteFirst = true)
        {
            Type t = typeof(T);
            string filepath = @"E:TESTMAPPER." + t.Name + ".xlsx";
            if (deleteFirst && File.Exists(filepath))
                File.Delete(filepath);

            var file = new FileInfo(filepath);
            var package = new ExcelPackage(file);
            ExcelWorksheet wSheet = package.Workbook.Worksheets.Add("prueba1");

            wSheet.Cells[1, 1].Value = $"MAPPERTEST:  MAP {t.Name} ; Tests: { testsTotal.ToString()}. Desde: { desde.ToString()}. Cada: { cada.ToString()}.";
            wSheet.Cells[2, 1].Value = "ID";
            wSheet.Cells[2, 2].Value = "TYPE";
            wSheet.Cells[2, 3].Value = "OBJECTS CREATED";
            wSheet.Cells[2, 4].Value = "SECONDS";
            wSheet.Cells[2, 5].Value = "MILISECONDS";
            wSheet.Cells[2, 6].Value = "SECONDS/OBJECTS";

            Console.WriteLine("MAPPERTEST:");
            Console.WriteLine($@"    1.- MAP {t.Name}
        tests: {testsTotal.ToString()}.
        desde: {desde.ToString()}.
        cada: {cada.ToString()}.");
            Console.WriteLine();
            Console.WriteLine("    -WAITING-");
            Console.ReadLine();
            Console.WriteLine("------------------");
            Console.WriteLine();

            for (int j = desde, x = 0; j < (testsTotal * cada); j += cada, x++)
            {
                Console.WriteLine("...");
                Console.WriteLine($"    Test Nº {x.ToString()}.");
                Console.WriteLine($"    Nº objetos a crear: {j.ToString()}");
                Console.WriteLine();
                Console.WriteLine("...");

                using (SqlConnection con = new SqlConnection(conString))//(MySqlConnection con = new MySqlConnection(conString))
                {
                    con.Open();
                    var result = con.Query(@"SELECT p.Id,p.Nombre,pif.pruebaI,pif.otro,a.Id aId,a.Acc aAcc,a.Persona aPersona 
FROM Persons p 
RIGHT JOIN Acc a ON p.Id=a.Persona 
RIGHT JOIN pruebaif pif ON p.Id=pif.Persona
WHERE p.Id=1");

                    Stopwatch watch = new Stopwatch();
                    watch.Reset();
                    watch.Start();

                    for (int i = 0; i < j; i++)
                    {
                        T p;
                        MapperStore store = new MapperStore();
                        DapperMapper<T> mapper = (DapperMapper<T>)store.GetMapper(t);
                        p = mapper.Map(result);
                    }

                    watch.Stop();
                    Console.WriteLine($"    +++++++++TEST {t.Name} - {x.ToString()} DONE+++++++++");
                    Console.WriteLine();
                    Console.WriteLine($"    TIME WATCH: {watch.Elapsed.ToString()}");
                    Console.WriteLine($"    TIME WATCH SECONDS: {watch.Elapsed.Seconds.ToString()}");
                    Console.WriteLine($"    TIME WATCH MILISECONDS: {watch.ElapsedMilliseconds.ToString()}");
                    Console.WriteLine();

                    wSheet.Cells[x + 2, 1].Value = x + idAlias;
                    wSheet.Cells[x + 2, 2].Value = t.Name;
                    wSheet.Cells[x + 2, 3].Value = j;
                    wSheet.Cells[x + 2, 4].Value = watch.Elapsed.Seconds;
                    wSheet.Cells[x + 2, 5].Value = watch.ElapsedMilliseconds;
                    wSheet.Cells[x + 2, 6].Formula = "D3/C3";

                    con.Close();
                }
                Console.WriteLine("------------------");
                Console.WriteLine("DB OK");
                Console.WriteLine();
            }

            package.Save();
            Console.WriteLine("    +++++++++ALL TESTS DONE+++++++++");
            Console.WriteLine();
            Console.ReadLine();
        }

PS.: There’s a SQL query builder in the same Github link that I made for the same project too, but that’s another story, it’s just string concatenations using some of the mapper dictionaries, just ignore it.

Edit:
Asked by @t3chb0t in the comments I’m putting an example of an object to be mapped:

public class Apunte : IObjModelBase, IOwnerComunidad, IObjWithDLO<ApunteDLO>
{
    public Apunte() { }
    public Apunte(int id, int idComunidad, Asiento asiento, string FacturaId = null)
    {
        this._Id = id;
        this._IdOwnerComunidad = idComunidad;
        this._Asiento = asiento;
        this._Factura = FacturaId;
    }

    #region fields
    private int _Id;
    private int _IdOwnerComunidad;
#pragma warning disable CS0649
    private int _OrdenEnAsiento;
#pragma warning restore CS0649
    private Asiento _Asiento;
    private DebitCredit _DebeHaber;
    private decimal _Importe;
    private string _Factura;
    #endregion

    #region properties
    public int Id { get { return this._Id; } }        
    public int IdOwnerComunidad { get { return this._IdOwnerComunidad; } }
    public int OrdenEnAsiento { get { return this._OrdenEnAsiento; } }
    public Asiento Asiento { get { return this._Asiento; } }
    public string Concepto { get; set; }        
    public DebitCredit DebeHaber
    {
        get { return this._DebeHaber; }
        set
        {
            if (this._DebeHaber != value)
            {                    
                if(!_Asiento.Abierto)
                    throw new CustomException_ObjModels(
                        $"Error cambiando DebeHaber de apunte numero {Id} de asiento numero {Asiento.Id}. Asiento cerrado.");
                this._Asiento.CambiaSaldo(this, value);
                this._DebeHaber = value;
            }
        }
    }        
    public decimal Importe
    {
        get { return this._Importe; }
        set
        {
            if (this._Importe != value)
            {
                if (!_Asiento.Abierto)
                    throw new CustomException_ObjModels(
                        $"Error cambiando DebeHaber de apunte numero {Id} de asiento numero {Asiento.Id}. Asiento cerrado.");
                this._Asiento.CambiaSaldo(this, value);
                this._Importe = value;
            }
        }
    }
    public CuentaMayor Cuenta { get; set; }
    public bool Punteo { get; set; }        
    public string Factura { get { return this._Factura; } }
    #endregion

    #region DLO
    public ApunteDLO GetDLO()
    {
        return new ApunteDLO(Id, IdOwnerComunidad, OrdenEnAsiento, Asiento.Codigo.CurrentCodigo, Concepto, DebeHaber, Importe, Cuenta.NumCuenta.ToString(),
            Punteo, Factura);
    }
    #endregion
}

It’s a model I use in the same project I did the mapper for. The database have all the plain data, at the beginning of the APP I configure the type (if you note it, it’s the same object I used as an example for the configuration), and the repository map the objects through this mapper when needed, again as I described before in the question.


Get this bounty!!!