#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $vec{y}$) and keep another set free (say $vec{x}$), the system becomes an under-determined, dense, and linear system of the subset of variables ($A(vec{y})vec{x} = vec{b}(vec{y})$, $A(vec{y})$ is a dense matrix, and $vec{b}(vec{y})$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $vec{x}$ and keep $vec{y}$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($F(vec{x}, vec{y}) = 0$). The Jacobian $J(vec{x}, vec{y})$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $vec{y}$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

  1. Initialize $vec{x} = vec{x}_0$, and $vec{y} = vec{y}_0$
  2. Fix $vec{y}_{n – 1}$, solve $A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$ for one of all the possible $vec{x}_{n}$ because this system is under-determined.
  3. Fix $vec{x}_{n – 1}$. Perform one iteration of Newton’s method for solving $F(vec{x}_{n-1}, vec{y}_{n}) = 0$ for $vec{y}_{n}$.
  4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf


Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $vec{y}$) and keep another set free (say $vec{x}$), the system becomes an under-determined, dense, and linear system of the subset of variables ($A(vec{y})vec{x} = vec{b}(vec{y})$, $A(vec{y})$ is a dense matrix, and $vec{b}(vec{y})$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $vec{x}$ and keep $vec{y}$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($F(vec{x}, vec{y}) = 0$). The Jacobian $J(vec{x}, vec{y})$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $vec{y}$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

  1. Initialize $vec{x} = vec{x}_0$, and $vec{y} = vec{y}_0$
  2. Fix $vec{y}_{n – 1}$, solve $A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$ for one of all the possible $vec{x}_{n}$ because this system is under-determined.
  3. Fix $vec{x}_{n – 1}$. Perform one iteration of Newton’s method for solving $F(vec{x}_{n-1}, vec{y}_{n}) = 0$ for $vec{y}_{n}$.
  4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf


Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $vec{y}$) and keep another set free (say $vec{x}$), the system becomes an under-determined, dense, and linear system of the subset of variables ($A(vec{y})vec{x} = vec{b}(vec{y})$, $A(vec{y})$ is a dense matrix, and $vec{b}(vec{y})$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $vec{x}$ and keep $vec{y}$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($F(vec{x}, vec{y}) = 0$). The Jacobian $J(vec{x}, vec{y})$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $vec{y}$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

  1. Initialize $vec{x} = vec{x}_0$, and $vec{y} = vec{y}_0$
  2. Fix $vec{y}_{n – 1}$, solve $A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$ for one of all the possible $vec{x}_{n}$ because this system is under-determined.
  3. Fix $vec{x}_{n – 1}$. Perform one iteration of Newton’s method for solving $F(vec{x}_{n-1}, vec{y}_{n}) = 0$ for $vec{y}_{n}$.
  4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf


Get this bounty!!!

#StackBounty: #algorithms #genetic-algorithms #algorithm-design Can genomes be heterogeneous and express entities with heterogeneous el…

Bounty: 50

I never took a formal GA course, so this question might be vague: I’m trying to see whether I’m approaching this problem well.

Usually a genome is represented as a sequence of homogeneous elements, such as binary numbers, logic gates, elementary functions, etc., which can then be assembled into a homogeneous structure like a syntax-tree for a computer program or a 3D object or whatever.

My problem involves evolving a graph of components, lets say X, Y and Z: the graph can have N nodes and each node is an instance of either X, Y or Z. Encoding such a graph structure in a genome is rather straightforward, however, I also need to attach additional information for what X, Y and Z do themselves–which is actually the main object of the GA.

So it seems like my genome should code for a heterogeneous entity: an entity which is composed both of a structure graph and a functionality specification. It is not impossible to subsume the elements (genes) which code for the structure and those that code for functionality under a single parent “gene”, and then simply separate them when the entity is being assembled, but this doesn’t feel like the right approach.

Is this a common problem in GA? Am I supposed to find a “lower-level” representation / genome encoding in this situation? What are the relevant considerations?


Get this bounty!!!

#StackBounty: #algorithms #cryptography #number-theory #modular-arithmetic Optimal parallel-time repeated modular squaring circuit

Bounty: 50

Given a 4096-bit integer $x$ and a 4096-bit RSA modulus $N$ (of unknown factorisation) what is the fastest circuit to compute $x^{2^T} mod N$ where $T=2^{40}$. That is, what is the fastest parallel-time (i.e. lowest latency) algorithm for repeatedly squaring $x$ modulo $N$ a trillion times?

For context, I am looking to build a Verifiable Delay Function ASIC to squeeze out opportunities for parallelisation that CPUs/GPUs cannot capture.

I have found hundreds of papers in the literature that cover modular multiplication. Families of algorithms include Montgomery, Barrett, Residue Number System (RNS), sum of residues, Kochanski, Brickell. Often the optimised parameter is not latency—e.g. it can be throughput, power consumption, die area, simplicity, side channel leak resistance, etc. The fastest implementation I found does a 4096-bit modular multiplication in 607ns.

Sub-questions:

  1. What is the optimal parallel-time algorithm for squaring a 4096-bit integer? One brute-force approach is to implement a 4096-by-4096 multiplier with $4096^2$ gates and a tree of 3-to-2 adders of depth 4096. Can we do better than that?
  2. The algorithms discussed in the literature often do modular multiplication (as opposed to modular squaring). What latency gains can be had from designing a circuit specific to modular squaring?


Get this bounty!!!

#StackBounty: #algorithms #network-flow Minimum cost circulation problem with bounded number of edges

Bounty: 50

During an article I am writing, I encountered the following problem:
Let $N=(G=(V,E),W,C)$ be a network with a graph $G$, a weight function $W:Eto R$ and an integer capacity function $C:E to N$.
Find a circulation $f$ with minimal cost $W(f)$ such that the number of edges used by the circulation (i.e., edges $e$ s.t. $f(e)> 0$) is smaller than or equal to a parameter $r$.

Note that if $r=|E|$, the problem is simply the well-known circulation, which is solvable in polynomial time.

I tried to search “Google Scholar” and use variations of the cycle canceling (and other min cost flow) algorithms to solve the problem but with no successes.


Get this bounty!!!

#StackBounty: #algorithms #data-structures #time-complexity Skip List estimate number of elements less than (or greater than) a value

Bounty: 50

I am looking for a thread safe data structure that can find an element in O(log n) and for which for any value, x, the number of elements greater than (or less than) can be estimated also in O(log n). Can a skip list or some modification of it be used? Is there an alternative data structures (Parallel Red-Black Tree for example?) that can be used?


Get this bounty!!!

#StackBounty: #algorithms #search-algorithms #binary-search Feeding real-time data and binary search algorithm termination

Bounty: 50

This question was asked in our exam long a go and I don’t remember exact words.

The scenario was,
Initially you are given a set of finite data to start with, and a key value (which you have to find in a given set). Can we terminate binary search algorithm/program, if we grow the size of the data by feeding new finite data in different quantum during the search is going on.

e.g. if I am searching name of a person in a directory, and there is a on-going survey of the new connections by telecom dept., which they are transmitting in database from remote location continuously (without interruption) and those newly found data are fed in binary-search in real time.

Can we say that binary search will terminate for the key value in this scenario before new data is fed?, if yes then how to prove it?

Sorry, if I am not clear enough. I will try to explain further if required.


Get this bounty!!!

#StackBounty: #machine-learning #classification #scikit-learn #algorithms Algorithm to move events of MC from one class to other to mat…

Bounty: 50

I have MC and data each having events in two classes 0 and 1. I am trying to write an algorithm such that I can match the number of events in class 0 and 1 of MC to data i.e I want to correct MC events by moving them from one class to other such that the ratio of events in the two classes for both data and MC is same. The way I proceeded is:

  1. Train a GradientBoostingClassifier from scikit ensemble for both data and MC individually(say data_clf and mc_clf)
     mc_clf.fit(X_mc, Y_mc)
     data_clf.fit(X_data , Y_data)
    

where Y_mc and Y_data is the corresponding class “mc_class” and “data_class” having values 0 or 1 depending on which class they belong to.

  1. Now, if X_mc is my input variable, use predict_proba to predict the probability of classifier of data and MC using MC inputs ONLY i.e
     y_mc = mc_clf.predict_proba(X_mc)
     y_data = data_clf.predict_proba(X_mc)
    
  2. After this, I try to move the events of MC from one class to another by comparing their probability in data and MC.
     for i in range(0, len(mc)):
         if (mc.loc[i]['mc_class'] == 0): 
             wgt = y_data[i][0]/ y_mc[i][0]
             if (wgt<1): mc.loc[i]['mc_class_corrected'] = 1
             else: mc.loc[i]['mc_class_corrected'] = mc.loc[i]['mc_class'] 
    
    
         if (mc.loc[i]['mc_class'] == 1): 
             wgt = y_data[i][1]/ y_mc[i][1]
             if (wgt<1) : mc.loc[i]['mc_class_corrected'] = 0
             else: mc.loc[i]['mc_class_corrected'] = mc.loc[i]['mc_class'] 
    

In the end what happens is that initially suppose I had more events in class 0 than 1 in MC as compared to data. So I expect events from class 0 to move to class 1. However, I see that almost >95% of my events in class 0 of MC are moving to class 1 while I was expecting only about 30% of events to move (when compared to the number of events in data and MC)?
Is there any mistake in this ideology of working?

Thanks a lot:)


Get this bounty!!!

#StackBounty: #algorithms #sorting #strings #binary Partitioning through block moves to the end

Bounty: 50

Suppose we have a binary string $s$. We wish to partition this string in a series of $0$s followed by $1$s (alternatively: we wish to sort), using only one operation: moving three consecutive elements to the end.

E.g. if our string is $ABCDEFGH$ we can do our operation at index $1$ to get $DEFGHABC$. If we did it at index $4$ we’d get $ABCGHDEF$.

I’m interested in optimal solutions – that is with the least amount of moves. I have a working algorithm using IDA* with heuristic “# of groups of ones with length $leq 3$ before the final zero”, with the logic that each such a group requires at least one move to fix. An example optimal solution (where ^ indicates where a block was chosen to move to the end):

1101101011101011010010010011                               
                       ^                                   
1101101011101011010010011100                               
                     ^                                     
1101101011101011010011100001                               
              ^                                            
1101101011101010011100001110                               
           ^                                               
1101101011110011100001110010                               
                        ^                                  
1101101011110011100001110001                               
     ^                                                     
1101111110011100001110001010                               
  ^                                                        
1111110011100001110001010011                               
                     ^                                     
1111110011100001110000011101                               
                       ^                                   
1111110011100001110000001111                               
               ^                                           
1111110011100000000001111111                               
        ^                                                  
1111110000000000001111111111                               
   ^                                                       
1110000000000001111111111111                               
^                                                          
0000000000001111111111111111 

However, this algorithm is exponential and not feasible for larger strings. After studying quite some optimal solutions, especially tough ones like above, I can’t immediately think of an optimal algorithm. Moves can be quite non-trivial.

Is there a feasible optimal algorithm? Or is this problem hard?


Get this bounty!!!