#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $$vec{y}$$) and keep another set free (say $$vec{x}$$), the system becomes an under-determined, dense, and linear system of the subset of variables ($$A(vec{y})vec{x} = vec{b}(vec{y})$$, $$A(vec{y})$$ is a dense matrix, and $$vec{b}(vec{y})$$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $$vec{x}$$ and keep $$vec{y}$$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($$F(vec{x}, vec{y}) = 0$$). The Jacobian $$J(vec{x}, vec{y})$$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $$vec{y}$$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

1. Initialize $$vec{x} = vec{x}_0$$, and $$vec{y} = vec{y}_0$$
2. Fix $$vec{y}_{n – 1}$$, solve $$A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$$ for one of all the possible $$vec{x}_{n}$$ because this system is under-determined.
3. Fix $$vec{x}_{n – 1}$$. Perform one iteration of Newton’s method for solving $$F(vec{x}_{n-1}, vec{y}_{n}) = 0$$ for $$vec{y}_{n}$$.
4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf

Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $$vec{y}$$) and keep another set free (say $$vec{x}$$), the system becomes an under-determined, dense, and linear system of the subset of variables ($$A(vec{y})vec{x} = vec{b}(vec{y})$$, $$A(vec{y})$$ is a dense matrix, and $$vec{b}(vec{y})$$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $$vec{x}$$ and keep $$vec{y}$$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($$F(vec{x}, vec{y}) = 0$$). The Jacobian $$J(vec{x}, vec{y})$$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $$vec{y}$$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

1. Initialize $$vec{x} = vec{x}_0$$, and $$vec{y} = vec{y}_0$$
2. Fix $$vec{y}_{n – 1}$$, solve $$A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$$ for one of all the possible $$vec{x}_{n}$$ because this system is under-determined.
3. Fix $$vec{x}_{n – 1}$$. Perform one iteration of Newton’s method for solving $$F(vec{x}_{n-1}, vec{y}_{n}) = 0$$ for $$vec{y}_{n}$$.
4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf

Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $$vec{y}$$) and keep another set free (say $$vec{x}$$), the system becomes an under-determined, dense, and linear system of the subset of variables ($$A(vec{y})vec{x} = vec{b}(vec{y})$$, $$A(vec{y})$$ is a dense matrix, and $$vec{b}(vec{y})$$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $$vec{x}$$ and keep $$vec{y}$$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($$F(vec{x}, vec{y}) = 0$$). The Jacobian $$J(vec{x}, vec{y})$$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $$vec{y}$$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

1. Initialize $$vec{x} = vec{x}_0$$, and $$vec{y} = vec{y}_0$$
2. Fix $$vec{y}_{n – 1}$$, solve $$A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$$ for one of all the possible $$vec{x}_{n}$$ because this system is under-determined.
3. Fix $$vec{x}_{n – 1}$$. Perform one iteration of Newton’s method for solving $$F(vec{x}_{n-1}, vec{y}_{n}) = 0$$ for $$vec{y}_{n}$$.
4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf

Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $$vec{y}$$) and keep another set free (say $$vec{x}$$), the system becomes an under-determined, dense, and linear system of the subset of variables ($$A(vec{y})vec{x} = vec{b}(vec{y})$$, $$A(vec{y})$$ is a dense matrix, and $$vec{b}(vec{y})$$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $$vec{x}$$ and keep $$vec{y}$$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($$F(vec{x}, vec{y}) = 0$$). The Jacobian $$J(vec{x}, vec{y})$$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $$vec{y}$$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

1. Initialize $$vec{x} = vec{x}_0$$, and $$vec{y} = vec{y}_0$$
2. Fix $$vec{y}_{n – 1}$$, solve $$A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$$ for one of all the possible $$vec{x}_{n}$$ because this system is under-determined.
3. Fix $$vec{x}_{n – 1}$$. Perform one iteration of Newton’s method for solving $$F(vec{x}_{n-1}, vec{y}_{n}) = 0$$ for $$vec{y}_{n}$$.
4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf

Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $$vec{y}$$) and keep another set free (say $$vec{x}$$), the system becomes an under-determined, dense, and linear system of the subset of variables ($$A(vec{y})vec{x} = vec{b}(vec{y})$$, $$A(vec{y})$$ is a dense matrix, and $$vec{b}(vec{y})$$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $$vec{x}$$ and keep $$vec{y}$$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($$F(vec{x}, vec{y}) = 0$$). The Jacobian $$J(vec{x}, vec{y})$$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $$vec{y}$$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

1. Initialize $$vec{x} = vec{x}_0$$, and $$vec{y} = vec{y}_0$$
2. Fix $$vec{y}_{n – 1}$$, solve $$A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$$ for one of all the possible $$vec{x}_{n}$$ because this system is under-determined.
3. Fix $$vec{x}_{n – 1}$$. Perform one iteration of Newton’s method for solving $$F(vec{x}_{n-1}, vec{y}_{n}) = 0$$ for $$vec{y}_{n}$$.
4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf

Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $$vec{y}$$) and keep another set free (say $$vec{x}$$), the system becomes an under-determined, dense, and linear system of the subset of variables ($$A(vec{y})vec{x} = vec{b}(vec{y})$$, $$A(vec{y})$$ is a dense matrix, and $$vec{b}(vec{y})$$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $$vec{x}$$ and keep $$vec{y}$$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($$F(vec{x}, vec{y}) = 0$$). The Jacobian $$J(vec{x}, vec{y})$$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $$vec{y}$$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

1. Initialize $$vec{x} = vec{x}_0$$, and $$vec{y} = vec{y}_0$$
2. Fix $$vec{y}_{n – 1}$$, solve $$A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$$ for one of all the possible $$vec{x}_{n}$$ because this system is under-determined.
3. Fix $$vec{x}_{n – 1}$$. Perform one iteration of Newton’s method for solving $$F(vec{x}_{n-1}, vec{y}_{n}) = 0$$ for $$vec{y}_{n}$$.
4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf

Get this bounty!!!

#StackBounty: #algorithms #linear-algebra Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Bounty: 100

Problem

Solving a non-linear system of equations.

Number of variable is same as number of equations.

When I fix a set variables (say $$vec{y}$$) and keep another set free (say $$vec{x}$$), the system becomes an under-determined, dense, and linear system of the subset of variables ($$A(vec{y})vec{x} = vec{b}(vec{y})$$, $$A(vec{y})$$ is a dense matrix, and $$vec{b}(vec{y})$$ is a dense vector). Let’s call this sub-system as system 1.

When I fix $$vec{x}$$ and keep $$vec{y}$$ free, the system becomes an over-determined, sparse, and non-linear system of the subset of variables ($$F(vec{x}, vec{y}) = 0$$). The Jacobian $$J(vec{x}, vec{y})$$ has closed form. Let’s call this sub-system as system 2.

About half of the equations in system 2 are equality constraints that are linear in terms of $$vec{y}$$. Each of the constraints are quite sparse and involves only about 5% of all variables.

Can I solve with the following?

Algorithm 1

1. Initialize $$vec{x} = vec{x}_0$$, and $$vec{y} = vec{y}_0$$
2. Fix $$vec{y}_{n – 1}$$, solve $$A(vec{y}_{n-1})vec{x}_{n} = vec{b}(vec{y}_{n-1})$$ for one of all the possible $$vec{x}_{n}$$ because this system is under-determined.
3. Fix $$vec{x}_{n – 1}$$. Perform one iteration of Newton’s method for solving $$F(vec{x}_{n-1}, vec{y}_{n}) = 0$$ for $$vec{y}_{n}$$.
4. If not converged, go to step 2.

Algorithm 2

If I replace step 2 by an iteration of Newton’s method for solving system 1, then I guess the steps become a block Newton’s method.

Question

But I don’t know if these two algorithms can work because system 1 is under-determined and system 2 is over-determined.

Can this work?

Thanks.

Related

An approximate block Newton method for coupled iterations of nonlinear solvers: Theory and conjugate heat transfer applications

https://www.sciencedirect.com/science/article/pii/S0021999109004379

Solving over-determined non-linear system by Newton’s method with Moore-Penrose inverse is fine: https://arxiv.org/pdf/1703.07810.pdf

Get this bounty!!!

#StackBounty: #algorithms #genetic-algorithms #algorithm-design Can genomes be heterogeneous and express entities with heterogeneous el…

Bounty: 50

I never took a formal GA course, so this question might be vague: I’m trying to see whether I’m approaching this problem well.

Usually a genome is represented as a sequence of homogeneous elements, such as binary numbers, logic gates, elementary functions, etc., which can then be assembled into a homogeneous structure like a syntax-tree for a computer program or a 3D object or whatever.

My problem involves evolving a graph of components, lets say X, Y and Z: the graph can have N nodes and each node is an instance of either X, Y or Z. Encoding such a graph structure in a genome is rather straightforward, however, I also need to attach additional information for what X, Y and Z do themselves–which is actually the main object of the GA.

So it seems like my genome should code for a heterogeneous entity: an entity which is composed both of a structure graph and a functionality specification. It is not impossible to subsume the elements (genes) which code for the structure and those that code for functionality under a single parent “gene”, and then simply separate them when the entity is being assembled, but this doesn’t feel like the right approach.

Is this a common problem in GA? Am I supposed to find a “lower-level” representation / genome encoding in this situation? What are the relevant considerations?

Get this bounty!!!

#StackBounty: #algorithms #cryptography #number-theory #modular-arithmetic Optimal parallel-time repeated modular squaring circuit

Bounty: 50

Given a 4096-bit integer \$x\$ and a 4096-bit RSA modulus \$N\$ (of unknown factorisation) what is the fastest circuit to compute \$x^{2^T} mod N\$ where \$T=2^{40}\$. That is, what is the fastest parallel-time (i.e. lowest latency) algorithm for repeatedly squaring \$x\$ modulo \$N\$ a trillion times?

For context, I am looking to build a Verifiable Delay Function ASIC to squeeze out opportunities for parallelisation that CPUs/GPUs cannot capture.

I have found hundreds of papers in the literature that cover modular multiplication. Families of algorithms include Montgomery, Barrett, Residue Number System (RNS), sum of residues, Kochanski, Brickell. Often the optimised parameter is not latency—e.g. it can be throughput, power consumption, die area, simplicity, side channel leak resistance, etc. The fastest implementation I found does a 4096-bit modular multiplication in 607ns.

Sub-questions:

1. What is the optimal parallel-time algorithm for squaring a 4096-bit integer? One brute-force approach is to implement a 4096-by-4096 multiplier with \$4096^2\$ gates and a tree of 3-to-2 adders of depth 4096. Can we do better than that?
2. The algorithms discussed in the literature often do modular multiplication (as opposed to modular squaring). What latency gains can be had from designing a circuit specific to modular squaring?

Get this bounty!!!

#StackBounty: #algorithms #network-flow Minimum cost circulation problem with bounded number of edges

Bounty: 50

During an article I am writing, I encountered the following problem:
Let \$N=(G=(V,E),W,C)\$ be a network with a graph \$G\$, a weight function \$W:Eto R\$ and an integer capacity function \$C:E to N\$.
Find a circulation \$f\$ with minimal cost \$W(f)\$ such that the number of edges used by the circulation (i.e., edges \$e\$ s.t. \$f(e)> 0\$) is smaller than or equal to a parameter \$r\$.

Note that if \$r=|E|\$, the problem is simply the well-known circulation, which is solvable in polynomial time.

I tried to search “Google Scholar” and use variations of the cycle canceling (and other min cost flow) algorithms to solve the problem but with no successes.

Get this bounty!!!