*Bounty: 50*

*Bounty: 50*

Suppose we have algorithms $A_k$ and $B_k$, $1leq kleq n$, and $alpha = 0.05$.

Friedman test for the algorithms $A_k$ shows that the null hypothesis that all $A_k$s perform equally well, can be rejected. Suppose $p_A = 10^{-5}llalpha$.

Friedman test for the algorithms $B_k$ shows that the null hypothesis that all $B_k$s perform equally well, cannot be rejected. Suppose $p_B = 0.4$.

My problem is that the $p$-value $p_{AB}$ from the Friedman test for $A_k$s and $B_k$s is greater than $alpha$, hence I cannot apply Nemenyi post-hoc test if I blindly follow the procedure

- perform Friedman on algorithms
- if $H_0$ is rejected, proceed to Nemenyi on the same algorithms

**Question:** Are the results from Nemenyi test for all algorithms still valid?

I am not sure, but I would say yes because

- I have shown that already not all $A_k$s have the same performance (and the results would still be significant with $2p_A$, i.e., if I used Bonferroni correction for multiple-hypothesis testing)
- Nemenyi for $A_k$s and $B_k$s found two groups of algorithms, i.e., some differences.