I’m not sure how to ask this question, so please forgive if I’m vague. I’m new to DS and don’t speak the language yet. 🙂
Business process A (bpA) assigns accounts to handlers based on existing logic.
Business process B (bpB) uses ML to assign accounts based on better logic, but I’m not sure how much better.
I have a pool of data stretching back a few years, which I take as my baseline. This is entirely based on bpA. I train and run my model to get bpB recommendations. I then compare performance rates between “bpA did what bpB recommended” and the inverse. I see a difference in performance between the two sets, but I am not sure that the number I see is the number I should be reporting.
Where should I be looking? What should I be reading? Thanks!