I am ranking a filtered item list as per user’s metadata and historical behaviour.
Now how to calculate metrices like precision at K?
One approach could be –
Divide historical data in training and test dataset(e.g. 90% and 10%). Now pick all the items for a user from test data and randomize the order and pass it to the ranking algorithm.
Now compare actual items for the user from test data to the ranked output.