We have recently implemented the AB-testing module for Episerver, but have run into a problem with 404 responses. For some AB tests, it is working just fine, but for a specific campaign about 15% of the requests end in a 404 response. Details about the campaign:

• 50% of users are included in the test
• For the first 15 minutes, no 404 responses appear
• After 15 minutes, about 15% of the users receive a 404 response
• The difference between A and B is the ordering of blocks in a content area

We do not see any errors in the logs indicating why certain requests return 404.

We think it may be a racing condition or similar, but do not know how to debug it further.

Has anyone experienced similar issues?
Does anyone have ideas about how to debug further?

Thanks

Thomas