Who Tests the Tests? - Automated UI Test Auditing for CI Stability

Writing automated tests is just the beginning. If your tests run in a browser or mobile VM, they're going to fail sometimes. This is a problem for developers' workflows -- how can you tell if the tests are useful? What happens when you can't merge and it's not your fault? My team has taken many stabs at the problem over the years. In short, you need to audit your tests, and doing that audit via automation offers a lot of benefits. This talk will cover: - Why having reliable pre-merge automation is so important - How previous attempts at wrangling unstable tests failed - Our current solution and how you could implement tooling like it yourself - How to both model and measure the effectiveness of your strategy - A short but pointed rant about Github Actions, if left unchecked Developers with experience writing CI tooling will benefit most, but if you're in the business of working with UI tests or enforcing standards in your codebase, there's plenty to pick up. I don't have any SaaS subscriptions to sell you, I promise.