Overview

Automated generation of Playwright test cases using AI, driven entirely by Pull Request (PR) merges. This system analyzes code changes in real time and produces relevant end-to-end tests without any human intervention, ensuring test coverage evolves alongside the application.

Key Features

• PR-Driven Test Generation: Automatically triggers when a Pull Request is merged into the main branch of the application repository, ensuring tests are always aligned with the latest changes.
• Diff-Based Intelligence: Extracts the diff between the latest merged PR and the previous main branch state, sending only the relevant changes to the AI model for precise test generation.
• AI-Generated Playwright Tests: Uses Google Gemini to generate Playwright .spec.ts files based solely on code changes, minimizing redundant or outdated tests.
• Decoupled Test Repository: Commits generated tests to a separate Playwright repository, keeping application code and test automation cleanly separated.
• Fully Automated Execution: A secondary GitHub Action in the Playwright repository automatically runs the newly generated tests against the application.
• Zero Manual Test Writing: Eliminates repetitive test authoring while maintaining strong regression coverage in the CI/CD pipeline.

Technologies Used

• Node.js: For scripting automation logic and integrating with AI APIs.
• GitHub Actions: To orchestrate workflows on PR merge, test generation, commits, and execution.
• Playwright: For robust, end-to-end browser testing.
• Google Gemini AI: To intelligently generate test cases based on code diffs.
• Git & GitHub: For version control and repository orchestration.

Challenges and Learnings

A key challenge was ensuring AI-generated tests were both relevant and reliable when based only on diffs. This was addressed by carefully structuring the prompt sent to Gemini and normalizing diffs to focus on user-impacting changes. Another learning was designing a clean multi-repo workflow that avoids circular triggers while keeping automation fully hands-off.

Outcome

This setup delivers continuously evolving test coverage with minimal maintenance overhead. Test generation now scales naturally with feature development, CI remains fast and reliable, and the overall development workflow benefits from faster feedback and higher confidence in changes. The result is a smarter, more autonomous testing pipeline powered by AI and automation.