Migrating Rails System Tests from Selenium to Playwright

Playwright is often recognized for its consistent test results and strong performance. Recent versions of Rails include Playwright as a first-class system test driver.

This means that switching a Rails application from Selenium to Playwright could be ( BUT not necessary) as simple as changing the driver configuration rather than rewriting the test suite, with Capybara serving as the layer in between.

This article documents the steps, as well as some lessons learnt.

Step 0 - Measure complexity

1) Check if there are any Selenium-native calls in your test codes. These don't exists on the Playwright driver, so mark them for translation / replacement later. If there is no hit, then the effort to migrate would be minimum.

grep -rn "driver\.browser\|execute_cdp\|\.native\|Selenium::\|switch_to\|manage\.window" test/system/

2) Find codes that continue to run, but could mean someting different. Nothing here is Selenium-native, these are ordinary Capybara calls that work on both drivers. What changes is what they hand back.

grep -rn "execute_script\|evaluate_script\|\[:class\]" test/system/

• An absent attribute returns nil instead of "", and a JavaScript null comes back as {} instead of nil so [:class] and evaluate_script hits fail quietly, or pass for the wrong reason. 
• execute_script is in that list for a different reason: it returns nothing useful and behaves identically on both drivers, but it is usually where Selenium workarounds hide. Each hit is worth re-reading to ask whether it is still needed at all.

Step 1: Swap the gem

 group :test do
   gem "capybara"
+  gem "capybara-playwright-driver"
-  gem "selenium-webdriver"
   gem "simplecov", require: false
 end

Then runs bundle install

Step 2: Add Node, and pin it to the gem

1) Check the playwright version required by the gem.

bundle exec ruby -e 'require "playwright"; puts Playwright::COMPATIBLE_PLAYWRIGHT_VERSION'
# => 1.62.1

2) Pin the exact version in package.json

{
  "name": "your-app",
  "private": true,
  "comment": "Node is here only to supply the Playwright CLI that playwright-ruby-client drives over a pipe. Assets are still importmap no bundler. The version must match Playwright::COMPATIBLE_PLAYWRIGHT_VERSION in the playwright-ruby-client gem.",
  "devDependencies": {
    "playwright": "1.62.1"
  }
}

3) Run npm install and add /node_modules/ to git ignore if it hasn't been added.

4) Bumping playwright-ruby-client requires bumping package.json, a test that guard this would be nice to have.

test "the pinned Playwright CLI matches what the gem speaks" do
  pinned = JSON.parse(Rails.root.join("package.json").read)
              .dig("devDependencies", "playwright")
  assert_equal Playwright::COMPATIBLE_PLAYWRIGHT_VERSION, pinned
end

Step 3: Install the browser

npx playwright install chromium

This downloads a Playwright-managed Chromium.

Step 4: Swap the driver

In your system test case class, replace the driven_by :selenium line with:

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  driven_by :playwright, screen_size: [ 1400, 1400 ], options: {
    browser_type: :chromium,
    headless: true,
    default_timeout: 8,             # seconds; matches Capybara.default_max_wait_time
    default_navigation_timeout: 8
  }
end

Step 5: The "translation" work

1) This is the whole of the real work. Some samples:

2) The repos might also include codes which are no longer necessary. For example, helper method that retries clicking for several times to reduce the flackiness in Selenium.

def click_until(element, attempts: 3)
  attempts.times do
    element.click
    return if yield
    sleep 0.25
  end
  page.execute_script("arguments[0].click()", element)   # JS fallback
  return if yield
  raise Capybara::ExpectationNotMet, "#{element.tag_name} never responded to a click"
end

3) While removing unecessary codes, do evaluate the true objectives.  There are two kinds and they look identical:

• "…until the element responds." Playwright covers this. Delete it.
• "…until the application finished something." Playwright does not cover this.

An example of the second kind - "wait until a request the page made has been answered". This can be translated by using Playwright raw API:

def await_response(url_pattern, body: nil)
  matched = lambda do |response|
    next false unless response.url.match?(url_pattern)
    next true unless body

    response.request.post_data.to_s.match?(body)
  end

  page.driver.with_playwright_page do |playwright_page|
    playwright_page.expect_response(matched) { yield }
  end
end

...

apply = find("button", text: "Apply")
await_response(%r{/payments/validate}, body: /SAVE20/) { apply.click }
apply.click   # now guaranteed to be after the first round trip

Step 6: Decide whether Chrome still belongs in your CI image

The google-chrome-stable often exists in the CI image as a dependency for Selenium driver. It can be removed if the only consumer is Selenium driver. In the sample below, google-chrome-stable is also used by Ferrum, so it must be kept.

- name: Install packages
  # google-chrome-stable is for Ferrum (the html_to_pdf tool renders with it
  # in a unit test); Playwright brings its own Chromium for system tests.
  run: sudo apt-get update && sudo apt-get install ... google-chrome-stable ...

Step 7: Update CI

Four steps, inserted after Ruby setup and before the test run:

- name: Set up Node
  uses: actions/setup-node@v7
  with:
    node-version: 24
    cache: npm

- name: Install the Playwright CLI
  run: npm ci

- name: Cache Playwright browsers
  uses: actions/cache@v6
  with:
    path: ~/.cache/ms-playwright
    key: ${{ runner.os }}-playwright-${{ hashFiles('package-lock.json') }}

- name: Install Chromium for Playwright
  run: npx playwright install --with-deps chromium

Note: Repos that already run Node in CI for something else (For example, lints ERB with herb), worth considering reuse the version already in the workflow.

Step 8: Verify

Run the full suite, then run the system tests again on fixed seeds:

bin/rails test
bin/rails test test/system --seed 1234
bin/rails test test/system --seed 4242

Additional Information

1) Two failure modes that are completely silent

# page.route takes its handler positionally, with two arguments.

# Wrong — the block is ignored, and so is a one-argument lambda
playwright_page.route(/example\.com/) { |route| route.abort }

# Right
playwright_page.route(/example\.com/, ->(route, _request) { route.abort })

# evaluate_script returns {}, not nil, for a JavaScript null

# Fails — the value is {}
assert_nil page.evaluate_script("localStorage.getItem('darkMode')")

# Works — assert the null in the page, not in Ruby
assert page.evaluate_script("localStorage.getItem('darkMode') === null")

2) Attribute semantics changed

Selenium returned "" for an absent attribute; Playwright returns nil. Anything shaped like find("html")[:class].include?("dark") is affected.

You should convert these to selector assertions:

# Before
html = find("html")
assert html[:class].include?("dark")

# After
assert_selector "html.dark"

3) Keeping Capybara.disable_animation

• Under Selenium it was a correctness measure. Clicks do not land on an element still animating.
• Under Playwright it is a speed measure. Playwright refuses to act on an element that is not stable, so it waits the transition out and gets it right. Disabling animations improve test speed.