Cross Platform WebRTC Browser Testing: Chrome, Firefox, Edge & Safari

5 Comments

This is a guest post by Philipp Hancke. He was kind enough to share the work he’s done already in automating his own WebRTC Safari tests.

Now that Apple added WebRTC to its Safari browser, it is time to ask –

How do you test WebRTC four browsers on different operating systems? Using Selenium Grid it is surprisingly easy, learn how.

Webkit recently joined the WebRTC ship which means we now have 5 major browsers to test:

  • Chrome, any operating system
  • Firefox, any operating system
  • Microsoft Edge, Windows 10
  • Safari (Technology Preview), OSX

Until a few days ago it was feasible to run tests all WebRTC supported browsers only on a single operating system (Getting Chrome, Firefox and Edge tested on Windows 10). With Safari, that has changed. Sounds like a big problem, no?

This kind of testing is typically done using Selenium which is used to remotely control the browser. And if it is remotely controlled it can be done with selenium’s grid functionality. Which is surprisingly easy.

Say we want to test Edge and Safari from a Windows machine. First step is to download the selenium standalone jar file. I am still sticking to version 3.3.1. Next, set up a selenium grid hub on one machine by running:

java -jar selenium-server-standalone-3.3.1.jar -role hub

This will start a hub which is where you connect your selenium nodes as well as test clients to. It will also tell you the URL that you need to connect your nodes to.

Next, start a node on your OSX machine by running:

java -jar selenium-server-standalone-3.3.1.jar -role node -hub http://where-your-hub-tells-you-it-is-running -browser “browserName=safari”

This will start a node that will run your Safari instance.

Now, in the code running selenium you need to instruct it to do two things:

  • Use Safari Technology Preview as shown in this PR
  • Make it use the hub as shown here

Next run your test code. If it works out of the box, great. If it does not you will probably spend a lot of time figuring out the issues. The most common issue here is forgetting to include the respective WebDriver binary in a path where it is found.

Dr Alex - June 17, 2017

From the selenium side, it will work just fine. Unfortunately, this will not work as you can not bypass the security prompt, create fake media, and/or use pages served from an HTTP server without further advanced settings that selenium cannot handle.

In chrome, you can use command lines, that can be passed through webdriver to the browser binary.
In firefox, you can use profile, that selenium can pass down to the browser as well.

It is doable, but you will to know a lot more than what is written here or handled in fippo code today, and you will need to maintain a dedicated selenium hub/node with a modified Safari tech preview which version needs to be at least 33 to include the right flags.

More details and screenshots of options here:
http://webrtcbydralex.com/index.php/2017/06/16/safari-technical-preview-33-the-most-webrtc-compliant-browser-out-there/

Video of automated STP 33 with selenium there:
https://twitter.com/agouaillard/status/875688355014561792

Tsahi Levent-Levi - June 19, 2017

Tokbox’ Adam Ullman has shared code on how to enable fake devices and all the nice things on travis-multirunner.

Check it out: https://github.com/DamonOehlman/travis-multirunner/pull/23

    Dr Alex Gouaillard - June 30, 2017

    Adam added some, but it is still not working the way you wrote it. Actually, you still have one option you cannot automate as would be needed by travis VM, before safari tech preview 34 (which went out the day before yesterday [1]), and a few other tricks that webkit developers and people that can build safari from source have.

    You can watch the presentation by Young Fablet at the latest webrtc meet-up in SF [2]. It describes this at 46:28 in the video, before mentioning how much the webrtc-in-webkit project contributions had been beneficial to all at 50:30.

    When would you post a video of fully automated safari with selenium / webdriver to show how your solution work?

    [1] – https://webkit.org/blog/7760/release-notes-for-safari-technology-preview-34/
    [2] – https://www.youtube.com/watch?v=S5p2ZCMTTUU&feature=youtu.be

      Tsahi Levent-Levi - July 1, 2017

      Alex, thanks for contributing.

      While I am sure what we write here on the blog is never the whole story and people need to figure out other pieces on their own, I am also certain that people are happy to read the information here and make use of it – otherwise we wouldn’t have spent the time and effort in creating it.

Cross Platform WebRTC Browser Testing: Chrome, Firefox, Edge & Safari | AllThingsRTC - August 8, 2017

[…] Read more at Testrtc […]

Comments are closed