Just a quick blog post: there's a new WebDriver release out. The files are on the Google Code site. Features include:
Woot! The next release should be a clean up release, rolling through our open issues and closing as many as we can.
One of the hidden gems of webdriver is the support library. This contains useful classes and utilities for dealing with common situations and cases. We recently checked in a wrapper for SELECT elements. Beforehand, interacting with a SELECT could be a little verbose:
WebElement select = driver.findElement(By.tagName("select"));
List<WebElement> options = select.findElements(By.tagName("option);
for (WebElement option : options) {
if ("want this".equals(option.getValue()) {
option.setSelected();
break;
}
}
With the Select support class, this becomes:
Select element = driver.findElement(By.tagName("select"));
Select select = new Select(element);
select.selectByValue("want this");
This does the same job, but someone reading your test can now gather the intent that little bit more easily.
The Select class will be made more widely available when do our next release, hopefully near the end of this month.
More a note to myself than a full-fledged blog entry, but this may be useful to other people too.
Well, that's interesting. There's nsPromptService, which leads to: commonDialog.js and commonDialog.xul. All I need to do now is implement some sort of Thread.sleep mechanism, and it should be possible to handle alerts and prompts in Firefox.
Now, I'd expect the sleep methods to be associated with therading, and it looks like nsIEventQueueService has become nsIThreadManager so that's where I plan on starting to start the hunt for a good mechanism to sleep. Oh, and the nsIProxyObjectManager looks interesting too....
I posted this to the WebDriver group today, and thought that it might benefit from a slightly wider audience. So, that'll be the four of you who read this blog, then!
I thought it may be useful to give a bit of a run-down on my current list of ideas and issues to do with WebDriver, so that people know what's coming and what I'm planning to work on. It's a bit of a ramble, and may be missing technical context in places, so if this isn't your sort of thing, please skip this post :)
Problem: Javascript Alerts and Prompts
Solution: This is pretty easy to do in IE (spawn a separate thread that watches out for alerts being opened and tieing them to an open IE instance) but a bit tricky in Firefox. To get the API I want, it would be useful to be able to make Firefox "sleep". Sadly, there's no primitive to do this in JS, but there's still some possibilities.
Problem: Native events for Firefox
Solution: I need to find a mechanism for determining the HWND associated with an HTML document in Windows. I also need to learn a little about X for UNIX. How hard can it be?
Problem: The Safari driver needs a lot of TLC
Solution: I'm planning on taking a leaf out of the Gears book, and implementing an Input Manager hack to get access to Safari. This means that we may encounter some rough weather when Snow Leopard is released, and that it will be Mac OS only. On the plus side, I know that it can made to work, and already have some sample code that reaches into the DOM of a page (kindly donated by a friend in Australia)
Problem: The Chrome driver isn't done
Solution: Someone's working on this, and I'm talking to him to try and get the code available. As soon as I can, it'll be there. FWIW, it looks like we'll be going in through the automation interfaces used by Chrome's own unit tests.
Problem: Selenium emulation isn't complete
Solution: This partly hinges on the "js alerts and prompts" problem, but also hangs on modularising Selenium Core and injecting it where that's the appropriate thing to do (because Selenium's pretty huge, and injecting the whole thing every time we wanted to emulate a method would be dumb) I plan to do this by reading the source of Core via Rhino and then outputting various functions into files. This allows us to automatically generate the modules of Core, which makes tracking HEAD of Selenium a lot easier.
Problem: Bindings for other languages (C#, Python and Ruby)
Solution: The reworking of the IE driver that's going on in the "dotnet_bindings" branch will facilitate writing bindings for IE (using ctypes, DL, pinvoke or JNA) The Firefox driver is a SMOP (Simple Matter of Programming) because the wire protocol is pretty easy. Remote support should be easy to write too, once I finish documenting the protocol (and implementing it properly) There are some nascent Python bindings which people are working on, and I'm using the .Net bindings for IE to rework the IE codebase. There are some ruby versions of webdriver on github too, which is great to see!
Problem: The IE driver crashes
Solution: We're working on making sure that the only exceptions that occur in the IE driver are things that are beyond our control. We're pushing more defensive checks into the codebase to stop some of the dafter mistakes and releases will be made with try/catch blocks and handling of runtime exceptions turned on. We may get some grotty behaviour, but it should never take down the JVM again.
Problem: The IE driver doesn't support 64bit
Solution: This is being worked on now!
That's the major list of issues that are occupying my thoughts right now. If anyone's interested in chipping in and getting involved, or just getting some more information on any of these points, please feel free to either ping me an email or ask about it on the list.
That's better.
Over the Christmas holidays I've been chipping away at something that I've wanted to do for a long time: rework the IE driver to sit more completely on the C++ underpinnings, removing the JNI layer and replacing it with JNA. As part of the process, I've finally started writing more code for the .Net bindings too.
There's still a chunk of work that needs doing, and I'm taking the time to also make some changes to bring the code more into line with the Google C++ coding style, but it's been a fun little project to work on during the break. It also means that the long-rumoured Python bindings should be a doddle to put together too.
Hurrah!
The work is in the "dotnet_bindings" branch of the webdriver project on Google Code, if you'd like to have a look.
Of course, I'm now in a bit of a quandary. Removing the complexity of JNI is a Good Thing, in and of itself, so continuing with the work of adding JNA makes a lot of sense. The problem is: do we write a native binding for each and every language and browser (so L*B different drivers) or do we write some minimal wrappers, allowing the use of idiomatic language constructs, which call down to an underlying Java driver?
I think that one of the problems that I'm considering is the resistance of some people towards "Java" as a concept. The old myths about it being slow, consuming too much memory or not being created by a sanctioned vendor are issues that need to be considered.
Given the fact that if you're using WebDriver you're about to bear the cost of starting a browser and bouncing around all over the network even the startup cost of the JVM is negligible. Once the JVM is up and running, it runs extremely well. More importantly, it's likely that our tests are IO-bound rather than CPU-bound, so even if our test language of choice was something tediously slow it's unlikely that it would be the bottleneck in our testing.
So, "pffft" say I to the argument that Java's too slow. It's a mature response, I think we can all agree.
The argument about consuming too much memory is an interesting one, but again it's not something that I'm buying into. Modern machines have gigabytes of RAM, and (again) browsers are probably going to consume more memory than the test itself, particularly if a long-running browser instance is used. So, unless we're working back on 1998-era machines, the memory argument is a bit of a non-starter too.
Finally, there's the argument that Java doesn't come from a sanctioned vendor. Normally, this attitude is most common in Microsoft shops, but I've heard the same thing from escapees from Java, and I've worked on projects that were tied to a 1.4 release of the JDK because key parts of the infrastructure couldn't be updated to use Java 5 (sounds unlikely, but it's true) This is unfortunate, because WebDriver requires Java 5 or above.
Now, there are three ways of dealing with this last argument: to ignore it, to hide Java, or to write a native binding. The first of these doesn't really appeal to me, because if it's a problem that needs addressing, we really should address it directly. The second of these is pretty acceptable to me, but I've worked in places where that sort of trick would be frowned upon (and by "frowned upon", I mean "have the technology thrown out of the project") The problem gets harder to deal with in the hypothetical case where another library does the same thing. I'd imagine that attempting to load multiple JVMs, particularly different versions, would lead to all sorts of fun and games.
It's probable that I've overlooked something obvious, or that my worries are as detached from reality as something not terribly real, such as a unicorn selling delicious bacon burgers. I suppose it would be possible to do what the existing Selenium RC implementations do (start an external process), but that's a little unwieldy, even if it is properly hidden away. If people want to weigh in with an opinion, please do so either on your blog, in the comments here or on the webdriver group. I'd love to hear some other thoughts about the best way to handle this sort of thing.
For now, the best way forward looks like writing separate bindings for the key languages, which are Java, C#, Ruby and Python in my view. On the down side, this means that in order to wander freely across the codebase, you need to know a heck of a lot of languages. On the plus side, this approach means that it's easy for anyone who has knowledge of any of the various languages to come and join the project, and just focus on the bit that they'd want to use, or which they care about.
I mentioned this in passing in my last post, but I should really make more noise about some great news in the world of WebDriver: there are ports now available for ColdFusion and Ruby.
The ColdFusion port was put together by Bill Shelton, and provides a friendly way for CF developers to test their code. There seem to be two ports to Ruby, one by Dan Walters and one by Jim Benton.
Much kudos to Dan, Jim and Bill for their work! Extremely cool.
Since I got here, the weather has been shocking. The people in the office are starting to become convinced that I drag along a good dose of English weather with me. I deny it. Totally.
Still, I've not been letting the weather slow me down. It's been a busy week! I've had a chance to meet up with a few friends, met new ones that I never knew I had, avoided dropping a baby (which can only be a Good Thing), attended a talk given by Dave Thomas at Atlassian about Cloud computing (very interesting), done some surfing (I'm worse that I've ever been before), and had some lovely meals out.
I've also been working hard, and managed to demo something that I've been working on for a while on Friday. I also gave a Tech Talk on how to write unmaintainable, hard to read code (the idea being that if you know what's bad, you can avoid doing it) At some point, I'll turn the talk into a series of posts --- they'll be fun to write, if nothing else.
The other thing that's been occupying a small amount of time is playing with JNA. WebDriver has been built with a layered architecture, which should make it easier to support languages other than Java. We've seen this already with the Firefox Python-bindings, and the remote bindings written in Ruby (and the ColdFusion port, too :) The thing is, Firefox and the remote drivers are the "easy" case: you just spool data over a TCP/IP connection. Internet Explorer is a lot harder because it has a native C component too. Fortunately, we've followed the layering here too, and the JNI code we have calls down into a series of wrappers.
The thing I was trying to do was to expose these layers through a simple C library, which will make it far easier to add support for Ruby, Python and .Net. Turns out, I don't think it'll be that hard. The first step will be to rip out the JNI code and replace it with JNA.
How hard can it be? :)
A couple of people have said things about WebDriver that made me smile today:
When asked whether WebDriver was ready for prime time: "WebDriver's ready to rock and roll"
When talking about using it: "The API hasn't annoyed me yet."
'Nuff said. :)
There are a handful of guiding principles behind WebDriver, and I have Michael Tamm to thank for reminding me that I should perhaps articulate them more clearly. Michael's a committer on the WebDriver project, and he had this to say about one of the features requested by a user: I like the philosophy "Do only one thing, but do it perfect".
That's essentially a reiteration of the UNIX tool philosophy, and that's been a guiding light when talking about which features to add. But what are the other principles?
One of the earliest goals was to offer a developer-focussed API. We strongly believe that developers are best placed to build the tools that are most suitable for their team, so giving them a strong building block that they can use makes sense. This has also influenced the design of the API. We're "Object Based". That is, rather than having a dictionary style API (as exemplified by the existing Selenium API) we have a few, coarse-grained objects. However, it's not so fine grained that you get lost in the detail. Instead, we try and guide a developer into understanding what actions might make sense given the context their in without nannying them too much.
Another principle is that of providing safety. As Jason Yip pointed out, there are two sorts of safety: "actual" and "perceived" The difference between these is that one sort (actual) describes whether or not the software does what it's meant to. This is measurable and isn't open to debate. What is open to debate is the perceived safety --- do I believe that this software is doing what it should? This depends on the point of view of the person asking the question, but it's one of the major reasons why we try so hard to work with real browsers rather than simply wrapping the rendering engines.
Safety also ties into the next principle: correctness. We should model user behaviour as closely as possible, since we want to know how the app behaves when a user is attempting to interact with it. It was this principle that led to me resisting adding a direct way to execute arbitrary chunks of javascript; too often I'd seen this capability used to work around limitations in the underlying framework. By not providing the mechanism to work around them, incorrect behaviour is reported as a bug, we fix it in WebDriver and then everyone has the fixed version.
Correctness is also the reason behind the most recent changes I've started feeding into the codebase, starting with the IE driver.
There are two ways in which events, such as "keydown" or "click" may be generated by a testing framework. The first of these is to synthesize the event within the HTML DOM, and fire things off manually. These are the "synthetic events". This approach has the advantage that it's relatively easy to do, but it requires a careful eye for detail, and it's easy to get caught out by the quirks of the various browsers (for example, which browsers generate a "textinput" event?) The major drawback is that it becomes hard to handle edge-cases: what if the element is obscured by another, possibly transparent, one? What about the case where the element has 0 height or width (no major framework handles this, as far as I can see)? How about if the element is located somewhere the user will never be able to click, such as at -1000? What about if the default event is cancelled?
It's a nightmare.
The alternative to synthesized events is to generate them at the OS level. I call these "native events" in a stunning lack of originality. These are harder to produce, harder to hook into the app and also requires an eye for detail. The good thing is, though, that once they're done once, the same approach works across multiple browsers. Better still, we automatically isolate ourselves from the mess that is handling all those edge cases, and figuring out which browsers fire what events in which circumstances.
One of the constraining factors is that we'd like to be able to run webdriver tests in the background, without stealing focus, and in parallel. We've got most of this problem solved on Windows for both keyboard and mouse entry. We'll solve it on other platforms too (starting with X, I believe)
I'll post here as we fix things up, but stay tuned: this is going to get very interesting.
There are two reasons why a user may want to simulate keyboard entry when automating tests of a webapp:
The first of these typically involves testing that the various events that might be fired on the DOM ("keyup", "keypress", "keydown" and so on) are handled correctly. This generally means automating the input of letters that appear on the keyboard, perhaps chorded in some way with meta characters such as "shift". The important feature is that we need to ensure that the right DOM events are fired in the right order.
The second case is more interesting. Why would you push text into form fields? In danger of sounding like the sort of person who believes that the world is split into two types ("those who believe that the world is divided into two sorts of people and those who don't"), there are two possible reasons:
Before going any further, it's probably reasonable to ask "why am I whittering on about automating keyboard entry?" It's because I want to get this right on WebDriver. We have a single API for this (WebElement.sendKeys) that we're attempting to shoe-horn both of these use-cases into, and I'm not quite sure how to handle it.
Rather than introduce a new API, I suspect that what we'll do is take a look to see what's being typed. If the text entry can be done directly with the current (hardware) keyboard, we'll attempt to do the full keyboard thing (including all the DOM events)
If, however, you attempt to start inputting Klingon, we're going to have to be a little smarter. In the ideal world, we'd "know" how your particular browser expected to receive this data, and jump through the large number of key presses that your multi-lingual input editor would make you do. This approach would probably entail a vast series of lookup tables, lots of experimentation and a careful eye to detail to get it right. More simply, we can push the entire multibyte character into the text field and pretend it all happened with a single key press.
Personally, I lean towards the simpler approach because in the case of multibyte input, we're probably going to be testing i18n. Also, I don't want to have to be the one who figures out the lookup tables....
I'd love to hear what you think, either in the comments of this blog post, or by joining the WebDriver mailing list, because I want to get this one right.