I am trying to use the Selenium webdriver in python (using Firefox in linux) to automate the use of a simple HTML5 application on a website. I need my Python program to click a button located in a particular spot on an HTML5 canvas, and then click and drag one or two objects around the canvas after pressing the button.

It seems that selenuim currently has no way to actually locate a button or any other object on an HTML5 canvas. First, I tried to find the coordinates on the page at which the HTML5 button appears, figure out its offset in pixels from an object on the page that selenium could find by ID, and use move_to_element_with_offset like this:

from selenium.webdrivermon.action_chains import ActionChains
actions = ActionChains(driver)
move_to_element_with_offset(elem, 50, -250)
actions.click()
actions.perform()

This didn't seem to work. It didn't raise an error, but nothing happened. It occurred to me that I could have gotten the coordinates wrong, but I couldn't figure out how to see where on the page the click was actually delivered in order to check this. Many people on various forums seem to think that the click action in selenium is unreliable anyway. Next, I found someone claiming to have been able to do what I am trying to do by running a javascript that simulates a click at a point given its coordinates on the page. see here: /

I am very good with Python, but know next to nothing about javascript. I tried something very much like what appears in this article:

x = '675'
y = '450'
driver.execute_script("var evt = $.Event('click', { pageX: " + x +", pageY: " + y + " } );" + "$('#diagramCanvas').trigger(evt);")

This caused an error:

WebDriverException: $.Event is not a function

How can I get the Python Selenium webdriver to click on a certain spot in an HTML5 canvas, given its coordinates? Is there perhaps a javascript script that won't raise an error like the one above, or some other way to do this?

I am trying to use the Selenium webdriver in python (using Firefox in linux) to automate the use of a simple HTML5 application on a website. I need my Python program to click a button located in a particular spot on an HTML5 canvas, and then click and drag one or two objects around the canvas after pressing the button.

It seems that selenuim currently has no way to actually locate a button or any other object on an HTML5 canvas. First, I tried to find the coordinates on the page at which the HTML5 button appears, figure out its offset in pixels from an object on the page that selenium could find by ID, and use move_to_element_with_offset like this:

from selenium.webdriver.mon.action_chains import ActionChains
actions = ActionChains(driver)
move_to_element_with_offset(elem, 50, -250)
actions.click()
actions.perform()

This didn't seem to work. It didn't raise an error, but nothing happened. It occurred to me that I could have gotten the coordinates wrong, but I couldn't figure out how to see where on the page the click was actually delivered in order to check this. Many people on various forums seem to think that the click action in selenium is unreliable anyway. Next, I found someone claiming to have been able to do what I am trying to do by running a javascript that simulates a click at a point given its coordinates on the page. see here: http://chariotsolutions./blog/post/automated-testing-of-html5-canvas/

I am very good with Python, but know next to nothing about javascript. I tried something very much like what appears in this article:

x = '675'
y = '450'
driver.execute_script("var evt = $.Event('click', { pageX: " + x +", pageY: " + y + " } );" + "$('#diagramCanvas').trigger(evt);")

This caused an error:

WebDriverException: $.Event is not a function

How can I get the Python Selenium webdriver to click on a certain spot in an HTML5 canvas, given its coordinates? Is there perhaps a javascript script that won't raise an error like the one above, or some other way to do this?

• javascript
• python
• html
• selenium
• browser-automation

Share Improve this question Follow asked Apr 14, 2015 at 10:26 cljclj 30322 gold badges55 silver badges1313 bronze badges

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 2

The thing you tried with javascript is using JQuery. That's why if you want use it, you should add jquery to yr page.

As for Selenium, it's bind to DOM, that's why it can't locate anything doesn't exist - as you know, canvas is just an image for a browser. Let's suppose we have canvas element in "canvas" var:

canvas = self.driver.find_element_by_id("canvas")
drawing = ActionChains(self.driver)\
    .move_by_offset(-10, -15)\
    .click(canvas)\
    .release()
drawing.perform()

If it doesn't work, you are clicking somewhere else :) You can try to determine coordinate you are clicking.

According to the docs, move_to_element_by_offset uses offsets relative to the top left corner of the element, not from the middle. Thus, the negative y-offset in your Python snippet indicates that Selenium is moving to a position above your canvas. If it's still not working, maybe updating you selenium Python bindings will fix it.