ANN: Robot - Simulate keyboard and mouse events under X11

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

ANN: Robot - Simulate keyboard and mouse events under X11

Chris Wong
I'm pleased to announce the initial release of Robot!

Robot lets you send fake keyboard and mouse events, just like its
namesake in Java.

Only X11 systems are supported right now (via XTest), but Windows and
Mac can be added later if anyone cares.


## Features

+ Simple API (only 9 functions and 3 types)

+ Few dependencies

+ Exception safe -- unlike with other libraries, all keys and buttons
are released automatically when the robot terminates. `bracket` and
`finally` are used where appropriate.


## Links

Hackage: http://hackage.haskell.org/package/robot-1.0

Examples: https://github.com/lfairy/robot/tree/master/examples


Happy hacking!
Chris

--
Chris Wong, fixpoint conjurer
  e: [hidden email]
  w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Jeanne-Kamikaze
Looks like an interesting library. Will it be able to read pixels from a window at some point?


On 23 April 2013 06:02, Chris Wong <[hidden email]> wrote:
I'm pleased to announce the initial release of Robot!

Robot lets you send fake keyboard and mouse events, just like its
namesake in Java.

Only X11 systems are supported right now (via XTest), but Windows and
Mac can be added later if anyone cares.


## Features

+ Simple API (only 9 functions and 3 types)

+ Few dependencies

+ Exception safe -- unlike with other libraries, all keys and buttons
are released automatically when the robot terminates. `bracket` and
`finally` are used where appropriate.


## Links

Hackage: http://hackage.haskell.org/package/robot-1.0

Examples: https://github.com/lfairy/robot/tree/master/examples


Happy hacking!
Chris

--
Chris Wong, fixpoint conjurer
  e: [hidden email]
  w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Niklas Hambüchen
In reply to this post by Chris Wong
Hi,

I just started using your library to move my cursor.

Is it possible that it ignores negative values in moveBy?

In other words, I can only move the cursor into one direction.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Chris Wong
On Thu, May 9, 2013 at 4:47 AM, Niklas Hambüchen <[hidden email]> wrote:
> Hi,
>
> I just started using your library to move my cursor.
>
> Is it possible that it ignores negative values in moveBy?
>
> In other words, I can only move the cursor into one direction.

I did some research, and traced this to a bug in an old (1.6) version
of xcb-proto [1]. The coordinates were declared incorrectly as Word16,
instead of Int16 as they should have been. It's been fixed in
xcb-proto since 1.7.

I've cc'd Antoine Latter, the maintainer of XHB, about this bug. Once
he uploads a new version of XHB, I'll be happy to fix it on my end.

[1] http://cgit.freedesktop.org/~keithp/xcb-proto/commit/src/xtest.xml?id=f3ae971edce37ad96ef0b8a6059c1f853e88fcf3

On Tue, May 7, 2013 at 5:18 AM, Jeanne-Kamikaze
<[hidden email]> wrote:
> Looks like an interesting library. Will it be able to read pixels from a
> window at some point?

Not sure -- I have no idea how screen capturing works in X11. Calling
gnome-screenshot should probably cover most use cases.

Chris

--
Chris Wong, fixpoint conjurer
  e: [hidden email]
  w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Tobias Dammers
On Thu, May 09, 2013 at 01:36:52PM +1200, Chris Wong wrote:
>
> On Tue, May 7, 2013 at 5:18 AM, Jeanne-Kamikaze
> <[hidden email]> wrote:
> > Looks like an interesting library. Will it be able to read pixels from a
> > window at some point?
>
> Not sure -- I have no idea how screen capturing works in X11. Calling
> gnome-screenshot should probably cover most use cases.

I just took 2 minutes looking at the source code for scrot (which is
about as simplistic a screen grabber as you'll find), and it looks
fairly simple: it uses giblib, which provides a function called
gib_imlib_create_image_from_drawable, which does exactly that. Not sure
if Haskell bindings for giblib exist already though.

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Chris Wong
In reply to this post by Chris Wong
On Thu, May 9, 2013 at 1:36 PM, Chris Wong
<[hidden email]> wrote:

> On Thu, May 9, 2013 at 4:47 AM, Niklas Hambüchen <[hidden email]> wrote:
>> Hi,
>>
>> I just started using your library to move my cursor.
>>
>> Is it possible that it ignores negative values in moveBy?
>>
>> In other words, I can only move the cursor into one direction.
>
> I did some research, and traced this to a bug in an old (1.6) version
> of xcb-proto [1]. The coordinates were declared incorrectly as Word16,
> instead of Int16 as they should have been. It's been fixed in
> xcb-proto since 1.7.

Okay, I've released a new version of Robot (1.0.1.1), that should fix
this bug. Niklas: can you try it out please?

Also, it turns out taking a screenshot is much easier than I thought.
A single call to getImage returns a list of bytes, which happens to
match exactly the internal structure used by JuicyPixels. I'll look
deeper into this when I get the time.

Chris

--
Chris Wong, fixpoint conjurer
  e: [hidden email]
  w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Niklas Hambüchen
Yes, that works now.

I have another problem though: I move the cursor at high resolution
(128 Hz) and it seems that when robot issues a command to X, it
disables (keyboard) state so far. This means that it's impossible for
me to Ctrl-C my program: Only "c" is sent all the time, me pressing
Ctrl seems to be reset with the next robot event.

On Sun 12 May 2013 16:02:06 SGT, Chris Wong wrote:

> On Thu, May 9, 2013 at 1:36 PM, Chris Wong
> <[hidden email]> wrote:
>> On Thu, May 9, 2013 at 4:47 AM, Niklas Hambüchen <[hidden email]> wrote:
>>> Hi,
>>>
>>> I just started using your library to move my cursor.
>>>
>>> Is it possible that it ignores negative values in moveBy?
>>>
>>> In other words, I can only move the cursor into one direction.
>>
>> I did some research, and traced this to a bug in an old (1.6) version
>> of xcb-proto [1]. The coordinates were declared incorrectly as Word16,
>> instead of Int16 as they should have been. It's been fixed in
>> xcb-proto since 1.7.
>
> Okay, I've released a new version of Robot (1.0.1.1), that should fix
> this bug. Niklas: can you try it out please?
>
> Also, it turns out taking a screenshot is much easier than I thought.
> A single call to getImage returns a list of bytes, which happens to
> match exactly the internal structure used by JuicyPixels. I'll look
> deeper into this when I get the time.
>
> Chris
>
> --
> Chris Wong, fixpoint conjurer
>   e: [hidden email]
>   w: http://lfairy.github.io/
>
> _______________________________________________
> Haskell-Cafe mailing list
> [hidden email]
> http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Chris Wong
On Sun, May 12, 2013 at 8:46 PM, Niklas Hambüchen <[hidden email]> wrote:
> Yes, that works now.

Excellent!

> I have another problem though: I move the cursor at high resolution
> (128 Hz) and it seems that when robot issues a command to X, it
> disables (keyboard) state so far. This means that it's impossible for
> me to Ctrl-C my program: Only "c" is sent all the time, me pressing
> Ctrl seems to be reset with the next robot event.

Can you show me the code that triggers that behavior?

--
Chris Wong, fixpoint conjurer
  e: [hidden email]
  w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Niklas Hambüchen
> Can you show me the code that triggers that behavior?

It is basically

Just connection <- connect
forever $ do
  (x,y) <- getGyroMovement
  runRobotWithConnection (moveBy x y) connection

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Chris Wong
Oh, I see now. I originally made the runRobot functions reset the
input state when the Robot finished running. That worked well for my
use case (testing GUIs), but as you have noticed, it causes
unintuitive behavior when runRobot is called at a high frequency.

In hindsight, that was a design flaw on my part: that resetting
behavior should be specified explicitly, not attached unconditionally
to every call to runRobot.

I've removed the offending code, and released it as version 1.1.
Hopefully I've ironed out the issues now :)


On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen <[hidden email]> wrote:
>> Can you show me the code that triggers that behavior?
>
> It is basically
>
> Just connection <- connect
> forever $ do
>   (x,y) <- getGyroMovement
>   runRobotWithConnection (moveBy x y) connection



--
Chris Wong, fixpoint conjurer
  e: [hidden email]
  w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Niklas Hambüchen
Awesome, that works very well, and it even made my program run faster /
with less CPU.

The reset functionality is useful, but I think optional is better. Did
you remove it entirely or is it still available?

On Tue 14 May 2013 08:25:04 SGT, Chris Wong wrote:

> Oh, I see now. I originally made the runRobot functions reset the
> input state when the Robot finished running. That worked well for my
> use case (testing GUIs), but as you have noticed, it causes
> unintuitive behavior when runRobot is called at a high frequency.
>
> In hindsight, that was a design flaw on my part: that resetting
> behavior should be specified explicitly, not attached unconditionally
> to every call to runRobot.
>
> I've removed the offending code, and released it as version 1.1.
> Hopefully I've ironed out the issues now :)
>
>
> On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen <[hidden email]> wrote:
>>> Can you show me the code that triggers that behavior?
>>
>> It is basically
>>
>> Just connection <- connect
>> forever $ do
>>   (x,y) <- getGyroMovement
>>   runRobotWithConnection (moveBy x y) connection
>
>
>
> --
> Chris Wong, fixpoint conjurer
>   e: [hidden email]
>   w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Chris Wong
I removed the functionality because I didn't really see a use for it
anymore. The `hold` and `tap` functions are already exception safe
(thanks to `bracket`), and anyone who uses the unguarded `press`
function probably wants to keep it held down anyway.

Chris


On Tue, May 14, 2013 at 12:46 PM, Niklas Hambüchen <[hidden email]> wrote:

> Awesome, that works very well, and it even made my program run faster /
> with less CPU.
>
> The reset functionality is useful, but I think optional is better. Did
> you remove it entirely or is it still available?
>
> On Tue 14 May 2013 08:25:04 SGT, Chris Wong wrote:
>> Oh, I see now. I originally made the runRobot functions reset the
>> input state when the Robot finished running. That worked well for my
>> use case (testing GUIs), but as you have noticed, it causes
>> unintuitive behavior when runRobot is called at a high frequency.
>>
>> In hindsight, that was a design flaw on my part: that resetting
>> behavior should be specified explicitly, not attached unconditionally
>> to every call to runRobot.
>>
>> I've removed the offending code, and released it as version 1.1.
>> Hopefully I've ironed out the issues now :)
>>
>>
>> On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen <[hidden email]> wrote:
>>>> Can you show me the code that triggers that behavior?
>>>
>>> It is basically
>>>
>>> Just connection <- connect
>>> forever $ do
>>>   (x,y) <- getGyroMovement
>>>   runRobotWithConnection (moveBy x y) connection
>>
>>
>>
>> --
>> Chris Wong, fixpoint conjurer
>>   e: [hidden email]
>>   w: http://lfairy.github.io/



--
Chris Wong, fixpoint conjurer
  e: [hidden email]
  w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe
Reply | Threaded
Open this post in threaded view
|

Re: ANN: Robot - Simulate keyboard and mouse events under X11

Niklas Hambüchen
Ah OK.

Thank you again for the fast fixes!

On Tue 14 May 2013 11:44:43 SGT, Chris Wong wrote:

> I removed the functionality because I didn't really see a use for it
> anymore. The `hold` and `tap` functions are already exception safe
> (thanks to `bracket`), and anyone who uses the unguarded `press`
> function probably wants to keep it held down anyway.
>
> Chris
>
>
> On Tue, May 14, 2013 at 12:46 PM, Niklas Hambüchen <[hidden email]> wrote:
>> Awesome, that works very well, and it even made my program run faster /
>> with less CPU.
>>
>> The reset functionality is useful, but I think optional is better. Did
>> you remove it entirely or is it still available?
>>
>> On Tue 14 May 2013 08:25:04 SGT, Chris Wong wrote:
>>> Oh, I see now. I originally made the runRobot functions reset the
>>> input state when the Robot finished running. That worked well for my
>>> use case (testing GUIs), but as you have noticed, it causes
>>> unintuitive behavior when runRobot is called at a high frequency.
>>>
>>> In hindsight, that was a design flaw on my part: that resetting
>>> behavior should be specified explicitly, not attached unconditionally
>>> to every call to runRobot.
>>>
>>> I've removed the offending code, and released it as version 1.1.
>>> Hopefully I've ironed out the issues now :)
>>>
>>>
>>> On Mon, May 13, 2013 at 12:49 PM, Niklas Hambüchen <[hidden email]> wrote:
>>>>> Can you show me the code that triggers that behavior?
>>>>
>>>> It is basically
>>>>
>>>> Just connection <- connect
>>>> forever $ do
>>>>   (x,y) <- getGyroMovement
>>>>   runRobotWithConnection (moveBy x y) connection
>>>
>>>
>>>
>>> --
>>> Chris Wong, fixpoint conjurer
>>>   e: [hidden email]
>>>   w: http://lfairy.github.io/
>
>
>
> --
> Chris Wong, fixpoint conjurer
>   e: [hidden email]
>   w: http://lfairy.github.io/

_______________________________________________
Haskell-Cafe mailing list
[hidden email]
http://www.haskell.org/mailman/listinfo/haskell-cafe