Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Mario Zechner » Tue, 17 May 2011 03:59:00 GMT


Sponsored Links
 A user reported an ArrayIndexOutOfBoundsException today in our multi-
touch handler. We store x/y/touchState on a per pointer id basis in an
array. Dirty? Yes, but it worked on all devices so far. Usually
pointer ids are handed out like this:

first finger goes down -> pointer Id 0
second finger goes down -> pointer id 1
second finger lifted
second finger goes down -> pointer id 1
second finger lifted
second finger goes down -> pointer id 1

This works on all phones and some discussions in this group also point
out this exact behaviour. Sadly, MotionEvent is underdocumented so it
seems this was an assumption after all, not a rule all manufacturers
would follow. On the Sony Xperia Play the following behaviour is
observed.

first finger goes down -> pointer Id 0
second finger goes down -> pointer id 1
second finger lifted
second finger goes down -> pointer id 2
second finger lifted
second finger goes down -> pointer id 3

Is this a bug in the Xperia touch driver or is this actually wanted
behaviour? Did we all (and there are a couple of people i know of
having this issue) created faulty apps based on a wrong assumption?

On a related note: The MotionEvent documentation has a new addition
which reads like this:

"The view implementation should be prepared to handle ACTION_CANCEL
and should tolerate anomalous situations such as receiving a new
ACTION_DOWN without first having received an ACTION_UP for the prior
gesture."

The handling of ACTION_CANCEL is a no-brainer of course. What is
making me a bit nervous is the fact that spurious ACTION_DOWN events
can now happen. It is not clear whether this will only happen if a
parent View consumed the accompanying ACTION_UP event or if this is
behaviour to be expected even in single View applications.

Any pointers would be appreciated, no pun intended.

-- 



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by MichaelEGR » Tue, 17 May 2011 06:57:14 GMT


 ugh!  Thanks for posting this message as this is a gnarly one and an
example of ODM fragmentation if pointed IDs are not indexing correctly
to the proper point data at least. I never was impressed by the
multitouch API and always thought it wasn't thought out well and
simply just tacked onto MotionEvent. Dirty as you say it is correct. I
also wasn't pleased with the lack of documentation and unclear
contract. Perhaps this lack of documentation led to this potential ODM
fault. As for spurious ACTION_DOWN events this could be from varied
quality of touchscreen hardware too.

So from my reading of your post you are saying that pointer IDs are
not recycled and that the data of subsequent pointer IDs is not
correct at the specified array index?

Not looking forward to coming up with a solution for my platform /
middleware / custom event system and certainly am not excited if I
need to buy an Experia Play just for this, but w/ talking with Robert
G. it sounds like button support on this device is a bit wacky too. If
I understand correctly button presses don't generate key events? Hrm;
I hope that is not the case as I hate to have to waste $500+ to fix
these kinds of issues. Likely I'll pick one up, fix the issues, then
sell it.

Regards,
--Mike




-- 


Sponsored Links


Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Mario Zechner » Tue, 17 May 2011 10:09:07 GMT


 I don't have direct physical access to an Xperia myself so i can't
comment on the button issues Robert discovered. It looks indeed as if
Sony was brewing their own special kind of "fragmentation". I have a
kind issue reporter that helps us test. That's of course a very
"remote" kind of debugging, but it's better than nothing.

whine, whine, whine :) i found a temporary workaround, but it's
fuglier and a tad bit slower than what we had previously. I basically
allocate my own pointer ids and associate them with the "real" pointer
ids as reported in the MotionEvents. This way i can guarantee our
semantics that go like "a finger going down on the screen will receive
the first free pointer id, where pointer id is in the range 0 to
#supportedfingers -1"





-- 



Re: Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Michael Leahy » Tue, 17 May 2011 10:26:10 GMT


 >"It looks indeed as if Sony was brewing their own special kind of
'fragmentation'"

I've come to a succinct definition of "fragmentation" at least from a
developer perspective as unfortunately this term is widely "misused and
abused" in the press and sometimes twisted in definition by organizations
that produce it.

"Fragmentation is what happens when OS and device differentiation fails to
honor standards and contracts of developer APIs."

This covers faults in various OS versions and various ODM faults as well at
least from a developer perspective.  OS / device differentiation should be
celebrated; fragmentation as defined above... not so much.

--Mike





-- 



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Mario Zechner » Tue, 17 May 2011 20:51:32 GMT


 So, i wondered why we had that assumption. Back in 2009/early 2010
when 2.0 came out and the Droid blew away the world this group was
full of multi-touch related topics. Dianne Hackborn answered ALL our
questions back then, a feat that probably took up all her work
time :P. She also explained how pointer ids are handed out by the
system.

original message: 



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Robert Green » Wed, 18 May 2011 03:25:22 GMT


 Testing on my xperia play shows that it goes up to Pointer ID 15 and
then starts over again!  Why 15?  No clue...

This is inconsistent behavior when compared to all other multitouch
Android devices.  I'm going to end up implementing the same remapping
solution for all my multitouch games and frameworks.



> message:



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by MichaelEGR » Wed, 18 May 2011 03:38:42 GMT


 


WHAT? You mean you don't use your toes? ;P


Yeppers.. doh.. meh... I suppose it's not the end of the world, but
yep I've got things isolated so I can just load a custom input filter
component for my event system to do the remapping just for this
device.

Can you please enlighten on the button matter for Java API support? I
recall you mentioning something about them only accessible on the
native side? It would be nice if they generated key events; please say
it's so!

BTW happy bday Rob!

-- 



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Robert Green » Wed, 18 May 2011 05:11:01 GMT


 The regular buttons fire key events (up/down/left/right/sony shapes/L/
R/select/start) so that is good but then there are these two magical
analog pads that are a lot like the analog sticks on a PS3 controller
and IIRC they are only accessible via a native library (though I
haven't dug for the code to access them so I don't know yet how to use
them).  Perhaps they will be supported as part of the peripherals in
future Android APIs but for now I can only imagine that a person will
need to link to a sony library and access via native so that means
some JNI for Java-side support.






-- 



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Robert Green » Wed, 18 May 2011 06:32:50 GMT


 Actually, it's not just native, but requires a native activity...
 http://developer.sonyericsson.com/cws/download/1/921/870/1297178992/Accessing_Touch %20Events.pdf

I don't plan on supporting it until they can provide a normal native
lib or header or Java access.






-- 



Re: Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Jeffrey Brown » Wed, 18 May 2011 06:41:26 GMT


 Pointer IDs are arbitrary integers.  The only guarantee is that they
will be unique for a given finger as long as that finger remains down.
 Once the finger goes up, the id may be reused for a different finger,
or it may not.  It really depends.

The problem with spurious ACTION_DOWN events is strictly a software
issue, not hardware.  It happens when applications filter or modify
the stream of touch events in a manner that causes them to become
incomplete or inconsistent.  The bugs can be quite subtle.

Jeff.





-- 



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Jeffrey Brown » Wed, 18 May 2011 06:42:09 GMT


 ointer ids should be considered to be arbitrary non-negative
integers.

The framework currently makes some assumptions about the range of
these ids, but applications must not, lest they be broken sometime in
the future. Imagine a device that supports 60 simultaneous finger
touches. It could happen. :)

Moreover, the order in which pointer ids are assigned is completely
arbitrary. It happens to be the case that on most devices, the first
pointer id assigned is 0. However the first pointer id could be
anything.

This is especially the case when the Honeycomb touch splitting feature
is enabled (as it is by default for new applications). Suppose the
user puts two fingers down within two different scrollable list views.
The first finger might have id 0 and be delivered to a scrollable
list view on the left. The second finger might have id 1 and be
delivered to a scrollable list view on the right. From the
perspective of the right hand list view, the first pointer id it
received was 1, not 0.

So I'm sad to say that many applications are making incorrect
assumptions about pointer id values. One common bug is to conflate
the meaning of the pointer id and pointer index values... crash!

I strongly recommend that you track pointers using a sparse data
structure like a map or android.util.SparseArray.


Now let's talk about MotionEvent.ACTION_DOWN. The framework
guarantees that motion events will always be delivered to a window in
a consistent sequence: down/move/up, down/cancel,
down/pointerdown/move/pointerup/up.

However, the framework cannot work around buggy views that drop or
corrupt part of the motion event stream before passing motion events
down to their children.

I've seen some custom views that do bad things like eat all multitouch
events, or intercept some part of the gesture without canceling the
rest. OnTouchListener implementations can also be problematic if they
consume part but not all of a gesture. Views downstream may receive
weird sequences of events as a result.

As a defensive coding practice, views should be implemented to always
reset themselves to a known ground state before handling ACTION_DOWN.
In this way, they can recover in case some of their ancestor views
fail to pass along the ACTION_UP or ACTION_CANCEL as would be
expected.

Likewise, bad things can happen during a gesture such as a pointer id
mysteriously vanishing from the event without a an ACTION_POINTER_UP
ever have been seen (because it was consumed somewhere else). Views
should always check the result of MotionEvent.findPointerIndex and
handle the case where a pointer went missing. Otherwise, the
application may crash when it tries to access a pointer with index -1.

Clearly if any of these things happen, there's a bug somewhere
(probably in the app). It might not be very obvious though. With a
little case, the app should be able to recover gracefully.

Hope that helps,
Jeff.

On May 16, 12:58pm, Mario Zechner <badlogicga...@gmail.com> wrote:

--



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Jeffrey Brown » Wed, 18 May 2011 06:42:10 GMT


 Pointer IDs are arbitrary integers.  The only guarantee is that they
will be unique for a given finger as long as that finger remains down.
 Once the finger goes up, the id may be reused for a different finger,
or it may not.  It really depends.

The problem with spurious ACTION_DOWN events is strictly a software
issue, not hardware.  It happens when applications filter or modify
the stream of touch events in a manner that causes them to become
incomplete or inconsistent.  The bugs can be quite subtle.

Jeff.





-- 



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Mario Zechner » Wed, 18 May 2011 07:29:54 GMT


 effrey, thanks for your excellent answer, that cleared everything up!

The confusion many people have concerning the pointer id versus
pointer index issue is probably due to the wrongly named mask
constants (ACTION_POINTER_ID_MASK or some such) in the original touch
APIs. While those are now deprecated i know of a lot of people that
still can't figure out the meaning behind that.

That we all assumed that pointer ids would be handed out in the "first
free pointer id is returned" kind of way is probably due to the
dicussions we had on here, including Android engineers. I recognize
that a lot has changed since then, and your use case scenario of touch
splitting clearly illustrates the difference to the "old" ways. Makes
total sense as well.

I'd strongly suggest to either write a new dev blog entry on multi-
touch that clearly states all your points in your post or updating the
MotionEvent documentation which is still very ambiguous.

Thanks again for the answers, much appreciated!

On 17 Mai, 22:42, Jeffrey Brown <jeffbr...@android.com> wrote:

--



Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by MichaelEGR » Wed, 18 May 2011 09:14:20 GMT


 @Jeffery. Thanks for the updated info. I definitely second Mario's
suggestions of writing a clear and informative post on the dev blog
and update MotionEvent documentation to state the things you did here.
Otherwise the contract is a bit of a mystery and there is no way for
us to know about changes either if it's not documented all the way. I
too was handling it by "first free pointer id is returned" kind of way
based on implementation tips and discussion on this list when the API
first appeared.




-- 



Re: Re: Multi-touch pointer ids on Sony Xperia Play & spurious ACTION_DOWN events

by Leigh McRae » Wed, 18 May 2011 23:15:59 GMT


 

I've seen some custom views that do bad things like eat all multitouch events, or intercept some part of the gesture without canceling the rest. OnTouchListener implementations can also be problematic if they consume part but not all of a gesture. Views downstream may receive weird sequences of events as a result.
Well then I'm crushed as I'm one of those apps :( I would expect most games that are drawing their own native controls would do this. I'm going to assume that if you don't have any child views then it's not a problem. -- Leigh McRae http://www.lonedwarfgames.com/ --



Other Threads

1. gimana kabar di citos Om

gimana neh yang mirink di citos kabarnya

-- 
"Indonesian Android Community [id-android]" 

2. MapView crushes in satellite mode if android:anydensity is set true

Hi,

On Android 1.6 WVGA device, a map view in satellite mode crushes when
the map zoom-in too much if I specify android:anyDensity="true" in the
manifest file.
But, if I set android:anyDensity="false", then the map does not crush.

I want my app to support a multiple densities and also do not want to
make the map crush.
Is there any way to do this?

I tried to control zoom level of the map, but getMaxZoomLevel() method
in satellite mode does not return a correct value.
Please help me out!!

Thanks

-- 

3. How t o make a magnifying glass On a picture?

4. Date type as a parameter on a SoapObject

5. Google TV frame capture?

6. G1 phone and update firmware 2.0-2.2 - question

7. how to access device contact book from my own application