One of the presenters at Droidcon UK 2013 made the point that View.findViewById() has been unjustly villainized and that the ViewHolder pattern recommended by Google is unnecessary. After hearing that I studied the code (in View and ViewGroup) and did not observe any heinous performance issues contained therein. It's a pretty simple recursive lookup that walks the View hierarchy for an id.
As this article points out, findViewById() is necessarily slower than ViewHolder because it's O(n) vs. O(1). But that doesn't mean findViewById() is slow. If the operation is sufficiently fast and n is low then it doesn't matter if it's O(n). Who cares if I have to do a 1 nanosecond operation a thousand times?
My hypothesis is that findViewById() is fast enough as to be negligible. With that in mind, I cooked up a sample application that tests findViewById(): https://github.com/dlew/android-findviewbyid
You can dynamically create a hierarchy of Views, either adding to the depth or the number of children at each node. When you hit "run" it searches for the furthest away id a number of times, then averages the time taken. I ran the test on my Nexus 1, to somewhat recreate the situation described when Google explicitly recommended the ViewHolder pattern three years ago.
Here's the results from a test of many possible depth/views per node values: http://goo.gl/dK2vo3
As can be seen from the chart, the time taken to use findViewById() is roughly linear with the number of Views in the hierarchy. What's key here, though, is that for a low number of Views it barely takes any time at all - usually a matter of several microseconds. And remember, this is on a rather old phone, the Nexus 1.
That said, if you were to call findViewById() many, many times it could cause a problem. Admittedly this part of the post is going to be a bit more hand-wavy than the rest, but: I tried out scrolling on a ListView on my Nexus 1 - there is an upper bound to how fast I can scroll and I can't seem to break 60 getViews() per second. That means that getView() is called once per 16ms frame. Given that we're measuring findViewById() in microseconds, even calling it a dozen times (with a reasonable number of Views) shouldn't cause performance issues.
So my conclusion is that findViewById() is harmless in most practical circumstances. I'll probably still use ViewHolder to avoid casting Views all the time, but performance will not be the main purpose anymore.
Disclaimer: I'm not a performance expert so I suspect there's something I'm missing (especially since I'm going directly against Google advice here). If someone knows of a missing link let me know!