Monday, December 16, 2013

Testing on Android (Part 1): Unit Tests

I've recently been doing research on Android testing.  I'm quite new to this business; automated testing has (for the longest time) felt unnecessary.  But I've been slowly convinced that the benefits outweigh the time costs - as long as you do it in an efficient manner.  To that end, I set out to evaluate the many, many Android testing frameworks to see which ones seemed to save the most time.

I began by summarizing all the tools/services I could find in a spreadsheet (let me know if I'm missing anything).  Beyond that, I am going to do a series of posts going into more detail about different parts of Android testing:
Let's start with unit testing.

Unit Tests

There's no reason you can't use normal JUnit 4 testing for Android applications... as long as you stay away from anything Android.

Normally you compile against the SDK's android.jar, which contains nothing but stubbed methods that throw exceptions when run.  When you actually upload your APK to a device, it uses the device's implementations of all those stubs.  As a result, when running normal unit tests in your IDE, you get no access to those framework implementations (instead receiving mountains of exceptions).  This is not a big deal if you're testing some simple functionality that doesn't touch Android itself.

Pros:
  • Fast and easy
Cons:
  • Cannot use any Android framework classes
The Android Testing Framework

The Android testing framework is the official method of unit testing on Android.  It loads your application onto a device, then runs JUnit-based test suites.  Since it runs on the actual OS you can use the Android framework as you normally would in your application and can conduct a series of realistic tests that way.

Ostensibly the testing framework is unit testing, but the slowness of having to fully compile and upload your app onto a device before executing any tests makes testing slow.  Plus, you have to make sure you've got a device attached or an emulator running.  As a result, I might consider the testing framework for semi-regular tests (e.g., whenever you push a new commit, or nightly tests) but I would have trouble using them while actively developing.

Pros:
  • Access to the Android framework
Cons:
  • Slow
  • Requires attached device or running emulator
  • Uses JUnit 3 (instead of the newer JUnit 4)
Robolectric

Robolectric is a project that unifies the speed of unit testing with the ability to access the Android framework.  It does this by implementing all those stubs with mocked classes.

Having tried it out, it is lightning fast and works as expected.  I'm also impressed with the amount of active development on it - this is a rapidly improving framework.  However, the active development does take a toll; documentation is a bit lacking, plus some new versions of Robolectric break things in previous versions.  Plus, it can't mock everything - for example, inter-app communication - since it's not on an actual Android OS.  That said, the benefits here far outweigh the negatives when it comes to unit testing Android.

Pros:
  • Fast
  • Can access mocked Android framework
  • Actively developed
Cons:
  • Not the true Android framework
  • Not everything is mocked
  • Lacking documentation
Conclusion

I'm a fan of what Jason Sankey said on StackOverflow about tiered unit testing: Prefer pure unit tests, then Robolectric, then the Android testing framework.  The tests get harder/slower the higher the tier, but sometimes you need it.

100 comments:

  1. What's really the difference between the Android Testing Framework (which is instrumentation testing), and Espresso and Robotium, which as far as I can tell, are just layers on top this to make doing certain things easier? Why wouldn't you want to use Robotium or Espresso to make writing your tests easier? The lines, at least it seems to me, are so blurry between what's a unit test and what's a functional test when it comes to Android.

    ReplyDelete
    Replies
    1. I think the difference is that I can use ADF for non-UI testing (that still required the Android framework). E.g., I've got a series of network calls I want to test, outside the context of a running app. Were I to be doing UI testing then I would only use Espresso or Robotium because they're designed for it.

      Delete
  2. These ways are very simple and very much useful, as a beginner level these helped me a lot thanks fore sharing these kinds of useful and knowledgeable information.
    Mobile App Development Company
    Android App Development Company

    ReplyDelete
  3. Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information. Devops Training in Chennai

    Devops Training Institute in Chennai

    ReplyDelete
  4. Nice information about test automation tools my sincere thanks for sharing post Please continue to share this post.


    Learn QTP Online

    ReplyDelete
  5. Whatever We Are Gathering the information from the blogs,we have to keep on updating it,Chicago Immediate care in Chicago.By getting them into one place.Really thanks for posting.Very Thankful for the Informative Post.Really thanks for sharing such an useful information.

    ReplyDelete
  6. Awe! What An Blog Very Helpful and interesting Really A great center for acquiring knowledge.Very Helpful Post And Explained Very Clearly About All the things.Very Helpful. Coming To Our Self We Provide Food Service Parts . Really Thankful For the blogger providing such a great information.Thank you. Have a Nice Day.

    ReplyDelete
  7. Excellent article, Cool, Looking ahead to reading a lot. Sensible article USMLE Thanks for posting.

    ReplyDelete
  8. Really Thanks For Posting Such an Useful and informative article. Mulesoft Certification Training

    ReplyDelete
  9. Thanks For Sharing Such an Informative Post.......

    Plots for Sale in Vizag

    ReplyDelete
  10. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command

    java training in omr

    java training in annanagar | java training in chennai

    java training in marathahalli | java training in btm layout

    java training in rajaji nagar | java training in jayanagar

    ReplyDelete
  11. I applaud the publication of your article on testing on android. It's a good reminder to look on the DevOps training.

    It is recommended to take DevOps training in Chennai quora

    Thank you for sharing with us the post that you have worked so hard to refine.


    DevOps training in chennai with placement | Best DevOps training in chennai | DevOps training in chennai OMR | DevOps training in chennai Velachery |DevOps training in chennai

    ReplyDelete
  12. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.
    python training in chennai | python training in bangalore

    python online training | python training in pune

    python training in chennai | python training in bangalore

    ReplyDelete
  13. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.

    rpa training in Chennai | rpa training in pune

    rpa training in tambaram | rpa training in sholinganallur

    rpa training in Chennai | rpa training in velachery

    rpa online training | rpa training in bangalore

    ReplyDelete
  14. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.

    rpa training in Chennai | rpa training in pune

    rpa training in tambaram | rpa training in sholinganallur

    rpa training in Chennai | rpa training in velachery

    rpa online training | rpa training in bangalore

    ReplyDelete
  15. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.

    angularjs Training in bangalore

    angularjs Training in bangalore

    angularjs Training in btm

    angularjs Training in electronic-city

    angularjs Training in online

    ReplyDelete
  16. Great content thanks for sharing this informative blog which provided me technical information keep posting.
    DevOps online Training

    ReplyDelete

  17. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.



    Best Selenium Training in Chennai | Selenium Training Institute in Chennai | Besant Technologies


    Selenium Training in Bangalore | Best Selenium Training in Bangalore

    ReplyDelete
  18. Good job in presenting the correct content with the clear explanation. The content looks real with valid information. Good Work

    DevOps is currently a popular model currently organizations all over the world moving towards to it. Your post gave a clear idea about knowing the DevOps model and its importance.

    Good to learn about DevOps at this time.



    devops training in chennai | devops training in chennai with placement | devops training in chennai omr | devops training in velachery | devops training in chennai tambaram | devops institutes in chennai | devops certification in chennai

    ReplyDelete
  19. This information is impressive. I am inspired with your post writing style & how continuously you describe this topic. Eagerly waiting for your new blog keep doing more.
    Ethical Hacking Training in Bangalore
    Ethical Hacking Course in Bangalore
    Hacking Course in Bangalore
    Best Ethical Hacking Institute in Bangalore

    ReplyDelete
  20. thanks for the useful information... JNTU99

    ReplyDelete
  21. Thanks for sharing this information admin, it helps me to learn new things

    mbaguide
    Article submission sites

    ReplyDelete
  22. Your blog is so inspiring for the young generations.thanks for sharing your information with us and please update more new ideas.
    Selenium Training in OMR
    Selenium Training in Sholinganallur
    Selenium Training in Amjikarai
    Selenium Training Institutes in Vadapalani

    ReplyDelete
  23. This is a terrific article, and that I would really like additional info if you have got any. I’m fascinated with this subject and your post has been one among the simplest I actually have read.

    Microsoft Azure online training
    Selenium online training
    Java online training
    Java Script online training
    Share Point online training


    ReplyDelete
  24. great information.
    It was such an informative post.
    thanks for sharing.

    Best software testing Training in Bengaluru

    ReplyDelete
  25. Attend The Python training in bangalore From ExcelR. Practical Python training in bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Python training in bangalore.
    python training in bangalore

    ReplyDelete
  26. I was very contented with their packing and moving services. Their purity, well-organized and disciplined services helped me a lot in getting my goods securely at the destination. Just go for their qualitative services. Their service providing mechanism was renewed with the latest machines and technologies. They won the heart by delivering the right information related to goods from time to time on the way. I just want to say just go and approach them.

    Agarwal Packers Reviews
    Agarwal Packers Feedback
    Agarwal Packers Complaint

    ReplyDelete
  27. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.
    Machine Learning Training in Chennai |Machine Learning Training Institute in Chennai

    ReplyDelete
  28. Its really helpful for the users of this site. I am also searching about these type of sites now a days. So your site really helps me for searching the new and great stuff.sap fiori Training in Bangalore

    ReplyDelete
  29. I gathered a lot of information through this article.Every example is easy to undestandable and explaining the logic easily.sap gts Training in Bangalore

    ReplyDelete
  30. Such a great information for blogger iam a professional blogger thanks…

    Get Best Service Now Training in Bangalore from Real Time Industry Experts with 100% Placement Assistance in MNC Companies. Book your Free Demo with Softgen Infotech.

    ReplyDelete
  31. It focuses on your responsibilities and safety when maintaining electricals on a daily basis. cursos de ti

    ReplyDelete
  32. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing.... sap bi tutorial

    ReplyDelete
  33. What a great site i really happy to read this post thanks sharing.....Sarkari Job are an attraction to a huge Indian population. Whenever one looks for a settled future, we tend to look at SSC Jobs, Railway Jobs, Bank Jobs, Defence Jobs and Civil Service Jobs..

    ReplyDelete
  34. Whatever we gathered information from the blogs, we should implement that in practically then only we can understand that exact thing clearly, but it’s no need to do it, because you have explained the concepts very well. It was crystal clear, keep sharing..

    sap bw tutorial

    ReplyDelete
  35. Effective blog with a lot of information. I just Shared you the link below for ACTE .They really provide good level of training and Placement,I just Had Angular JS Classes in ACTE , Just Check This Link You can get it more information about the Angular JS course.

    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  36. Thanks for giving great kind of information. So useful and practical for me. Thanks for your excellent blog, nice work keep it up thanks for sharing the knowledge. very happy to see this page
    Ai & Artificial Intelligence Course in Chennai
    PHP Training in Chennai
    Ethical Hacking Course in Chennai Blue Prism Training in Chennai
    UiPath Training in Chennai

    ReplyDelete
  37. interesting piece of information, I had come to know about your web-page from my friend, i have read atleast eight posts of yours by now, and let me tell you, your blog gives the best and the most interesting information. This is just the kind of information that i had been looking for, i'm already your rss reader now and i would regularly watch out for the new posts, once again hats off to you! Thanks a million once again, Regards,
    Salesforce Training in Chennai | Certification | Online Course | Salesforce Training in Bangalore | Certification | Online Course | Salesforce Training in Hyderabad | Certification | Online Course | Salesforce Training in Pune | Certification | Online Course | Salesforce Online Training | Salesforce Training

    ReplyDelete
  38. Excellent blog. Lots of useful information here, thanks for your effort!
    Real Estate plots in Vizag

    ReplyDelete
  39. Very Informative blog thank you for sharing. Keep sharing.

    Best software training institute in Chennai. Make your career development the best by learning software courses.

    devops certification in chennai
    rpa training in chennai
    best msbi training institute in chennai

    ReplyDelete
  40. Here is the best Software training institute to study AWS training in Chennai Infycle Technologies and we also provide courses like Power BI, Cyber Security, Graphic Design and Animation, Block Security, Oracle, Java, Data Science, Big data, AWS, Python, etc., After the completion, of course, students will be able to crack their jobs in top MNC's. For more queries dial 7504633633.

    ReplyDelete
  41. Your very own commitment to getting the message throughout came
    to be rather powerful and have consistently enabled employees just
    like me to arrive at their desired goals.
    mysql training in chennai
    unix training in chennai

    ReplyDelete
  42. Amazingly by and large very interesting post. I was looking for such an information and thoroughly enjoyed examining this one. Keep posting. An obligation of appreciation is all together for sharing.data science training in gwalior

    ReplyDelete
  43. What are the rules and how do you know if I can win
    That is the biggest barrier to 공주 출장마사지 winning real money 평택 출장마사지 and playing 천안 출장마사지 at a casino is that the payout percentages are very low. But if 상주 출장마사지 you want to 김포 출장샵 get a big payout,

    ReplyDelete
  44. This post is so useful and informative.Keep updating with more information.....
    Python Training In Bangalore
    Python Course In Bangalore

    ReplyDelete
  45. This comment has been removed by the author.

    ReplyDelete