Monday, December 16, 2013

Testing on Android (Part 1): Unit Tests

I've recently been doing research on Android testing.  I'm quite new to this business; automated testing has (for the longest time) felt unnecessary.  But I've been slowly convinced that the benefits outweigh the time costs - as long as you do it in an efficient manner.  To that end, I set out to evaluate the many, many Android testing frameworks to see which ones seemed to save the most time.

I began by summarizing all the tools/services I could find in a spreadsheet (let me know if I'm missing anything).  Beyond that, I am going to do a series of posts going into more detail about different parts of Android testing:
Let's start with unit testing.

Unit Tests

There's no reason you can't use normal JUnit 4 testing for Android applications... as long as you stay away from anything Android.

Normally you compile against the SDK's android.jar, which contains nothing but stubbed methods that throw exceptions when run.  When you actually upload your APK to a device, it uses the device's implementations of all those stubs.  As a result, when running normal unit tests in your IDE, you get no access to those framework implementations (instead receiving mountains of exceptions).  This is not a big deal if you're testing some simple functionality that doesn't touch Android itself.

Pros:
  • Fast and easy
Cons:
  • Cannot use any Android framework classes
The Android Testing Framework

The Android testing framework is the official method of unit testing on Android.  It loads your application onto a device, then runs JUnit-based test suites.  Since it runs on the actual OS you can use the Android framework as you normally would in your application and can conduct a series of realistic tests that way.

Ostensibly the testing framework is unit testing, but the slowness of having to fully compile and upload your app onto a device before executing any tests makes testing slow.  Plus, you have to make sure you've got a device attached or an emulator running.  As a result, I might consider the testing framework for semi-regular tests (e.g., whenever you push a new commit, or nightly tests) but I would have trouble using them while actively developing.

Pros:
  • Access to the Android framework
Cons:
  • Slow
  • Requires attached device or running emulator
  • Uses JUnit 3 (instead of the newer JUnit 4)
Robolectric

Robolectric is a project that unifies the speed of unit testing with the ability to access the Android framework.  It does this by implementing all those stubs with mocked classes.

Having tried it out, it is lightning fast and works as expected.  I'm also impressed with the amount of active development on it - this is a rapidly improving framework.  However, the active development does take a toll; documentation is a bit lacking, plus some new versions of Robolectric break things in previous versions.  Plus, it can't mock everything - for example, inter-app communication - since it's not on an actual Android OS.  That said, the benefits here far outweigh the negatives when it comes to unit testing Android.

Pros:
  • Fast
  • Can access mocked Android framework
  • Actively developed
Cons:
  • Not the true Android framework
  • Not everything is mocked
  • Lacking documentation
Conclusion

I'm a fan of what Jason Sankey said on StackOverflow about tiered unit testing: Prefer pure unit tests, then Robolectric, then the Android testing framework.  The tests get harder/slower the higher the tier, but sometimes you need it.

249 comments:

  1. What's really the difference between the Android Testing Framework (which is instrumentation testing), and Espresso and Robotium, which as far as I can tell, are just layers on top this to make doing certain things easier? Why wouldn't you want to use Robotium or Espresso to make writing your tests easier? The lines, at least it seems to me, are so blurry between what's a unit test and what's a functional test when it comes to Android.

    ReplyDelete
    Replies
    1. I think the difference is that I can use ADF for non-UI testing (that still required the Android framework). E.g., I've got a series of network calls I want to test, outside the context of a running app. Were I to be doing UI testing then I would only use Espresso or Robotium because they're designed for it.

      Delete
  2. These ways are very simple and very much useful, as a beginner level these helped me a lot thanks fore sharing these kinds of useful and knowledgeable information.
    Mobile App Development Company
    Android App Development Company

    ReplyDelete
  3. Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information. Devops Training in Chennai

    Devops Training Institute in Chennai

    ReplyDelete
  4. Nice information about test automation tools my sincere thanks for sharing post Please continue to share this post.


    Learn QTP Online

    ReplyDelete
  5. Whatever We Are Gathering the information from the blogs,we have to keep on updating it,Chicago Immediate care in Chicago.By getting them into one place.Really thanks for posting.Very Thankful for the Informative Post.Really thanks for sharing such an useful information.

    ReplyDelete
  6. Awe! What An Blog Very Helpful and interesting Really A great center for acquiring knowledge.Very Helpful Post And Explained Very Clearly About All the things.Very Helpful. Coming To Our Self We Provide Food Service Parts . Really Thankful For the blogger providing such a great information.Thank you. Have a Nice Day.

    ReplyDelete
  7. Excellent article, Cool, Looking ahead to reading a lot. Sensible article USMLE Thanks for posting.

    ReplyDelete
  8. Really Thanks For Posting Such an Useful and informative article. Mulesoft Certification Training

    ReplyDelete
  9. Thanks For Sharing Such an Informative Post.......

    Plots for Sale in Vizag

    ReplyDelete
  10. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command

    java training in omr

    java training in annanagar | java training in chennai

    java training in marathahalli | java training in btm layout

    java training in rajaji nagar | java training in jayanagar

    ReplyDelete
  11. I applaud the publication of your article on testing on android. It's a good reminder to look on the DevOps training.

    It is recommended to take DevOps training in Chennai quora

    Thank you for sharing with us the post that you have worked so hard to refine.


    DevOps training in chennai with placement | Best DevOps training in chennai | DevOps training in chennai OMR | DevOps training in chennai Velachery |DevOps training in chennai

    ReplyDelete
  12. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.
    python training in chennai | python training in bangalore

    python online training | python training in pune

    python training in chennai | python training in bangalore

    ReplyDelete
  13. I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.is article.
    java training in chennai | java training in bangalore

    java online training | java training in pune

    ReplyDelete
  14. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.

    rpa training in Chennai | rpa training in pune

    rpa training in tambaram | rpa training in sholinganallur

    rpa training in Chennai | rpa training in velachery

    rpa online training | rpa training in bangalore

    ReplyDelete
  15. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.

    rpa training in Chennai | rpa training in pune

    rpa training in tambaram | rpa training in sholinganallur

    rpa training in Chennai | rpa training in velachery

    rpa online training | rpa training in bangalore

    ReplyDelete
  16. This is a nice article here with some useful tips for those who are not used-to comment that frequently. Thanks for this helpful information I agree with all points you have given to us. I will follow all of them.
    Data Science training in marathahalli
    Data Science training in btm
    Data Science training in rajaji nagar
    Data Science training in chennai
    Data Science training in kalyan nagar
    Data Science training in electronic city
    Data Science training in USA




    ReplyDelete
  17. It would have been the happiest moment for you,I mean if we have been waiting for something to happen and when it happens we forgot all hardwork and wait for getting that happened.
    python training in annanagar
    python training in chennai
    python training in chennai
    python training in Bangalore

    ReplyDelete
  18. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.
    DevOps online Training

    ReplyDelete
  19. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this.

    angularjs Training in bangalore

    angularjs Training in bangalore

    angularjs Training in btm

    angularjs Training in electronic-city

    angularjs Training in online

    ReplyDelete
  20. I am so proud of you and your efforts and work make me realize that anything can be done with patience and sincerity. Well I am here to say that your work has inspired me without a doubt.
    python training in tambaram
    python training in annanagar
    python training in jayanagar

    ReplyDelete
  21. Great content thanks for sharing this informative blog which provided me technical information keep posting.
    DevOps online Training

    ReplyDelete
  22. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.
    AWS Training in Bangalore |Best AWS Training Institute in Bangalore BTM, Marathahalli
    AWS Training in Chennai | AWS Training Institute in Chennai Velachery, Tambaram, OMR

    ReplyDelete

  23. Whoa! I’m enjoying the template/theme of this website. It’s simple, yet effective. A lot of times it’s very hard to get that “perfect balance” between superb usability and visual appeal. I must say you’ve done a very good job with this.



    Best Selenium Training in Chennai | Selenium Training Institute in Chennai | Besant Technologies


    Selenium Training in Bangalore | Best Selenium Training in Bangalore

    ReplyDelete
  24. Good job in presenting the correct content with the clear explanation. The content looks real with valid information. Good Work

    DevOps is currently a popular model currently organizations all over the world moving towards to it. Your post gave a clear idea about knowing the DevOps model and its importance.

    Good to learn about DevOps at this time.



    devops training in chennai | devops training in chennai with placement | devops training in chennai omr | devops training in velachery | devops training in chennai tambaram | devops institutes in chennai | devops certification in chennai

    ReplyDelete
  25. This information is impressive. I am inspired with your post writing style & how continuously you describe this topic. Eagerly waiting for your new blog keep doing more.
    Ethical Hacking Training in Bangalore
    Ethical Hacking Course in Bangalore
    Hacking Course in Bangalore
    Best Ethical Hacking Institute in Bangalore

    ReplyDelete
  26. thanks for the useful information... JNTU99

    ReplyDelete
  27. Thanks for sharing this information admin, it helps me to learn new things

    mbaguide
    Article submission sites

    ReplyDelete
  28. Your blog is so inspiring for the young generations.thanks for sharing your information with us and please update more new ideas.
    Selenium Training in OMR
    Selenium Training in Sholinganallur
    Selenium Training in Amjikarai
    Selenium Training Institutes in Vadapalani

    ReplyDelete

  29. Thank you for taking the time to write about this much needed subject. I felt that your remarks on this technology is helpful and were especially timely.

    Right now, DevOps is currently a popular model currently organizations all over the world moving towards to it. Your post gave a clear idea about knowing the DevOps model and its importance.

    devops course fees in chennai | devops training in chennai with placement | devops training in chennai omr | best devops training in chennai quora | devops foundation certification chennai

    ReplyDelete
  30. Outstanding blog thanks for sharing such wonderful blog with us ,after long time came across such knowlegeble blog. keep sharing such informative blog with us.
    artificial intelligence and machine learning course in chennai
    machine learning with python course in Chennai
    machine learning certification in chennai

    ReplyDelete
  31. This is a nice article here with some useful tips for those who are not used-to comment that frequently. Thanks for this helpful information I agree with all points you have given to us. I will follow all of them.

    angularjs online training

    apache spark online training

    informatica mdm online training

    devops online training

    aws online training

    ReplyDelete
  32. This is a terrific article, and that I would really like additional info if you have got any. I’m fascinated with this subject and your post has been one among the simplest I actually have read.

    Microsoft Azure online training
    Selenium online training
    Java online training
    Java Script online training
    Share Point online training


    ReplyDelete
  33. great information.
    It was such an informative post.
    thanks for sharing.

    Best software testing Training in Bengaluru

    ReplyDelete
  34. I am really very happy to find this particular site. I just wanted to say thank you for this huge read!! I absolutely enjoying every petite bit of it and I have you bookmarked to test out new substance you post.
    devops online training

    aws online training

    data science with python online training

    data science online training

    rpa online training

    ReplyDelete
  35. Nice post. I learned some new information. Thanks for sharing.

    Education
    Technology

    ReplyDelete
  36. This blog is very much helpful to us. Thanks for your information

    Guest posting sites
    Technology

    ReplyDelete
  37. Attend The Python training in bangalore From ExcelR. Practical Python training in bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Python training in bangalore.
    python training in bangalore

    ReplyDelete
  38. Alleyaaircool is the one of the best home appliances repair canter in all over Delhi we deals in repairing window ac, Split ac , fridge , microwave, washing machine, water cooler, RO and more other home appliances in cheap rates

    Window AC Repair in vaishali
    Split AC Repair in indirapuram
    Fridge Repair in kaushambi
    Microwave Repair in patparganj
    Washing Machine Repair in vasundhara
    Water Cooler Repair in indirapuram
    RO Service AMC in vasundhara
    Any Cooling System in vaishali
    Window AC Repair in indirapuram

    ReplyDelete
  39. Get the best nursing services baby care services medical equipment services and allso get the physiotherapist at home in Delhi NCR For more information visit our site

    nursing attendant services in Delhi NCR
    medical equipment services in Delhi NCR
    nursing services in Delhi NCR
    physiotherapist at home in Delhi NCR
    baby care services in Delhi NCR

    ReplyDelete

  40. I like your post very much. It is very much useful for my research. I hope you to share more info about this. Keep posting!!
    Best Devops Training Institute

    ReplyDelete
  41. Excellent Blog. I really want to admire the quality of this post. I like the way of your presentation of ideas, views and valuable content. No doubt you are doing great work. I’ll be waiting for your next post. Thanks .Keep it up! Kindly visit us @Luxury Boxes
    Premium Packaging
    Luxury Candles Box
    Earphone Packaging Box
    Wireless Headphone Box
    Innovative Packaging Boxes
    Wedding gift box
    Leather Bag Packaging Box
    Cosmetics Packaging Box
    Luxury Chocolate Boxes

    ReplyDelete
  42. I was very contented with their packing and moving services. Their purity, well-organized and disciplined services helped me a lot in getting my goods securely at the destination. Just go for their qualitative services. Their service providing mechanism was renewed with the latest machines and technologies. They won the heart by delivering the right information related to goods from time to time on the way. I just want to say just go and approach them.

    Agarwal Packers Reviews
    Agarwal Packers Feedback
    Agarwal Packers Complaint

    ReplyDelete
  43. thanks for your information really good and very nice web design company in velachery

    ReplyDelete
  44. Great Explanation with lots of useful information about aws and great explanation in this blog.

    AWS Training in Chennai | SAP Training in Chennai

    ReplyDelete
  45. awesome blog it's very nice and useful i got many more information it's really nice i like your blog styleweb design company in velachery

    ReplyDelete
  46. I learned World's Trending Technology from certified experts for free of cost. I Got a job in decent Top MNC Company with handsome 14 LPA salary, I have learned the World's Trending Technology from Python training in pune experts who know advanced concepts which can help to solve any type of Real-time issues in the field of Python. Really worth trying instant approval blog commenting sites

    ReplyDelete
  47. Attend The Data Science Course From ExcelR. Practical Data Science Course Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Science Course.
    ExcelR Data Science Course

    ReplyDelete
  48. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.
    Machine Learning Training in Chennai |Machine Learning Training Institute in Chennai

    ReplyDelete
  49. Visit for Python training in Bangalore:- Python training in Bangalore

    ReplyDelete
  50. Visit for AWS training in Bangalore :- AWS training in Bangalore

    ReplyDelete
  51. This is the first & best article to make me satisfied by presenting good content. I feel so happy and delighted.By Learn Digital Marketing Course Training in Chennai it will help to get Digital Marketing Course Training in Chennai.Learn Best Digital Marketing Course Training in Chennai's professional institute to get reputed job.

    ReplyDelete
  52. Very interesting blog Thank you for sharing such a nice and interesting blog and really very helpful article.sap hana Training in Bangalore

    ReplyDelete
  53. Its really helpful for the users of this site. I am also searching about these type of sites now a days. So your site really helps me for searching the new and great stuff.sap fiori Training in Bangalore

    ReplyDelete
  54. Very useful and information content has been shared out here, Thanks for sharing it.sap testing Training in Bangalore

    ReplyDelete
  55. I gathered a lot of information through this article.Every example is easy to undestandable and explaining the logic easily.sap gts Training in Bangalore

    ReplyDelete
  56. These provided information was really so nice,thanks for giving that post and the more skills to develop after refer that post.sap apo Training in Bangalore

    ReplyDelete
  57. Your articles really impressed for me,because of all information so nice.sap security Training in Bangalore

    ReplyDelete
  58. Linking is very useful thing.you have really helped lots of people who visit blog and provide them use full information.sap wm Training in Bangalore

    ReplyDelete
  59. Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me as well as giving.sap srm Training in Bangalore

    ReplyDelete
  60. Really it was an awesome article,very interesting to read.You have provided an nice article,Thanks for sharing.sap ps Training in Bangalore

    ReplyDelete
  61. I know that it takes a lot of effort and hard work to write such an informative content like this.sap ehs Training in Bangalore

    ReplyDelete
  62. Your articles really impressed for me,because of all information so nice.angular 7 training in bangalore

    ReplyDelete
  63. These provided information was really so nice,thanks for giving that post and the more skills to develop after refer that post.sap s4 hana simple finance training in bangalore

    ReplyDelete
  64. I gathered a lot of information through this article.Every example is easy to undestandable and explaining the logic easily.javascript training in bangalore

    ReplyDelete
  65. Very useful and information content has been shared out here, Thanks for sharing it.sap hr training in bangalore

    ReplyDelete
  66. This is really an awesome post, thanks for it. Keep adding more information to this.html training in bangalore

    ReplyDelete
  67. thank you so much for this nice information Article, Digitahanks for sharing your post with us.Real Time Experts training center bangalore

    ReplyDelete
  68. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful . student review for Realtime Experts marathahalli bangalore

    ReplyDelete
  69. I have read your blog its very attractive and impressive. I like it your blog.Real Time Experts Training in Bangalore center address bangalore

    ReplyDelete
  70. Such a great information for blogger iam a professional blogger thanks…

    Get Best Service Now Training in Bangalore from Real Time Industry Experts with 100% Placement Assistance in MNC Companies. Book your Free Demo with Softgen Infotech.

    ReplyDelete
  71. keep up the good work. this is an Assam post. this to helpful, i have reading here all post. i am impressed. thank you. this is our digital marketing training center. This is an online certificate course
    digital marketing training in bangalore / https://www.excelr.com/digital-marketing-training-in-bangalore

    ReplyDelete
  72. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    top servicenow online training

    ReplyDelete
  73. Here is the details of BMIT (Medical Imaging Technology) colleges in Bangalore. If you are looking to study BMIT in Bangalore, the below will help you to find the best BMIT colleges in Bangalore.
    BSc Medical Imaging Technology Colleges in Bangalore | Medical Imaging Technology Colleges in Bangalore |

    ReplyDelete
  74. Here is the details of B.Sc Optometry colleges in Bangalore. If you are looking to study BSc Optometry in Bangalore, the below will help you to find the best Optometry colleges in Bangalore.
    BSc Optometry Colleges in Bangalore | Optometry Colleges in Bangalore |

    ReplyDelete
  75. Here is the details of B.Sc Renal Dialysis Technology colleges in Bangalore. If you are looking to study BSc Renal Dialysis Technology in Bangalore, the below will help you to find the best Renal Dialysis Technology colleges in Bangalore.
    BSc Renal Dialysis Colleges in Bangalore | Renal Dialysis Technology Colleges in Bangalore |

    ReplyDelete
  76. Here is the details of B.Sc Respiratory Care Technology colleges in Bangalore. If you are looking to study BSc Respiratory Care Technology in Bangalore, the below will help you to find the best Respiratory Technology colleges in Bangalore.
    BSc Respiratory Care Technology Colleges in Bangalore | Respiratory Care Colleges in Bangalore |

    ReplyDelete
  77. Here is the details of B.Sc Cardiac Care Technology colleges in Bangalore. If you are looking to study BSc Cardiac Care Technology in Bangalore, the below will help you to find the best Cardiac Care Technology colleges in Bangalore.
    BSc Cardiac Care Technology Colleges in Bangalore | Cardiac Care Colleges in Bangalore |

    ReplyDelete
  78. Here is the details of B.Sc Perfusion Technology colleges in Bangalore. If you are looking to study BSc Perfusion Technology in Bangalore, the below will help you to find the best Perfusion Technology colleges in Bangalore.
    BSc Perfusion Technology Colleges in Bangalore | Perfusion Technology Colleges in Bangalore |

    ReplyDelete