It's really important to test your applications with real data. I learned this when I worked in QA for Ramsoft Inc. I found it much easier to spot problems with the product when there were real data values in the application. Certainly you don't need real values to spot glaring errors like segfaults or something, but to find the more subtle bugs, having real data really helps. Some usability problems become very transparent when you use real data instead of "ksodaguhkudhgau".
When you use real data, you put yourself in your user's shoes. Having this perspective on the application really helps create a better application. You can solve a lot of usability issues by using real data during development and testing, instead of discovering them when you put the application in front of your users for the first time.
To this end, when I do user interface development at Karos Health, I try to use real data as much as possible. This helps me find ways to improve the UI to make it more usable, and helps me understand the high level purpose of the application better. It also makes for much better demos at the end of our sprints. :)
On the flip side, only using real data for tests can lead to you missing a lot of problems, since the inputs that cause the problems may only appear rarely in real life. Using mock data allows you to test edge cases and create unit tests for scenarios that would be hard to reproduce in a live system.
ReplyDeleteOf course. Testing with real data should by no means try to replace testing with edge cases. I would just say that you should have real data in the system "most of the time". Also, nonsensical keyboard mashing isn't really edge case testing anyway.
ReplyDelete