Playing Outside Your Sandbox: Developing a Custom Research Application

By Carey Rellis and Jessica Spencer, Good Karma Consulting, St. Joseph, Michigan, carey@goodkarmaconsulting.com, jessica@goodkarmaconsulting.com

The thing about growing up with Fred and George,” said Ginny thoughtfully, “is that you start thinking anything’s possible if you’ve got enough nerve.”
– J.K. Rowling, Harry Potter and the Half-Blood Prince

Carey Rellis and Jessica Spencer, Good Karma Consulting

Although J.K. Rowling’s words apply to Ginny pushing Harry to meet the wizard, Sirius, they could just have easily applied to the situation I found myself in more than a decade ago along with Jessica Spencer, my colleague. If you are like us, you too may have found yourself dissatisfied with existing tools or techniques for your research practice—even with the impressive array of digital tools currently available. Yet, this story begins back when we had far fewer solutions.

At the time, we were (and still are!) passionate qualitative researchers with little interest in or experience with technology. Our motivation sprang from the desire for human connection, joyfully employing tried and true projective, storytelling techniques, and projectives to elicit underlying truths with consumers. Like many of you, collage was considered a go-to technique for our qualitative research practice, but we realized that traditional collage posed several creative and logistical challenges.

Respondents would forget their homework and arrive empty-handed to the focus group. Others would seemingly spend more time poring over publications in search of inspiration. Even the most articulate and artistic participants often included images carefully culled for their pages or poster that were lacking in depth, emotion, and insight—being literal and surface-level—leaving us at ground zero in our excavation into their behaviors and motivations. Even over time, as Google became a go-to source for images, respondents would type the topic into the search engine and show up printouts of someone else’s vision for the collage themes.

After one too many subpar experiences, we decided to create our own tool that would make collage experiences better for the moderator, respondents, and the clients. We knew nothing about how to create a technology application and even less about what the journey would have in store for us. But from our frustrations, we were inspired to create karmaCollage 1.0.

The Path to Progress

Preparing an RFP

The first, and arguably most important step, is preparing an RFP. It requires a tremendous amount of thought regarding the tool and exactly what you need it to do. Clarity in this step is crucial to obtain accurate bids. You may benefit from bouncing ideas off a colleague in the application development world who can help you articulate your goals and aspirations for your project in a way that enhances clarity for the supplier you choose.

Finding Developers

For karmaCollage 1.0, we found three domestic companies via a simple Google search and asked them to provide detailed bids. A few years later, however, we had outgrown the technology and platform. We discovered additional levels of access and enhanced features we wished to incorporate. Our lead developer from the original supplier shop was no longer available, and we cast our net farther afield to Asia using tools like Upwork, Freelancer, and Fiverr to find developers.

Keep in mind that particularly in countries with less expensive labor, competition is so fierce among development companies that they seem to be solely focused on winning the work, even before they fully understand what you need. We sought international partners for karmaCollage 2.0 and tried to share with them flowcharts and visuals to fully explain what we needed the tool to accomplish, but were confidently dismissed—until the work began. Ultimately, our partner delivered a working solution, but it was missing some of the key features we had expected. The hours spent driving to alignment and covering necessary revisions easily surpassed any cost savings benefits we had anticipated during the supplier selection process.

Test Early and Often

Plan to spend a tremendous amount of time and energy putting the application through its paces. Test every conceivable usage scenario, and enlist friends and family to help. The goal is to uncover “bugs,” get them fixed, and not have the fix cause any additional problems.


After launch, take time to revel in the excitement of bringing something new and uniquely yours to the world. For those of us accustomed to written report deliverables, creating a standalone technology—no matter the type—is no small accomplishment.

The Maintenance Plan

Here lies a hidden source of ongoing expense and required effort. Someone will need to be designated, hired, or retained to manage updates to licenses, privacy compliance, security, patches, bug fixes, and the list goes on and on. Jessica’s expertise and experience meant that maintaining karmaCollage was not overly time-consuming, but it did detract attention from other projects and occasionally could end up deprioritized.

To Sell or Not to Sell?

So many researchers we have met tout proprietary methods and techniques to which we shrug, smile, and conclude that we all have our personal spin on approaches. Some advised against us sharing the digital collage platform with others, but we were enamored with the concept of another revenue stream, and the management of it seemed to complement Jessica’s project support/non-traveling role. For us, it was a win-win. It would not detract from our core business of custom research projects, and when asked for consultation on projects, it was exciting to see the array of topics and approaches our colleagues employed. The platform provided another idea exchange opportunity and chance to network within our cherished QRCA circle.

Yet, despite the allure of a separate revenue stream, selling to others does have its downsides. For us, the challenge was balancing the necessary marketing and customer support efforts. Sometimes it required a dedicated, in-house person to manage maintenance, patches, and security fixes. Other times it did not. When we needed updates, we often relied upon the initial developer who became a freelancer soon after our launch. However, it became cost prohibitive to work with him on an hourly basis, and our needs seemed too small to keep someone or a technology company on retainer.

Looking Back and Thinking Ahead

Today, karmaCollage remains a reliable asset for Good Karma as well as other qualitative and quantitative researchers and has expanded into other realms through creative problem-solving colleagues, facilitators, and human resource applications. It is a valuable tool that we return to regularly for projects, especially now that nearly all of our primary research is conducted online.

Would we build karmaCollage again today? Probably not, particularly considering the vast array of tools and platforms now available to researchers. We would get creative with existing options and meet with suppliers who are often eager to share ideas and help you employ their tools to meet your needs. Unless you are confident you have a mind-blowing new approach, what you seek may already be waiting for you. Still, if you’ve thoroughly searched without finding what you are looking for and want to develop your own custom solution, take our advice:

  • Outline exactly what you want the tool to do in detail.
  • Gather multiple bids via virtual marketplaces, but look beyond the proposed price, and ensure each supplier understands the scope. Be wary of any company that dismisses your requests for detailed conversations about scope. Balance what you are willing to spend and how much you are able to invest in hand-holding to get what you need. The lowest bidder will likely require more man hours and hand-holding than you or they had envisioned.
  • Take advantage of free advice by offering your prototype and beta applications to fellow qualitative researchers free of charge.
  • Plan for the future. After a couple of years using karmaCollage, we identified additional features, access levels, and customization opportunities we wanted to implement. In addition, mobile was becoming more important, and our platform ran on Adobe Flash, which had become obsolete. We redoubled our efforts and, (nearly) our investment to rebuild the platform in a more relevant format.
  • Confirm your commitment and develop stamina for your journey. Constantly changing privacy laws and evolving coding languages mean you are never done. Data security issues demand you have a secure host for your application and that protocols are in place to maintain compliance.
  • Know that things will go wrong. You (and your clients) will need fixes in a pinch. Have a plan for these contingencies.

When asked to write this article, we were rejuvenated with pride in what we accomplished with karmaCollage. Retracing our steps along the path we took, the funds we spent, and the countless hours we invested during development was eye-opening. Although we still wonder whether we would do it all again, we would not trade the experience for anything.

Be the first to comment

Leave a Reply