The Numbers Behind the Stars

Jim Firstbrook Engagement 0 Comments

about a 5 minute read

The numbers (and thinking) behind the Stars!

Stars are brilliant points of light. In the context of our work, they also represent the brilliance of group intelligence. First, some background. If you’ve been following our blog, you may recall a recent post on the renaming of Thoughtexchange (formerly Thoughtstream) and the three steps in our process; which are now Share, Star, and Discover. Once participants complete the Share step, where they submit thoughts on issues that matter to them, they move on to what our customers say is their favorite part – and most certainly the single, biggest differentiator of our engagement process – the Star feature. “Starring” enables stakeholders to consider and communicate which ideas are most important to them.

In addition to all the positive feedback we got on our Star step from participants, we also received a number of inquisitive questions, which inspired us to share the thinking behind our Stars with you here.

Verbify: Share. Star. Discover.

Starring before Converge

Last April, based on user input, we changed how we handle Thoughts. Previously, Facilitators converged, or grouped, Thoughts before participants assigned Stars. However, the process of grouping Thoughts introduces Facilitator bias, before participants were able to see the diversity of viewpoints. With this in mind, participants now see and prioritize each other’s “raw thoughts” before they are grouped together by our Facilitators. Those who’ve experienced this change say it feels more honest, authentic and informative; increasing the opportunities for real community learning. And we couldn’t agree more.

Priorities before Converge

Less is more

Another piece of very important feedback we received from our users was to be careful not to overwhelm them with too many Thoughts at a time. It would be unrealistic to expect participants to prioritize the thousands of individual Thoughts garnered from our engagements, without being overwhelmed with the feeling that there is just too much to consider. For example, if 500 participants in one school each enter 3 thoughts that’s 1,500 Thoughts to be viewed from just one question.​ At the same time though, we didn’t want to reduce the number of Thoughts by grouping them together, as doing so would remove valuable information and effectively silence the genuine voice of the participants. ​Like a town hall meeting, we believe it’s crucial for participants to hear (or in our case, read) the actual raw thoughts of other participants.

This thinking led us to “subsetting”, whereby each participant sees a randomly selected subset of Thoughts in addition to their own. Our software algorithm ensures all Thoughts are seen, as much as possible, an equal number of times. And with research suggesting that people are better at processing smaller chunks of information, we believe that showing each participant a subset of Thoughts, rather than every single Thought shared, makes good sense. Furthermore, while participants might not see each and every Thought put forth, our new patent-pending software adjusts the Star values during the grouping process to ensure that participants cannot unfairly influence the results by assigning many Stars to similar Thoughts – ensuring that unique ideas shine through.

So What’s the right combination? We believe it’s 9-16-5-3.

Well, ok, up until the very recent release of version 3.1.1 – which will officially roll out to customers in January – for many years we actually thought it was 9-7-5-3. But that’s another story we’ll explain in a minute!

After years of experimenting, we believe a 9-16-5-3 formula is optimal.

9 Thoughts to a page
Participants are presented with 9 Thoughts at a time.

16 Stars per page
Participants can assign up to 16 Stars per page, each containing 9 Thoughts.

5 Stars per Thought
Although participants can assign up to 16 Stars per page, only 5 of the 16 Stars can be assigned to an individual Thought.

3 pages of Thoughts
In total, each participant will be presented with 3 pages of Thoughts, each with 9 Thoughts per page, for a total of 27 Thoughts.

In summary, we designed the Star application to be as simple and user-friendly as possible, to maximize participation and the assignment of Stars. With that in mind we limited the Thoughts per page to 9 – an amount that also fits nicely on an iPad for our mobile users. We think putting more Thoughts on a page would reduce participation and also reduce the participants’ ability to accurately prioritize all the Thoughts. Similarly, we believe 27 Thoughts is the right balance per question for each participant to view. Any less and they aren’t seeing a significant sample, any more and it becomes overwhelming, decreasing participation. And to make things easier on the eyes, rather than presenting all 27 Thoughts at once, they’re spread out over 3 pages, with 9 Thoughts on each.

Finally, this brings us to the number of Stars participants are given to assign on a given page. Think of the assigning of Stars as a “passion meter” of sorts. The more Stars assigned to an individual Thought, the more passionate community members are about it. We used to think the right amount of Stars per page was 7, but have recently increased this to 16. Why? Because we found participants were so passionate about assigning Stars, they wanted more of them. Because they didn’t want to worry or second-guess their Star placement, with what they felt was such a small ration. And because we didn’t want participants to be frustrated having to choose. So we increased the Star count from 7 to 16, knowing participants will feel more liberated in assigning their Stars, with the hope that we’ll see more Stars assigned to fewer Thoughts – indicating passion, rather than fewer Stars being assigned to more Thoughts – indicating support.

Have we got it right?

Until our experience indicates otherwise, we feel that our 9-16-5-3 formula provides the best opportunity for the highest levels of participation and simplicity; while ensuring that all points of view are seen and considered, letting unique ideas shine through. Using Thoughtexchange, the collaborative leaders we work with are able to provide their stakeholders with greater group insight and greater group agreement, compared to traditional listening methods such as town halls. And while no tool is perfect, we believe our process still allows for the solitary voice with a brilliant idea to be heard. Just like in a town hall though, the larger the meeting the more competing voices there are. Fortunately for the leaders we assist, it also means that more brilliant ideas will be voiced and that a single brilliant voice will be recognized as such.

Tell us what you think. We’d love your input. Or if you want to see how the Star application works, we’d love to show you that too.

P.S. Stay tuned for our next article on the new features of the Star App, designed to make it even easier for participants to navigate and assign their Stars!

About the Author

Jim Firstbrook

Jim is the co-Founder, President and COO of Thoughtexchange. Jim had 27 years of software engineering experience prior to founding Thoughtexchange, 15 of these years with product vision, architecture and management responsibilities. He has held positions with MacDonald Dettwiler, Creo Products and Kodak Graphic Communications. Since university, Jim has had parallel careers in outdoor education, mountain guiding and organizational development. Thoughtexchange combines his passions for software and people in a single product.

Share this Post

Leave a Reply

Your email address will not be published. Required fields are marked *