TLDR:
Instead of relying solely on Discord’s definition of engagement, we can better measure engagement using additional metrics including:
- Number of First Quest Tag(s) achieved
- Amount of Tips Received
- Level of Churn
Discord measures engagement using message activity, voice activity, including the number of readers and chatters in various channels. We spend so much time in Discord, this became a convenient way to consider engagement at BanklessDAO as documented here, here, and here
The metrics Discord provides are aggregates of our entire Discord,
but sometimes we want more granular data. DAO Dash allows us to captures this. Here’s a chart of the most messages sent by members in the last 30-days.
Or if we wanted to see activity over the past week, DAO Dash helps us filter for that:
Now this might raise the question,
How many weekly active members are there within the DAO by roles?
One role I’ve been looking at recently are the First Quest related roles.
We could ask, out of all the Bankless DAO members who have attained their “First Quest Complete” role tags, who sent the most messages in the past 7-days?
We can see differences in the list between these two charts showing us there could be meaningful insights to filtering by roles.
Because members can have multiple roles, it can be tricky to attribute differences in engagement levels to any specific role.
What if roles were actually indicators of engagment themselves?
We’ve shared data with the First Quest team to see how new members were progressing through the various quests, but we hadn’t considered the attainment of First Quest tag(s) themselves to be indicators of engagement.
Consider the First Quest funnel.
As mentioned in this post, although it should take new comers 15-20 minutes to work their way through the quests, we see a significant drop off after First Quest Welcome. By the time a new member completes First Quest (attain their ‘First Quest Complete tag’, there’s been roughly an 80% drop-off).
In the past 30-days, only 110 out of 693 new members attained their First Quest Complete tag (only 15% follow through).
The data indicates that completing all First Quests requires effort given how many people drop-off or fail to follow through, so we can argue that having the First Quest tags are a sign of engagement.
Another metric worth considering are Tips Received.
Earning tips indicates a member is around long enough to add value. They’re engaged enough to contribute ideas, effort, knowledge, skills and abilities to receive tips.
There are certain benefits to using Tips Received, it’s a indication of an active behavior on the part of the recipient to have contributed in some way. Economically, tips go towards helping guest pass holders become L1 and there’s no better sign of engagement than someone earning their BANK.
Now that we’ve considered progressively more behavioral indicators of engagement, from messages to completing First Quest(s) to earning tips, we might combine metrics.
Do people who go further down the First Quest funnel earn more tips?
Data from the past 7-days suggests that the further down the First Quest funnel a member goes, the more tips they’re likely to earn. We can’t make any conclusive statements (yet) and we’ll have more confidence as we get more data:
One possible explanation is that member who make the effort to go further down the First Quest funnel (i.e., complete more Quests) will, over time, display the same level of effort that earns tips through various contributions.
This might useful data for the First Quest team where First Quest isn’t just an onboarding tool, but could potentially be predictive of a new joinee’s tips earning potential (although we can’t make any conclusive statements at the moment) cc: @angyts @Behold @ThePsychGuy
Finally, our community member @JokerDelMar has created a table looking at Bankless DAO member churn, which is the opposite of engagement, another metric we’ll monitor moving forward:
TLDR:
Instead of relying solely on Discord’s definition of engagement, we can measure engagement using a mix of metrics including:
- Number of First Quest Tag(s) achieved
- Amount of Tips Received
- Level of Churn
The DAO Dash team will stabilize infrastructure to allow continued data collection so we can further explore the evolving nature of Engagement at Bankless DAO.
