This document provides information about how to make use of the metrics dashboards for ONOS and CORD.
ONOS Dashboard: https://onos.biterg.io
CORD Dashboard: https://cord.biterg.io
These dashboards are public so there is no log-in needed. There are two administrative URLs that do require a log-in (/edit and /data). These two URLs give you access to making customizations to the dashboards and to direct query the database. If you need access to that, please ask ONF staff.
Using the dashboards
To learn about how to use the dashboards, please see the following resources.
Bitergia ran three training sessions when the dashboards were set up and those recordings are available.
- Training #1: Introduction to the Bitergia Analytics Dashboard (Video)
- Training #2: Building visualizations and panels for the Bitergia Analytics Dashboard (Video)
- Training #3: Querying and scripting the database supporting the Bitergia Analytics Dashboard (Video– starts at 1 hour in | Slides)
There are other resources available as well.
- GrimoireLab Tutorial
- How to Merge Request to a repository (useful for making changes to dashboard administrative files on Gitlab.com)
There is also a tutorial at: https://grimoirelab.gitbooks.io/tutorial
Issues with dashboards
If there is an issue with a dashboard (such as it is not collecting data from a source in the community such as a new mailing list or Slack channel), issues can be filed with the company that maintains the dashboard. Access on Gitlab.com is needed to submit issues, so you can either send your issues to someone at ONF who has access or if you are interested in helping us maintain the dashboards you can get access directly.
ONOS issue tracker: https://gitlab.com/Bitergia/c/ONOS/support
CORD issue tracker: https://gitlab.com/Bitergia/c/CORD/support
Dealing with Unknowns in the data
You may notice that you'll see some people marked as 'Unknown' in the dashboard or that you're looking for information about an organization but not seeing it. This is because data from some activities isn't connected to an organization (for instance, if someone is using an @gmail.com email address the tool won't know what organization that person is affiliated with). We've mapped most of these affiliations (as of January 2018 both the ONOS and CORD dashboards are showing less than 2% unknown when looking at the last 5 years of data) but there are still some Unknowns and new members coming into the community may be unaffiliated as well.
Helping manage these Unknowns is something that needs to be done on an ongoing basis and we welcome your help with this. If you're interested in that, we can walk you through the process and get you access to the files to update. Please note though that it isn’t possible to get this number down to 0% for a few reasons:
We don’t know everyone in the system. For some people all we have is a Gmail address or Slack handle with no other information and staff at ONF aren’t familiar with that person. This is often true with lurkers who have accounts on our systems but don’t actively participate.
There are some accounts that aren’t tied to people but are there for administrative purposes, such as admin@DevA-ONOS1.(none) or admin@cordsrv. Those are set as unknown now but we could change these to ‘Administrative’ if we wanted to lower the unknown count further
New users that we haven’t mapped yet keep coming into our tools and show up in the dashboard. Because of this on ongoing attempt to map affiliations is needed
If the small number of Unknowns is distracting (for example, if you wanted a list of all organizations that contributed to the most recent ONOS release with showing the Unknowns) you can get a view of the data that filters these small number of unknowns by searching for: “NOT author_org_name:Unknown”.
Customizing the dashboard
Another area we could use help with is to customize the dashboards with metrics that the community is interested in. The dashboard is very flexible and can be edited to meet the needs we've identified. If you're interested in that, let us know and we can give you access. And here are some of the ideas for customizations we've come up with so far:
Sub-project: Setting up sub-projects so we can see stats for part of the ONOS and/or CORD projects. For instance, seeing stats just for M-CORD.
Cohort analysis: this report would give us a regular report of everyone who had been active but had become inactive (defined as no activity within last 6 months) and that give us data around retention and would give us emails of people to reach out to so we could encourage them to come back or learn why they left
Code review metrics: We have been talking for a while about how to use data to find people who would make good new module owners. There are a variety of ways to look into the code review activities of community members and flag who is doing good work.
Event analysis: It should be possible to use the dashboard to get information about how effective events have been about onboarding new people or deepening contributions. We could use the attendee list and then match that up to activity levels in the dashboard that looked at the time period before and then after the event.
Churn chart: The following churn chart was manually put together with data from the dashboard but it should be possible to create a dynamic metrics panel that shows this information. Knowing about community churn is useful and can give us goals about getting more community members contributing over more releases.
Community multiplier chart: This is another chart I put together manually from data from the dashboard and that could be turned into a dynamic panel. This looks at the number of contributors for each release and then divides the number of non-ONF contributors by the number of ONF contributors to create the multiplier number. So a higher multiplier means there were more contributions coming from community members outside of ONF. This is worth tracking to determine how much leverage we’re getting from the community.