Welcome to Radar! This is a project that attempts to quantify and analyze the OStatus Fediverse - the collection of OStatus-compatible communities on the internet.
The main objective of Radar is to provide insight into how big the Fediverse is growing, where the users are, what software they use, and in time, what languages and topics are commonly discussed.
The intention is to eventually monitor all public instances of OStatus-compatible community software. At the moment, all the data being displayed is gathered from Mastodon instances, but more software will be added over time.
For each instance, Radar attempts to identify:
This information is gathered every hour, and all of it is made available through aggregate dashboards, and instance-specific pages.
Radar is a Laravel 5.4 web application, and will eventually use NodeJS for stream subscriptions.
The charts are a combination of d3.js (for more complex visuals) and charts.js (for the chart types it supports out of the box).
Eventually, yes! This is a tricky one, however.
There is no centralized authority that makes the rules for how OStatus instances are meant to be run. This means there is no global policy on how data from those instances are handled. There's always the possibility that some admins may not want their instances tracked, or may not want historical data available to the public.
The plan is to eventually allow instance administrators to "claim" their instances in the Radar database, and give them the option of whether or not to display their instance information in charts, or in JSON API exports.
If you're an instance administrator and you'd like to remove your data from Radar right now, you can message me at https://wogan.im/@wogan from the instance you want removed.
At present there are two means of obtaining data.
A scheduled task runs every hour to visit the "about" pages of all the Mastodon instances in the database, and reads the Users, Statuses and Connection numbers off the page. There's a PR in progress to expose that information via the Instances API instead, which will make this easier.
The next challenge will be scraping GNUSocial/postActiv and StatusNet instances, since they do not neatly provide this information in a standardized way.
A substantial portion of Mastodon servers have their public timelines, well, public - it's possible to read them without authentication. Every 15 minutes, Radar pings as many as it can find, and scans the posts to identify new domains that host OStatus-compatible content.
Radar will eventually run a realtime indexer. The idea is to have instances opt-in to be monitored, and have Radar deploy a bot that authenticates to the instance and subscribes to activity streams. This data would be used to feed a realtime display of trending topics, as well as identify new OStatus domains. This is still quite a long way off though.
There's quite a lot I still want to do:
Share it out, with as many people as you can!
Someday I hope to have easily-boostable stats posted to a dedicated Mastodon account to make that a bit easier.
If you have any questions, comments or complaints, I'm on the Fediverse at https://wogan.im/@wogan