Portal:Toolforge/Sustainability Scoring Framework Proposal/Initial assessment
| Category | Subcategory | Max Points / Weight | Scoring Type | Definition of Done | Raw Score | Normalized Score |
|---|---|---|---|---|---|---|
| Observability | Centralized Logging | 15 | Binary | Logs accessible through centralized UI or API. | No | 0 |
| Observability | Metrics Dashboard | 7 | Percent | Basic CPU/memory/request metrics available per tool. | 0 | 0 |
| Observability | Abuse Monitoring | 8 | Binary | Automated detection of crypto mining, resource abuse, etc. | 0 | 0 |
| Platform Access & Deployment | CLI Access | 10 | Binary | CLI access available without SSH. | No | 0 |
| Platform Access & Deployment | WebUI Coverage | 10 | Percent | Percentage of operations that can be performed via Web UI. | 5 | 0.88 |
| Platform Access & Deployment | API Coverage | 10 | Percent | Percentage of Toolforge functions supported by API. | 35 | 6.14 |
| Platform Access & Deployment | SSH-Free Workflow | 10 | Binary | Deployment possible without SSH. | No | 0 |
| Source Code Availability | Policy Enforcement | 5 | Binary | Public source code policy enforced. | No | 0 |
| Source Code Availability | Public Code Availability | 20 | Percent | Percentage of tools with publicly viewable source code and metadata. | 972 | 7.62 |
| Source Code Availability | Automation | 5 | Binary | Automated public source code detection in place. | No | 0 |
| Total | 100 | 14.64 | ||||
The table above presents an initial assessment of Toolforge platform sustainability using the proposed scoring framework. The assessment focuses on three key areas identified through previous admin surveys and platform discussions: Observability, Platform Access & Deployment, and Source Code Availability.
This assessment should be considered a preliminary baseline rather than a definitive measurement. Some metrics are based on currently available platform data, while others required manual estimation and may be refined as the framework matures.
Using the current methodology, Toolforge receives an overall score of 14.64 out of 100 points. The assessment highlights several opportunities for improvement, particularly in observability capabilities, SSH-independent workflows, and automated enforcement of source code availability requirements.
As the framework evolves, individual metrics, scoring criteria, and data collection methods may be refined. The primary purpose of this baseline is to establish a starting point against which future improvements can be measured and tracked over time.
Data Collection Notes
- Public Code Availability was calculated using 972 tools with publicly available source code.
- The denominator used for this assessment was 2,552 Toolforge tools.
- Tool counts were collected in February 2026 and represent a point-in-time snapshot.
- Unless otherwise noted, percentages and scores in this assessment are based on data available at the time of collection.
- Future assessments may differ due to platform growth, tool retirement, policy changes, or improvements in measurement methodology.