Github CI is not useful for external contributors

[PhilLab]

As I am not an expert with Github CI (I am more familiar with Gitlab), parts of these might be Github's fault, but nonetheless - the CI is not doing any favor to external contributors. IMHO, it should support them better, to motivate more outside contributions

1. Email flood

   • After a lot of quiet on the PR, I am suddenly bombarded with ~13 emails about failing jobs. Especially if I have multiple PRs open, it is quite hard to keep track of what is affected
   • Any chance of having one email per PR instead, reporting the CI failure once, per PR?
   • 

2. It is unclear when the CI runs and where

   • We should explain in the CONTRIBUTING.md that most jobs (but not all, I think?) run only when a maintainer starts them
   • There is also a Drone CI running somewhere?
   • It is unclear where I see my newly added tests actually running
   • I tried to get to the bottom of this here, but didn't get far: Minor improvements for automated testing #15424 (comment)

3. Some jobs simply fail by design, when it is on a fork

   • Example: https://github.com/nextcloud/android/actions/runs/20276837329/job/59030849217?pr=16145
   • this leads to an unnecessary and wrongful impression that there is something wrong. Rather, any CI failure should directly map to an action item - but in these cases, there is nothing you can do.

4. It is unclear what checks should be run locally, to ensure a smooth CI experience

   • E.g. gradlew lint is failing the CI, but it is not explained anywhere that you should run it. The pre-commit hook also doesn't execute it (probably because it is too expensive to run every time)
   • Example: https://github.com/nextcloud/android/actions/runs/20276837368/job/59030849289?pr=16145

5. Details of a CI failure are hard to understand

   • For example no full lint report downloadable on https://github.com/nextcloud/android/actions/runs/20276837368/job/59030849289?pr=16145
   • For example no access to details for screenshot test failures: https://github.com/nextcloud/android/actions/runs/20276837409/job/59030849251?pr=16145

6. Said linter job just always fails, unrelated to the changes in the PR

   • Example: https://github.com/nextcloud/android/actions/runs/20276837368/job/59030849289?pr=16145 fails due to issues in files the PR didn't even touch. So it is unclear, if and what I am supposed to do.

7. The detectWrongSettings job is destined to fail

   • Because the maintainer triggering of the CI job is happening weeks later than the original PR, at time of triggering the PR is almost always out-of-date, so it doesn't use one of the latest 10 android-library commits.
   • This issue is fixable by the contributor, by rebasing, but it does cause additional friction as well (yet another failure, yet another alert, ...)

cc @AndyScherzinger

[AndyScherzinger]

CC @tobiasKaminsky @alperozturk96

[AndyScherzinger]

Added the devs and teamlead.

As a quick fix for you, since you already contributed, sent you an invite for the Nextcloud org on Github, so you can don't need to work on a fork.

I agree that docs can be improve. Several CI checks terminate early when executing on a fork for security reasons to protect the original repo.