Updated documentation and CHANGELOG for v1.0.15 - Ready to launch!

Author: llavarello

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## Release v1.0.15 (September 20th, 2024)
+
+* Added searching for similar repository names in GitHub, Warning if another repository with the same name and better reputation is found.
+* Added commit time analysis, grouping commit hours per contributor and calculating the percentage of commits at each hour. This feature provides insights into contributors' activity patterns and helps identify potential anomalies.
+* Added new Workflows X-Ray module which contains all Workflow-related logic. Moved in some of the logic that was under the Repository x-Ray.
+* Added counts of Workflow Runs to identify when Workflow Runs were DELETED, which may have been the result of an attacker erasing their tracks, or legitimate cleanup.
+* Added a series of basic Workflow security checks which might be an indicator of a vulnerable Workflow.
+* Added to the Workflows X-Ray the ability to print, for each workflow, how many times it was executed by non-contributors as well as contributors.
+* Added to the Workflows X-Ray the ability to parse and print any secret names used in a Workflow.
+* Added a display of Progress % for time consuming queries and a time estimate in seconds-left prior to resuming execution.
+* Added ability to SKIP heavy querying live by handling CTRL+C, which means we've also removed any caps or limits recently introduced.
+* Fixed parsing of dict-formatted results coming from the REST API so that we keep the last key and not the second one.
+* Fixed a few exceptions which arise by hitting CTRL+C and skipping or breaking API calls
+
 ## Release v1.0.14 (September 1st, 2024)
 
 * Added a new check on workflow runs for accounts which are NOT contributors, presenting a WARNING on screen. This could help identify hack attempts via Workflow runs.

docs/code_structure.md

@@ -23,5 +23,5 @@ Finally, last but not least important, the X-Rays under the xrays directory:
 
 * `contributors_xray.py` - Handles all Contributor-related data and decides what to log.
 * `repository_xray.py` - Handles all Repository-related data and decides what to log.
+* `workflows_xray.py` - Handles all Workflow-related analysis and decides what to log.
 * `associations_xray.py` - Analyzes and reports all associations carried from prior X-Ray modules.
-

docs/awesome_features.md docs/features.mddocs/awesome_features.md renamed to docs/features.md

@@ -1,4 +1,4 @@
-# Awesome Features 💫
+# Features 💫
 
 Because of the amount of data it analyzes, `gitxray` can be a bit overwhelming at first. Let's look at a few examples of potential awesome findings which can better explain why you're here and why `gitxray` is awesome ♥. 
 
@@ -35,6 +35,10 @@ gitxray -r https://github.com/SampleOrg/SampleRepo -v -f user_input
 
 Associations MUST NOT be directly and blindly used to report fake or shadow accounts. They are automatic observations from a piece of well-intended code. Do NOT treat association results as findings directly. We must protect open-source projects by first and foremost respecting open-source developers. Ensure that any actions taken are thoughtful and based on solid evidence, not just automated associations. 
 
+## Duplicate Repository Name Check 🛡
+
+`gitxray` will always check and detect duplicate repository names across different organizations and user profiles. This helps identify potential cloned or fake repositories. `gitxray` compares your target repository against others with the same name and the highest star count, ensuring you are engaging with the most popular (and likely legitimate) one.
+
 ## Forensics: What happened on the day of an incident? 🔍
 
 Because `gitxray` collects data from multiple sources of activity including Commits, Comments, Workflow Runs, Issues, Deployments and more; and because Verbose mode in `gitxray` shows activity in a standarized YYYY-MM-DD format, it becomes possible to use Filtering in order to place focus on specific activity happening at a specific point in time.
@@ -46,6 +50,10 @@ gitxray -r https://github.com/SampleOrg/SampleRepo -v -f 2024-08
 gitxray -r https://github.com/SampleOrg/SampleRepo -v -f 2024-09-01
 ```
 
+## Analyzing Commit Hours to Identify Anomalies 🕛
+
+`gitxray` provides a summary of contributor commit hours, allowing you to profile contributor activity and detect potential anomalies. This feature helps you understand typical patterns and flag unusual behavior for further investigation.
+
 ## Untrustworthy Repositories and Activity 🏴
 
 `gitxray` can be used to protect yourself, your team and your customers from fake Repositories and suspicious activity. For example, by analyzing commit dates and comparing them to the account creation timestamp of contributors, `gitxray` can flag inconsistencies that may indicate:
@@ -60,6 +68,22 @@ Although we always recommend running a full unfiltered verbose X-Ray, it is poss
 gitxray -o https://github.com/SampleOrg -v -f warning
 ```
 
+## X-Raying GitHub Workflows ⚙ 
+
+The Workflows X-Ray module is executed upon identifying existing Workflows or Actions. `gitxray` provides in-depth analysis and monitoring of GitHub workflows, including:
+
+* Execution Analysis: Provides detailed insights into workflow execution, showing how many times each workflow was executed by contributors and non-contributors. This allows for better understanding of usage patterns and detection of unauthorized or unexpected activity.
+
+* Detection of deleted runs: This feature helps identify whether workflow runs have been deleted, potentially indicating an attempt to erase traces of malicious activity or legitimate maintenance actions.
+
+* Security Checks for Workflows: Performs a series of basic security checks on workflows to identify uses of Secrets, Self-hosted Runners and Potentially dangerous triggers (eg pull_request_target) that could pose a security risk.
+
+### Disclaimer: Gitxray is NOT a complete Workflow Security Scanner
+
+For more information on tools which are specialized in scanning Workflows refer to our [Vulnerable Workflows section](vulnerable_workflows.md).
+
+
+
 ## The PR Rejection Awards 🏆 
 
 Another `gitxray` feature is the ability to list a TOP 3 of GitHub accounts that have tried to submit Pull Requests to the repository, which ended up closed AND NOT merged. In certain emotional scenarios, this could be paraphrased as _rejected PRs_. Kidding aside, in some cases, this could lead to identifying Contributors who have repeatedly failed at merging a very evidently unaligned piece of code to a branch (I know, it sounds unlikely for an account to try and merge backdoor.py repeatedly... but is it?).
@@ -75,7 +99,7 @@ or targetting a specific Repository with (_Verbose is always optional_):
 ``` bash
 gitxray -r https://github.com/SampleOrg/SampleRepo -v -f contributors
 ```
-## Fake Stars, Private repos gone Public and more 🙈 
+## Fake Starring, Private repos gone Public and more 🙈 
 
 GitHub shares publicly [up to 90 days of past Events](https://docs.github.com/en/rest/activity/events?apiVersion=2022-11-28) for any User account, which include actions such as Repository creation, Watching, Committing, Pull Requesting, and more. `gitxray` summarizes these events for you and prints them out under a `90d_events` category in the results included for each Contributor, summarized in order to reduce the amount of data listed by default. 
 
@@ -98,4 +122,55 @@ And you could then enable _Verbose_ (or before, you decide) and target a specifi
 ``` 
 gitxray -r https://github.com/SampleOrg/SampleRepo -v -c some_user
 ```
+## Lots of e-mail addresses 📧 and Profiling data 👤
+
+`gitxray` will report for each Contributor, an `emails` category listing all unique e-mail address collected from parsing:
+
+* The User's profile
+* Each commit made by the User
+* PGP Primary Keys and PGP SubKeys
+
+Additionally, Personal Information (e.g. social networks) voluntarily made Public by the User is extracted from multiple sources including PGP Key BLOBs and reported under a `personal` category.
+
+Finally, the `profiling` category tends to display information related to the account itself (e.g. creation date, last updated, and more.)
+
+You may focus specifically on `emails`, `personal`, and `profiling` fields with (Verbose is optional):
+```py
+gitxray -o https://github.com/SampleOrg -v -f emails,personal,profiling
+```
+or, for a specific repository, with: 
+``` py
+gitxray -r https://github.com/SampleOrg/SampleRepo -v -f emails,personal,profiling
+```
+
+## Looking out for malicious Releases and Assets 👁 
+
+It is possible for Threat Actors to compromise credentials of a Repository Maintainer in order to deploy malware by silently updating released Assets (the typical package you would download when a Release includes downloadable Assets); which is why `gitxray` looks at all Repository Releases and informs of:
+
+* Assets that were **updated** at least a day **AFTER** their release, which might lead to suggest they've been infected and/or tampered with. Or it could just be a maintainer fixing an asset without wanting to create a new release.
+
+* Users who have historically created releases and/or uploaded assets, as well as the % vs. the total amount of releases or assets uploaded in the repository; which may allow you to flag potential suspicious activity. For example, you might notice an account which never created Releases before now uploading assets.
+
+
+All of this information is included by `gitxray` in a `releases` category, which means you can focus on those results (if any exist) with:
+
+``` bash
+gitxray -o https://github.com/SampleOrg -f releases
+```
+
+## Anonymous contributors 👁 
+
+As stated in [GitHub documentation](https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#list-repository-contributors), only the first 500 author email addresses in a Repository will link to actual GitHub users or accounts. The rest will appear as "anonymous" contributors without associated GitHub information.
+
+Additionally, when an author's email address in a commit is not associated with a GitHub account, the User will also be considered Anonymous.
+
+Lucky for us, `gitxray` also includes within its output the entire list of Anonymous contributors received from GitHub. The list is first processed to combine all variations of Names used by the author for a same e-mail, which means the list can also be pretty useful when, for example, executing OSINT.
+
+To filter for anonymous contributors, you may use:
+``` bash
+gitxray -o https://github.com/SampleOrg -f anonymous
+```
+
+## And so much more.
 
+We've covered a large amount of use-cases for `gitxray`, yet we're nowhere finished. Start X-Raying today your favorite Organizations and Repositories and discover more ways of connecting dots. 

docs/index.md

@@ -7,17 +7,16 @@ The Octocat getting X-Rayed  | [![Build Workflows](https://github.com/kulkansecu
 <div style="clear: both;"></div>
 
 # What is it for?
-* [Finding sensitive information in contributor profiles](/awesome_features/#unintended-disclosures-in-contributor-profiles) disclosed by accident within, for example, Armored PGP Keys, or Key Names.
-* Identifying threat actors in a Repository. [You may spot co-owned or shared accounts](/awesome_features/#spotting-shared-co-owned-or-fake-contributors), as well as inspect public events to [spot fake Stargazers](/awesome_features/#fake-stars-private-repos-gone-public-and-more).
-* Collecting [email addresses and analyzing contributor accounts](/more_features/#lots-of-e-mail-addresses-and-profiling-data) belonging to GitHub organizations and repositories.
-* Identifying fake or infected Repositories. It can [detect tampered commit dates](/awesome_features/#untrustworthy-repositories-and-activity) as well as, for example, [Release assets updated post-release](/more_features/#looking-out-for-malicious-releases-and-assets).
-* Forensics use-cases, such as [finding out what else happened on the day of an Incident](/awesome_features/#forensics-what-happened-on-the-day-of-an-incident).
+* Identifying threat actors in a Repository. [You may spot co-owned or shared accounts](/features/#spotting-shared-co-owned-or-fake-contributors), as well as inspect public events to [spot fake Stargazers](/features/#fake-stars-private-repos-gone-public-and-more).
+* Forensics use-cases, such as [finding out what else happened on the day of an Incident](/features/#forensics-what-happened-on-the-day-of-an-incident).
+* [Finding sensitive information in contributor profiles](/features/#unintended-disclosures-in-contributor-profiles) disclosed by accident within, for example, Armored PGP Keys, or Key Names.
+* Collecting [email addresses and analyzing contributor accounts](/features/#lots-of-e-mail-addresses-and-profiling-data) belonging to GitHub organizations and repositories.
+* Identifying fake or infected Repositories. It can [detect tampered commit dates](/features/#untrustworthy-repositories-and-activity) as well as, for example, [Release assets updated post-release](/features/#looking-out-for-malicious-releases-and-assets).
 * And so. much. more.
 
 # Getting started
 * [Installing Gitxray](installing.md)
-* [Awesome Features](awesome_features.md) &#128171;
-* [More Features](more_features.md) &#129470;
+* [Features](features.md) &#128171;
 
 ## Rate Limits and the GitHub API
 

docs/more_features.md

@@ -0,0 +1,32 @@
+# Vulnerable Workflows
+
+You may have landed here because Git X-Ray suggested that you further inspect a specific Workflow in a repository that you were X-Raying, or because of some other reason.
+
+Either way, here's a list of specialized software and documentation on how to proceed with analyzing the security of your workflow.
+
+# Tools for Workflow analysis
+
+* [https://github.com/synacktiv/octoscan](https://github.com/synacktiv/octoscan) - A SAST tool for GitHub Workflows.
+
+* [https://github.com/AdnaneKhan/Gato-X](https://github.com/AdnaneKhan/Gato-X) - Excellent for identifying at scale vulnerable Workflows.
+
+* [https://semgrep.dev/p/github-actions](https://semgrep.dev/p/github-actions) - Semgrep rules for GitHub Workflows.
+
+* [https://github.com/tindersec/gh-workflow-auditor](https://github.com/tindersec/gh-workflow-auditor) - A script by Tinder Security which analyzes multiple aspects of a Workflow.
+
+# Articles about Github Workflows and Security
+
+## Official by GitHub
+
+* [https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/](https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/)
+* [https://securitylab.github.com/resources/github-actions-untrusted-input/](https://securitylab.github.com/resources/github-actions-untrusted-input/)
+* [https://securitylab.github.com/resources/github-actions-building-blocks/](https://securitylab.github.com/resources/github-actions-building-blocks/)
+* [https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions](https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions)
+
+## By independent researchers and organizations
+
+* [https://medium.com/tinder/exploiting-github-actions-on-open-source-projects-5d93936d189f](https://medium.com/tinder/exploiting-github-actions-on-open-source-projects-5d93936d189f)
+
+# Videos 
+
+* [https://www.youtube.com/watch?v=Ers-LcA7Nmc](https://www.youtube.com/watch?v=Ers-LcA7Nmc) - A great video and slides by Rob Bos on GitHub Actions with security in mind

docs/vulnerable_workflows.md

@@ -10,8 +10,9 @@ theme:
 
 nav:
   - 'installing.md'
-  - 'awesome_features.md'
-  - 'more_features.md'
+  - 'features.md'
+  - 'vulnerable_workflows.md'
+  - 'code_structure.md'
 
 markdown_extensions:
   - toc: